[Gluster-devel] trash.t failure with brick multiplexing [Was Re: Build failed in Jenkins: regression-test-with-multiplex #574]

Milind Changire mchangir at redhat.com
Tue Jan 2 12:02:19 UTC 2018 

On Tue, Jan 2, 2018 at 10:44 AM, Atin Mukherjee <amukherj at redhat.com> wrote:

>
>
> On Thu, Dec 21, 2017 at 7:27 PM, Atin Mukherjee <amukherj at redhat.com>
> wrote:
>
>>
>>
>> On Wed, Dec 20, 2017 at 11:58 AM, Atin Mukherjee <amukherj at redhat.com>
>> wrote:
>>
>>> ./tests/bugs/glusterd/bug-1230121-replica_subvol_count_correct_cal.t
>>>
>>
>> Unfortunately the above is passing in my setup. I'll be checking the logs
>> to see if I can figure out the issue.
>>
>> ./tests/features/trash.t
>>>
>>
>> Rebalance fails here consistently when brick mux is enabled with the
>> following message:
>>
>> [2017-12-21 13:55:31.881268] I [MSGID: 109081]
>> [dht-common.c:5538:dht_setxattr] 0-patchy-dht: fixing the layout of /
>> [2017-12-21 13:55:31.881289] W [MSGID: 109016]
>> [dht-selfheal.c:1930:dht_fix_layout_of_directory] 0-patchy-dht: Layout
>> fix failed: 1 subvolume(s) are down. Skipping fix layout. path:/
>> gfid:00000000-0000-0000-0000-000000000001
>> [2017-12-21 13:55:31.881525] E [MSGID: 109026]
>> [dht-rebalance.c:4444:gf_defrag_start_crawl] 0-patchy-dht: fix layout on
>> / failed [Transport endpoint is not connected]
>>
>> When I revert the commit 56e5fda "rpc: merge ssl infra with epoll infra"
>> the test passes through.
>> I tend to believe that the other failures especially ssl-ciphers.t
>> <http://git.gluster.org/cgit/glusterfs.git/tree/tests/features/ssl-ciphers.t>
>> from fstat.gluster.log could be due to the same patch.
>>
>> @Milind - Need your attention here.
>>
>
> Since the test failures are constant, I propose to revert the commit
> 56e5fda from mainline till these issues are looked at.
>

After setting the SSL CRL path to NULL, if the volume is restarted, the
test tests/features/ssl-ciphers.t passes in multiplex mode.
SSL socket options aren't handled in the reconfigure() entry-point for the
socket transport.
I'll post a patch soon to fix the test case.


>
>>
>>> The above two are new failures since day before yesterday. Job link is
>>> at https://build.gluster.org/job/regression-test-with-multiplex
>>> /574/consoleFull .
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: <jenkins at build.gluster.org>
>>> Date: Wed, Dec 20, 2017 at 12:24 AM
>>> Subject: Build failed in Jenkins: regression-test-with-multiplex #574
>>> To: maintainers at gluster.org, amukherj at redhat.com, jeff at pl.atyp.us,
>>> jahernan at redhat.com, jdarcy at fb.com, kdhananj at redhat.com,
>>> rgowdapp at redhat.com, khiremat at redhat.com, amarts at redhat.com,
>>> nbalacha at redhat.com, nigelb at redhat.com, srangana at redhat.com
>>>
>>>
>>> See <https://build.gluster.org/job/regression-test-with-multiple
>>> x/574/display/redirect?page=changes>
>>>
>>> Changes:
>>>
>>> [Jeff Darcy] protocol/server: add dump_metrics method
>>>
>>> [Jeff Darcy] snapshot: Fix several coverity issues in
>>> glusterd-snapshot-utils.c
>>>
>>> [Kotresh H R] feature/bitrot: remove internal xattrs from lookup cbk
>>>
>>> ------------------------------------------
>>> [...truncated 784.50 KB...]
>>> ./tests/bugs/nfs/bug-904065.t  -  9 second
>>> ./tests/bugs/nfs/bug-1157223-symlink-mounting.t  -  9 second
>>> ./tests/bugs/md-cache/bug-1211863.t  -  9 second
>>> ./tests/bugs/glusterd/bug-949930.t  -  9 second
>>> ./tests/bugs/glusterd/bug-1420637-volume-sync-fix.t  -  9 second
>>> ./tests/bugs/glusterd/bug-1121584-brick-existing-validation-
>>> for-remove-brick-status-stop.t  -  9 second
>>> ./tests/bugs/glusterd/bug-1104642.t  -  9 second
>>> ./tests/bugs/distribute/bug-961615.t  -  9 second
>>> ./tests/bugs/distribute/bug-1247563.t  -  9 second
>>> ./tests/bugs/distribute/bug-1086228.t  -  9 second
>>> ./tests/bugs/cli/bug-1087487.t  -  9 second
>>> ./tests/bugs/bitrot/1209752-volume-status-should-show-bitrot-scrub-info.t
>>> -  9 second
>>> ./tests/basic/tier/ctr-rename-overwrite.t  -  9 second
>>> ./tests/basic/stats-dump.t  -  9 second
>>> ./tests/basic/quota_aux_mount.t  -  9 second
>>> ./tests/basic/inode-quota-enforcing.t  -  9 second
>>> ./tests/basic/fop-sampling.t  -  9 second
>>> ./tests/gfid2path/get-gfid-to-path.t  -  8 second
>>> ./tests/bugs/upcall/bug-1227204.t  -  8 second
>>> ./tests/bugs/replicate/bug-1448804-check-quorum-type-values.t  -  8
>>> second
>>> ./tests/bugs/glusterfs/bug-902610.t  -  8 second
>>> ./tests/bugs/glusterd/bug-889630.t  -  8 second
>>> ./tests/bugs/glusterd/bug-859927.t  -  8 second
>>> ./tests/bugs/glusterd/bug-1323287-real_path-handshake-test.t  -  8
>>> second
>>> ./tests/bugs/glusterd/bug-1213295-snapd-svc-uninitialized.t  -  8 second
>>> ./tests/bugs/glusterd/bug-1109741-auth-mgmt-handshake.t  -  8 second
>>> ./tests/bugs/glusterd/bug-1046308.t  -  8 second
>>> ./tests/bugs/ec/bug-1179050.t  -  8 second
>>> ./tests/bugs/distribute/bug-1122443.t  -  8 second
>>> ./tests/bugs/distribute/bug-1088231.t  -  8 second
>>> ./tests/bugs/changelog/bug-1208470.t  -  8 second
>>> ./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t  -
>>> 8 second
>>> ./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t
>>> -  8 second
>>> ./tests/bitrot/br-stub.t  -  8 second
>>> ./tests/basic/volume-status.t  -  8 second
>>> ./tests/basic/quota-nfs.t  -  8 second
>>> ./tests/basic/glusterd/arbiter-volume-probe.t  -  8 second
>>> ./tests/basic/ec/ec-anonymous-fd.t  -  8 second
>>> ./tests/gfid2path/block-mount-access.t  -  7 second
>>> ./tests/features/ssl-authz.t  -  7 second
>>> ./tests/bugs/upcall/bug-1458127.t  -  7 second
>>> ./tests/bugs/tier/bug-1205545-CTR-and-trash-integration.t  -  7 second
>>> ./tests/bugs/snapshot/bug-1260848.t  -  7 second
>>> ./tests/bugs/replicate/bug-767585-gfid.t  -  7 second
>>> ./tests/bugs/replicate/bug-1101647.t  -  7 second
>>> ./tests/bugs/quota/bug-1243798.t  -  7 second
>>> ./tests/bugs/nfs/bug-1143880-fix-gNFSd-auth-crash.t  -  7 second
>>> ./tests/bugs/glusterfs/bug-856455.t  -  7 second
>>> ./tests/bugs/glusterd/bug-948729/bug-948729-force.t  -  7 second
>>> ./tests/bugs/glusterd/bug-1482906-peer-file-blank-line.t  -  7 second
>>> ./tests/bugs/glusterd/bug-1242875-do-not-pass-volinfo-quota.t  -  7
>>> second
>>> ./tests/bugs/glusterd/bug-1223213-peerid-fix.t  -  7 second
>>> ./tests/bugs/glusterd/bug-1179175-uss-option-validation.t  -  7 second
>>> ./tests/bugs/glusterd/bug-1094119-remove-replace-brick-support-from-glusterd.t
>>> -  7 second
>>> ./tests/bugs/ec/bug-1227869.t  -  7 second
>>> ./tests/bugs/distribute/bug-1368012.t  -  7 second
>>> ./tests/bugs/core/bug-986429.t  -  7 second
>>> ./tests/bugs/bug-1258069.t  -  7 second
>>> ./tests/bugs/bitrot/bug-1229134-bitd-not-support-vol-set.t  -  7 second
>>> ./tests/basic/ec/ec-read-policy.t  -  7 second
>>> ./tests/basic/distribute/bug-1265677-use-readdirp.t  -  7 second
>>> ./tests/basic/afr/heal-info.t  -  7 second
>>> ./tests/basic/afr/gfid-heal.t  -  7 second
>>> ./tests/basic/afr/arbiter-remove-brick.t  -  7 second
>>> ./tests/bugs/upcall/bug-upcall-stat.t  -  6 second
>>> ./tests/bugs/upcall/bug-1369430.t  -  6 second
>>> ./tests/bugs/snapshot/bug-1178079.t  -  6 second
>>> ./tests/bugs/snapshot/bug-1064768.t  -  6 second
>>> ./tests/bugs/shard/bug-1258334.t  -  6 second
>>> ./tests/bugs/shard/bug-1256580.t  -  6 second
>>> ./tests/bugs/rpc/bug-954057.t  -  6 second
>>> ./tests/bugs/replicate/bug-976800.t  -  6 second
>>> ./tests/bugs/replicate/bug-966018.t  -  6 second
>>> ./tests/bugs/replicate/bug-1365455.t  -  6 second
>>> ./tests/bugs/quota/bug-1287996.t  -  6 second
>>> ./tests/bugs/quota/bug-1104692.t  -  6 second
>>> ./tests/bugs/nfs/bug-915280.t  -  6 second
>>> ./tests/bugs/nfs/bug-877885.t  -  6 second
>>> ./tests/bugs/nfs/bug-1116503.t  -  6 second
>>> ./tests/bugs/md-cache/bug-1211863_unlink.t  -  6 second
>>> ./tests/bugs/md-cache/afr-stale-read.t  -  6 second
>>> ./tests/bugs/io-cache/bug-read-hang.t  -  6 second
>>> ./tests/bugs/io-cache/bug-858242.t  -  6 second
>>> ./tests/bugs/glusterfs-server/bug-873549.t  -  6 second
>>> ./tests/bugs/glusterfs/bug-895235.t  -  6 second
>>> ./tests/bugs/glusterd/bug-948729/bug-948729.t  -  6 second
>>> ./tests/bugs/glusterd/bug-948729/bug-948729-mode-script.t  -  6 second
>>> ./tests/bugs/glusterd/bug-888752.t  -  6 second
>>> ./tests/bugs/glusterd/bug-1499509-disconnect-in-brick-mux.t  -  6 second
>>> ./tests/bugs/glusterd/bug-1102656.t  -  6 second
>>> ./tests/bugs/distribute/bug-884597.t  -  6 second
>>> ./tests/bugs/core/bug-913544.t  -  6 second
>>> ./tests/bugs/core/bug-908146.t  -  6 second
>>> ./tests/bugs/core/bug-834465.t  -  6 second
>>> ./tests/bugs/bug-1371806_2.t  -  6 second
>>> ./tests/bugs/bug-1371806_1.t  -  6 second
>>> ./tests/bugs/bitrot/bug-1210684-scrub-pause-resume-error-handling.t  -
>>> 6 second
>>> ./tests/bitrot/bug-1221914.t  -  6 second
>>> ./tests/basic/nl-cache.t  -  6 second
>>> ./tests/basic/hardlink-limit.t  -  6 second
>>> ./tests/basic/ec/nfs.t  -  6 second
>>> ./tests/basic/ec/ec-internal-xattrs.t  -  6 second
>>> ./tests/basic/ec/ec-fallocate.t  -  6 second
>>> ./tests/basic/ec/dht-rename.t  -  6 second
>>> ./tests/performance/quick-read.t  -  5 second
>>> ./tests/gfid2path/gfid2path_nfs.t  -  5 second
>>> ./tests/features/delay-gen.t  -  5 second
>>> ./tests/bugs/unclassified/bug-1034085.t  -  5 second
>>> ./tests/bugs/snapshot/bug-1111041.t  -  5 second
>>> ./tests/bugs/shard/bug-1342298.t  -  5 second
>>> ./tests/bugs/shard/bug-1272986.t  -  5 second
>>> ./tests/bugs/shard/bug-1259651.t  -  5 second
>>> ./tests/bugs/shard/bug-1250855.t  -  5 second
>>> ./tests/bugs/replicate/bug-886998.t  -  5 second
>>> ./tests/bugs/replicate/bug-880898.t  -  5 second
>>> ./tests/bugs/replicate/bug-1480525.t  -  5 second
>>> ./tests/bugs/readdir-ahead/bug-1439640.t  -  5 second
>>> ./tests/bugs/posix/bug-765380.t  -  5 second
>>> ./tests/bugs/nl-cache/bug-1451588.t  -  5 second
>>> ./tests/bugs/nfs/zero-atime.t  -  5 second
>>> ./tests/bugs/nfs/subdir-trailing-slash.t  -  5 second
>>> ./tests/bugs/nfs/bug-847622.t  -  5 second
>>> ./tests/bugs/nfs/bug-1210338.t  -  5 second
>>> ./tests/bugs/glusterfs-server/bug-864222.t  -  5 second
>>> ./tests/bugs/glusterfs/bug-893378.t  -  5 second
>>> ./tests/bugs/glusterfs/bug-848251.t  -  5 second
>>> ./tests/bugs/glusterfs/bug-1482528.t  -  5 second
>>> ./tests/bugs/glusterd/bug-839595.t  -  5 second
>>> ./tests/bugs/glusterd/bug-1454418-seg-fault.t  -  5 second
>>> ./tests/bugs/glusterd/bug-1446172-brick-mux-reset-brick.t  -  5 second
>>> ./tests/bugs/glusterd/bug-1163108-min-free-disk-option-validation.t  -
>>> 5 second
>>> ./tests/bugs/glusterd/bug-1091935-brick-order-check-from-cli-to-glusterd.t
>>> -  5 second
>>> ./tests/bugs/glusterd/bug-1022055.t  -  5 second
>>> ./tests/bugs/geo-replication/bug-1296496.t  -  5 second
>>> ./tests/bugs/distribute/bug-912564.t  -  5 second
>>> ./tests/bugs/distribute/bug-907072.t  -  5 second
>>> ./tests/bugs/core/io-stats-1322825.t  -  5 second
>>> ./tests/bugs/core/bug-1168803-snapd-option-validation-fix.t  -  5 second
>>> ./tests/bugs/core/949327.t  -  5 second
>>> ./tests/bugs/cli/bug-961307.t  -  5 second
>>> ./tests/bugs/access-control/bug-1387241.t  -  5 second
>>> ./tests/bugs/access-control/bug-1051896.t  -  5 second
>>> ./tests/basic/gfapi/glfs_sysrq.t  -  5 second
>>> ./tests/basic/gfapi/gfapi-load-volfile.t  -  5 second
>>> ./tests/features/leases.t  -  4 second
>>> ./tests/bugs/upcall/bug-1422776.t  -  4 second
>>> ./tests/bugs/upcall/bug-1394131.t  -  4 second
>>> ./tests/bugs/trace/bug-797171.t  -  4 second
>>> ./tests/bugs/shard/bug-1245547.t  -  4 second
>>> ./tests/bugs/read-only/bug-1134822-read-only-default-in-graph.t  -  4
>>> second
>>> ./tests/bugs/readdir-ahead/bug-1446516.t  -  4 second
>>> ./tests/bugs/quick-read/bug-846240.t  -  4 second
>>> ./tests/bugs/posix/disallow-gfid-volumeid-removexattr.t  -  4 second
>>> ./tests/bugs/nfs/showmount-many-clients.t  -  4 second
>>> ./tests/bugs/nfs/bug-1166862.t  -  4 second
>>> ./tests/bugs/nfs/bug-1161092-nfs-acls.t  -  4 second
>>> ./tests/bugs/md-cache/bug-1476324.t  -  4 second
>>> ./tests/bugs/glusterfs-server/bug-861542.t  -  4 second
>>> ./tests/bugs/glusterfs/bug-869724.t  -  4 second
>>> ./tests/bugs/glusterfs/bug-844688.t  -  4 second
>>> ./tests/bugs/glusterd/bug-905307.t  -  4 second
>>> ./tests/bugs/glusterd/bug-878004.t  -  4 second
>>> ./tests/bugs/glusterd/bug-1344407-volume-delete-on-node-down.t  -  4
>>> second
>>> ./tests/bugs/glusterd/bug-1266818-shared-storage-disable.t  -  4 second
>>> ./tests/bugs/glusterd/bug-1095097.t  -  4 second
>>> ./tests/bugs/glusterd/bug-1092841.t  -  4 second
>>> ./tests/bugs/fuse/bug-1309462.t  -  4 second
>>> ./tests/bugs/fuse/bug-1283103.t  -  4 second
>>> ./tests/bugs/fuse/bug-1126048.t  -  4 second
>>> ./tests/bugs/distribute/bug-1204140.t  -  4 second
>>> ./tests/bugs/core/bug-1119582.t  -  4 second
>>> ./tests/bugs/cli/bug-1004218.t  -  4 second
>>> ./tests/bitrot/bug-internal-xattrs-check-1243391.t  -  4 second
>>> ./tests/basic/quota-rename.t  -  4 second
>>> ./tests/basic/gfapi/gfapi-trunc.t  -  4 second
>>> ./tests/basic/gfapi/gfapi-async-calls-test.t  -  4 second
>>> ./tests/gfid2path/gfid2path_fuse.t  -  3 second
>>> ./tests/bugs/unclassified/bug-991622.t  -  3 second
>>> ./tests/bugs/shard/bug-1261773.t  -  3 second
>>> ./tests/bugs/replicate/bug-884328.t  -  3 second
>>> ./tests/bugs/readdir-ahead/bug-1512437.t  -  3 second
>>> ./tests/bugs/posix/disallow-gfid-volumeid-fremovexattr.t  -  3 second
>>> ./tests/bugs/nfs/socket-as-fifo.t  -  3 second
>>> ./tests/bugs/nfs/bug-970070.t  -  3 second
>>> ./tests/bugs/nfs/bug-1302948.t  -  3 second
>>> ./tests/bugs/logging/bug-823081.t  -  3 second
>>> ./tests/bugs/glusterfs-server/bug-889996.t  -  3 second
>>> ./tests/bugs/glusterfs-server/bug-877992.t  -  3 second
>>> ./tests/bugs/glusterfs/bug-892730.t  -  3 second
>>> ./tests/bugs/glusterfs/bug-860297.t  -  3 second
>>> ./tests/bugs/glusterfs/bug-811493.t  -  3 second
>>> ./tests/bugs/glusterd/bug-958790.t  -  3 second
>>> ./tests/bugs/glusterd/bug-862834.t  -  3 second
>>> ./tests/bugs/glusterd/bug-1482344-volume-option-set-cluster-level.t  -
>>> 3 second
>>> ./tests/bugs/glusterd/bug-1321836-fix-opret-for-volume-info-xml-output.t
>>> -  3 second
>>> ./tests/bugs/glusterd/bug-1314649-group-virt.t  -  3 second
>>> ./tests/bugs/glusterd/bug-1238135-lazy-daemon-initialization-on-demand.t
>>> -  3 second
>>> ./tests/bugs/glusterd/bug-1199451-op-version-retrieving-fix.t  -  3
>>> second
>>> ./tests/bugs/glusterd/bug-1085330.t  -  3 second
>>> ./tests/bugs/fuse/bug-1336818.t  -  3 second
>>> ./tests/bugs/distribute/bug-924265.t  -  3 second
>>> ./tests/bugs/core/bug-903336.t  -  3 second
>>> ./tests/bugs/core/bug-845213.t  -  3 second
>>> ./tests/bugs/core/bug-1421721-mpx-toggle.t  -  3 second
>>> ./tests/bugs/core/bug-1135514-allow-setxattr-with-null-value.t  -  3
>>> second
>>> ./tests/bugs/core/bug-1117951.t  -  3 second
>>> ./tests/bugs/cli/bug-983317-volume-get.t  -  3 second
>>> ./tests/bugs/cli/bug-977246.t  -  3 second
>>> ./tests/bugs/cli/bug-921215.t  -  3 second
>>> ./tests/bugs/cli/bug-867252.t  -  3 second
>>> ./tests/bugs/cli/bug-1378842-volume-get-all.t  -  3 second
>>> ./tests/basic/md-cache/bug-1418249.t  -  3 second
>>> ./tests/basic/fops-sanity.t  -  3 second
>>> ./tests/bugs/glusterfs/bug-853690.t  -  2 second
>>> ./tests/bugs/glusterd/bug-916549.t  -  2 second
>>> ./tests/bugs/glusterd/bug-913487.t  -  2 second
>>> ./tests/bugs/glusterd/bug-1433578-invalid-peer-glusterd-crash.t  -  2
>>> second
>>> ./tests/bugs/glusterd/bug-1315186-reject-lowering-down-op-version.t  -
>>> 2 second
>>> ./tests/bugs/glusterd/bug-000000.t  -  2 second
>>> ./tests/bugs/core/log-bug-1362520.t  -  2 second
>>> ./tests/bugs/core/bug-924075.t  -  2 second
>>> ./tests/bugs/core/bug-1111557.t  -  2 second
>>> ./tests/bugs/cli/bug-969193.t  -  2 second
>>> ./tests/bugs/cli/bug-949298.t  -  2 second
>>> ./tests/bugs/cli/bug-764638.t  -  2 second
>>> ./tests/bugs/cli/bug-1047378.t  -  2 second
>>> ./tests/basic/peer-parsing.t  -  2 second
>>> ./tests/basic/afr/arbiter-cli.t  -  2 second
>>> ./tests/features/glupy.t  -  1 second
>>> ./tests/basic/gfapi/sink.t  -  1 second
>>> ./tests/bugs/shard/zero-flag.t  -  0 second
>>> ./tests/basic/posixonly.t  -  0 second
>>> ./tests/basic/netgroup_parsing.t  -  0 second
>>> ./tests/basic/first-test.t  -  0 second
>>> ./tests/basic/exports_parsing.t  -  0 second
>>>
>>> 3 test(s) failed
>>> ./tests/basic/tier/new-tier-cmds.t
>>> ./tests/bugs/glusterd/bug-1230121-replica_subvol_count_correct_cal.t
>>> ./tests/features/trash.t
>>>
>>> 0 test(s) generated core
>>>
>>>
>>> Result is 1
>>>
>>> tar: Removing leading `/' from member names
>>> kernel.core_pattern = /%e-%p.core
>>> Build step 'Execute shell' marked build as failure
>>>
>>>
>>
>


-- 
Milind
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20180102/7278f324/attachment-0001.html>

More information about the Gluster-devel mailing list