[Gluster-devel] bug-1421590-brick-mux-reuse-ports.t failure
Amar Tumballi
atumball at redhat.com
Wed Apr 12 06:58:35 UTC 2017
On Wed, Apr 12, 2017 at 12:07 PM, Atin Mukherjee <amukherj at redhat.com>
wrote:
> As per http://fstat.rht.gluster.org/weeks/1 the test in $Subject has
> failed multiple times and is now blocking most of the patches to pass the
> regression. I have a patch https://review.gluster.org/#/c/17033/ to
> remove this test entirely and I have the reason in the commit message.
>
> Can this patch be reviewed and merged asap?
>
+1. This is stopping us from many patches which could have passed otherwise.
Regards,
Amar
>
> ---------- Forwarded message ----------
> From: Atin Mukherjee <amukherj at redhat.com>
> Date: Mon, Apr 10, 2017 at 2:57 PM
> Subject: Re: [Gluster-Maintainers] Build failed in Jenkins:
> regression-test-burn-in #2932
> To: GlusterFS Maintainers <maintainers at gluster.org>, Jeff Darcy <
> jeff at pl.atyp.us>
>
>
> bug-1421590-brick-mux-reuse-ports.t seems to be a bad test to me and here
> is my reasoning:
>
> This test tries to check if the ports are reused or not. When a volume is
> restarted, by the time glusterd tries to allocate a new port to the one of
> the brick processes of the volume there is no guarantee that the older port
> will be allocated given the kernel might take some extra time to free up
> the port between this time frame. From https://build.gluster.org/job/
> regression-test-burn-in/2932/console we can clearly see that post restart
> of the volume, glusterd allocated port 49153 & 49155 for brick1 & brick2
> respectively but the test was expecting the ports to be matched with 49155
> & 49156 which were allocated before the volume was restarted.
>
> @Jeff - Is there any specific reason we want to keep this test running?
>
>
> On Sat, Apr 8, 2017 at 8:12 AM, Atin Mukherjee <amukherj at redhat.com>
> wrote:
>
>>
>> On Sat, 8 Apr 2017 at 08:06, <jenkins at build.gluster.org> wrote:
>>
>>> See <http://build.gluster.org/job/regression-test-burn-in/2932/d
>>> isplay/redirect>
>>>
>>> ------------------------------------------
>>> [...truncated 12020 lines...]
>>> ok 5, LINENUM:32
>>> ok 6, LINENUM:33
>>> ok 7, LINENUM:35
>>> not ok 8 , LINENUM:37
>>> FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
>>> ok 9, LINENUM:38
>>> not ok 10 , LINENUM:40
>>> FAILED COMMAND: gluster --mode=script --wignore volume start patchy
>>> not ok 11 Got "" instead of "49152", LINENUM:42
>>> FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
>>> not ok 12 , LINENUM:47
>>> FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
>>> not ok 13 , LINENUM:48
>>> FAILED COMMAND: gluster --mode=script --wignore volume start patchy
>>> not ok 14 Got "" instead of "49152", LINENUM:50
>>> FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
>>> not ok 15 Got "" instead of "get_nth_brick_port_for_volume", LINENUM:51
>>> FAILED COMMAND: get_nth_brick_port_for_volume patchy 2
>>> not ok 16 , LINENUM:53
>>> FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
>>> ok 17, LINENUM:55
>>> not ok 18 , LINENUM:57
>>> FAILED COMMAND: gluster --mode=script --wignore volume start patchy
>>> not ok 19 Got "" instead of "49152", LINENUM:59
>>> FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
>>> volume set: success
>>> Failed 10/19 subtests
>>>
>>> Test Summary Report
>>> -------------------
>>> ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t (Wstat: 0 Tests:
>>> 19 Failed: 10)
>>> Failed tests: 8, 10-16, 18-19
>>> Files=1, Tests=19, 249 wallclock secs ( 0.03 usr 0.01 sys + 13.40 cusr
>>> 3.36 csys = 16.80 CPU)
>>> Result: FAIL
>>> End of test ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t
>>
>>
>> Something is wrong with this test, have seen it failing in many
>> regression test burns. I'll take a look at it.
>>
>>
>>> ============================================================
>>> ====================
>>>
>>>
>>> Run complete
>>> ============================================================
>>> ====================
>>> Number of tests found: 199
>>> Number of tests selected for run based on pattern: 199
>>> Number of tests skipped as they were marked bad: 7
>>> Number of tests skipped because of known_issues: 4
>>> Number of tests that were run: 188
>>>
>>> 1 test(s) failed
>>> ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t
>>>
>>> 0 test(s) generated core
>>>
>>>
>>> Tests ordered by time taken, slowest to fastest:
>>> ============================================================
>>> ====================
>>> ./tests/basic/ec/ec-12-4.t - 336 second
>>> ./tests/basic/ec/ec-7-3.t - 199 second
>>> ./tests/basic/ec/ec-6-2.t - 178 second
>>> ./tests/basic/ec/self-heal.t - 158 second
>>> ./tests/basic/afr/split-brain-favorite-child-policy.t - 151 second
>>> ./tests/basic/ec/ec-5-2.t - 150 second
>>> ./tests/basic/ec/ec-5-1.t - 150 second
>>> ./tests/basic/afr/entry-self-heal.t - 150 second
>>> ./tests/basic/afr/self-heal.t - 137 second
>>> ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t - 131 second
>>> ./tests/basic/tier/legacy-many.t - 127 second
>>> ./tests/basic/tier/tier.t - 126 second
>>> ./tests/basic/ec/ec-4-1.t - 123 second
>>> ./tests/basic/ec/ec-optimistic-changelog.t - 111 second
>>> ./tests/basic/afr/self-heald.t - 109 second
>>> ./tests/basic/ec/ec-3-1.t - 95 second
>>> ./tests/basic/volume-snapshot-clone.t - 88 second
>>> ./tests/bugs/core/bug-1110917.t - 85 second
>>> ./tests/basic/tier/new-tier-cmds.t - 83 second
>>> ./tests/basic/afr/split-brain-heal-info.t - 82 second
>>> ./tests/bugs/cli/bug-1320388.t - 81 second
>>> ./tests/basic/ec/heal-info.t - 80 second
>>> ./tests/basic/ec/ec-new-entry.t - 79 second
>>> ./tests/basic/ec/ec-background-heals.t - 77 second
>>> ./tests/basic/afr/split-brain-healing.t - 76 second
>>> ./tests/basic/afr/metadata-self-heal.t - 74 second
>>> ./tests/basic/ec/ec-notify.t - 73 second
>>> ./tests/basic/afr/sparse-file-self-heal.t - 73 second
>>> ./tests/basic/afr/granular-esh/cli.t - 69 second
>>> ./tests/basic/quota.t - 67 second
>>> ./tests/bugs/bug-1368312.t - 63 second
>>> ./tests/basic/uss.t - 52 second
>>> ./tests/basic/tier/frequency-counters.t - 52 second
>>> ./tests/bugs/cli/bug-770655.t - 50 second
>>> ./tests/basic/volume-snapshot.t - 49 second
>>> ./tests/basic/afr/quorum.t - 48 second
>>> ./tests/basic/tier/fops-during-migration-pause.t - 47 second
>>> ./tests/basic/mount-nfs-auth.t - 46 second
>>> ./tests/basic/ec/ec.t - 46 second
>>> ./tests/basic/ec/ec-readdir.t - 43 second
>>> ./tests/basic/afr/inodelk.t - 40 second
>>> ./tests/basic/mpx-compat.t - 39 second
>>> ./tests/basic/afr/arbiter.t - 38 second
>>> ./tests/basic/ec/ec-cpu-extensions.t - 37 second
>>> ./tests/basic/tier/locked_file_migration.t - 36 second
>>> ./tests/bitrot/br-state-check.t - 35 second
>>> ./tests/basic/tier/tier-heald.t - 35 second
>>> ./tests/bitrot/bug-1294786.t - 34 second
>>> ./tests/basic/volume-snapshot-xml.t - 33 second
>>> ./tests/basic/mgmt_v3-locks.t - 33 second
>>> ./tests/basic/tier/unlink-during-migration.t - 32 second
>>> ./tests/basic/afr/granular-esh/conservative-merge.t - 32 second
>>> ./tests/basic/afr/gfid-self-heal.t - 31 second
>>> ./tests/basic/geo-replication/marker-xattrs.t - 29 second
>>> ./tests/basic/quota-ancestry-building.t - 27 second
>>> ./tests/bugs/cli/bug-1353156-get-state-cli-validations.t - 26 second
>>> ./tests/basic/afr/arbiter-add-brick.t - 26 second
>>> ./tests/bugs/core/bug-1402841.t-mt-dir-scan-race.t - 25 second
>>> ./tests/basic/afr/heal-quota.t - 25 second
>>> ./tests/basic/afr/data-self-heal.t - 25 second
>>> ./tests/basic/afr/arbiter-mount.t - 24 second
>>> ./tests/bugs/bitrot/bug-1227996.t - 23 second
>>> ./tests/bitrot/bug-1373520.t - 23 second
>>> ./tests/basic/nufa.t - 23 second
>>> ./tests/basic/afr/durability-off.t - 23 second
>>> ./tests/basic/mount.t - 22 second
>>> ./tests/basic/gfapi/gfapi-ssl-test.t - 22 second
>>> ./tests/basic/ec/quota.t - 22 second
>>> ./tests/basic/afr/replace-brick-self-heal.t - 22 second
>>> ./tests/basic/afr/granular-esh/replace-brick.t - 22 second
>>> ./tests/basic/op_errnos.t - 21 second
>>> ./tests/basic/glusterd/volfile_server_switch.t - 21 second
>>> ./tests/bugs/bitrot/bug-1245981.t - 20 second
>>> ./tests/basic/afr/granular-esh/granular-indices-but-non-granular-heal.t
>>> - 20 second
>>> ./tests/basic/afr/granular-esh/granular-esh.t - 20 second
>>> ./tests/basic/0symbol-check.t - 20 second
>>> ./tests/bugs/cli/bug-1113476.t - 19 second
>>> ./tests/bugs/changelog/bug-1225542.t - 19 second
>>> ./tests/bugs/bitrot/bug-1288490.t - 19 second
>>> ./tests/basic/tier/file_with_spaces.t - 19 second
>>> ./tests/basic/glusterd/heald.t - 19 second
>>> ./tests/basic/afr/client-side-heal.t - 19 second
>>> ./tests/basic/afr/add-brick-self-heal.t - 19 second
>>> ./tests/bugs/cli/bug-1077682.t - 18 second
>>> ./tests/bugs/bitrot/1209751-bitrot-scrub-tunable-reset.t - 18 second
>>> ./tests/basic/ec/statedump.t - 18 second
>>> ./tests/basic/afr/split-brain-resolution.t - 18 second
>>> ./tests/basic/afr/granular-esh/add-brick.t - 18 second
>>> ./tests/bugs/cli/bug-1169302.t - 17 second
>>> ./tests/bugs/bug-1110262.t - 17 second
>>> ./tests/bugs/bitrot/bug-1228680.t - 17 second
>>> ./tests/basic/tier/readdir-during-migration.t - 17 second
>>> ./tests/basic/volume.t - 16 second
>>> ./tests/basic/afr/resolve.t - 16 second
>>> ./tests/basic/afr/read-subvol-data.t - 16 second
>>> ./tests/bugs/cli/bug-1047416.t - 15 second
>>> ./tests/bugs/changelog/bug-1321955.t - 15 second
>>> ./tests/bugs/bitrot/1209752-volume-status-should-show-bitrot-scrub-info.t
>>> - 15 second
>>> ./tests/basic/quota-anon-fd-nfs.t - 15 second
>>> ./tests/basic/pump.t - 15 second
>>> ./tests/basic/glusterd/arbiter-volume-probe.t - 15 second
>>> ./tests/basic/gfid-access.t - 15 second
>>> ./tests/bugs/changelog/bug-1211327.t - 14 second
>>> ./tests/bitrot/bug-1207627-bitrot-scrub-status.t - 14 second
>>> ./tests/basic/rpc-coverage.t - 14 second
>>> ./tests/basic/multiplex.t - 14 second
>>> ./tests/basic/glusterd/disperse-create.t - 14 second
>>> ./tests/basic/cdc.t - 14 second
>>> ./tests/basic/bd.t - 14 second
>>> ./tests/basic/afr/stale-file-lookup.t - 14 second
>>> ./tests/basic/afr/root-squash-self-heal.t - 14 second
>>> ./tests/basic/afr/read-subvol-entry.t - 14 second
>>> ./tests/bugs/cli/bug-822830.t - 13 second
>>> ./tests/bugs/cli/bug-1030580.t - 13 second
>>> ./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t
>>> - 13 second
>>> ./tests/basic/volume-status.t - 13 second
>>> ./tests/basic/quota-nfs.t - 13 second
>>> ./tests/basic/inode-quota-enforcing.t - 13 second
>>> ./tests/basic/ec/ec-root-heal.t - 13 second
>>> ./tests/basic/stats-dump.t - 12 second
>>> ./tests/basic/glusterd/arbiter-volume.t - 12 second
>>> ./tests/basic/ec/ec-read-policy.t - 12 second
>>> ./tests/bugs/cli/bug-1087487.t - 11 second
>>> ./tests/bugs/changelog/bug-1208470.t - 11 second
>>> ./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t -
>>> 11 second
>>> ./tests/bugs/access-control/bug-887098-gmount-crash.t - 11 second
>>> ./tests/bitrot/bug-1244613.t - 11 second
>>> ./tests/basic/fop-sampling.t - 11 second
>>> ./tests/basic/ec/ec-anonymous-fd.t - 11 second
>>> ./tests/bugs/core/bug-1119582.t - 10 second
>>> ./tests/bugs/bug-1258069.t - 10 second
>>> ./tests/bugs/bitrot/bug-1229134-bitd-not-support-vol-set.t - 10 second
>>> ./tests/bugs/access-control/bug-958691.t - 10 second
>>> ./tests/basic/tier/ctr-rename-overwrite.t - 10 second
>>> ./tests/basic/meta.t - 10 second
>>> ./tests/basic/afr/arbiter-remove-brick.t - 10 second
>>> ./tests/bugs/cli/bug-1022905.t - 9 second
>>> ./tests/bugs/bitrot/bug-1210684-scrub-pause-resume-error-handling.t -
>>> 9 second
>>> ./tests/basic/ec/nfs.t - 9 second
>>> ./tests/basic/distribute/bug-1265677-use-readdirp.t - 9 second
>>> ./tests/basic/afr/gfid-mismatch.t - 9 second
>>> ./tests/basic/afr/gfid-heal.t - 9 second
>>> ./tests/basic/afr/compounded-write-txns.t - 9 second
>>> ./tests/basic/afr/arbiter-statfs.t - 9 second
>>> ./tests/bugs/access-control/bug-1051896.t - 8 second
>>> ./tests/bitrot/br-stub.t - 8 second
>>> ./tests/basic/pgfid-feat.t - 8 second
>>> ./tests/basic/md-cache/bug-1317785.t - 8 second
>>> ./tests/basic/gfapi/upcall-cache-invalidate.t - 8 second
>>> ./tests/basic/gfapi/gfapi-dup.t - 8 second
>>> ./tests/basic/gfapi/anonymous_fd.t - 8 second
>>> ./tests/basic/ec/ec-internal-xattrs.t - 8 second
>>> ./tests/basic/ec/dht-rename.t - 8 second
>>> ./tests/basic/afr/heal-info.t - 8 second
>>> ./tests/bugs/core/bug-1168803-snapd-option-validation-fix.t - 7 second
>>> ./tests/bugs/core/949327.t - 7 second
>>> ./tests/bugs/cli/bug-983317-volume-get.t - 7 second
>>> ./tests/bugs/cli/bug-982174.t - 7 second
>>> ./tests/bugs/cli/bug-961307.t - 7 second
>>> ./tests/bitrot/bug-1221914.t - 7 second
>>> ./tests/basic/quota-rename.t - 7 second
>>> ./tests/basic/md-cache/bug-1418249.t - 7 second
>>> ./tests/basic/gfapi/glfs_sysrq.t - 7 second
>>> ./tests/basic/gfapi/gfapi-trunc.t - 7 second
>>> ./tests/basic/gfapi/gfapi-async-calls-test.t - 7 second
>>> ./tests/basic/gfapi/bug-1241104.t - 7 second
>>> ./tests/basic/fops-sanity.t - 7 second
>>> ./tests/basic/distribute/throttle-rebal.t - 7 second
>>> ./tests/bugs/core/bug-1135514-allow-setxattr-with-null-value.t - 6
>>> second
>>> ./tests/bugs/core/bug-1111557.t - 6 second
>>> ./tests/bugs/cli/bug-977246.t - 6 second
>>> ./tests/bugs/cli/bug-921215.t - 6 second
>>> ./tests/bugs/cli/bug-867252.t - 6 second
>>> ./tests/bugs/cli/bug-1378842-volume-get-all.t - 6 second
>>> ./tests/bugs/cli/bug-1004218.t - 6 second
>>> ./tests/bugs/access-control/bug-1387241.t - 6 second
>>> ./tests/bitrot/bug-internal-xattrs-check-1243391.t - 6 second
>>> ./tests/basic/gfapi/libgfapi-fini-hang.t - 6 second
>>> ./tests/basic/afr/arbiter-cli.t - 6 second
>>> ./tests/bugs/core/bug-1117951.t - 5 second
>>> ./tests/bugs/cli/bug-969193.t - 5 second
>>> ./tests/bugs/cli/bug-949298.t - 5 second
>>> ./tests/bugs/cli/bug-764638.t - 5 second
>>> ./tests/bugs/cli/bug-1047378.t - 5 second
>>> ./tests/basic/posixonly.t - 1 second
>>> ./tests/basic/netgroup_parsing.t - 1 second
>>> ./tests/basic/exports_parsing.t - 1 second
>>> ./tests/basic/first-test.t - 0 second
>>>
>>> Result is 1
>>>
>>> tar: Removing leading `/' from member names
>>> Logs archived in http://slave28.cloud.gluster.o
>>> rg/logs/glusterfs-logs-regression-test-burn-in-2932.tgz
>>> kernel.core_pattern = /%e-%p.core
>>> Build step 'Execute shell' marked build as failure
>>> _______________________________________________
>>> maintainers mailing list
>>> maintainers at gluster.org
>>> http://lists.gluster.org/mailman/listinfo/maintainers
>>>
>> --
>> - Atin (atinm)
>>
>
>
>
> --
>
> ATin Mukherjee
>
> Associate Manager, RHGS Development
>
> Red Hat
>
> <https://www.redhat.com>
>
> amukherj at redhat.com M: +919739491377
> <http://redhatemailsignature-marketing.itos.redhat.com/> IM: IRC:
> atinm, twitter: @mukherjee_atin
> <https://red.ht/sig>
>
>
>
> --
>
> ATin Mukherjee
>
> Associate Manager, RHGS Development
>
> Red Hat
>
> <https://www.redhat.com>
>
> amukherj at redhat.com M: +919739491377
> <http://redhatemailsignature-marketing.itos.redhat.com/> IM: IRC:
> atinm, twitter: @mukherjee_atin
> <https://red.ht/sig>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
--
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20170412/b7e1f92a/attachment-0001.html>
More information about the Gluster-devel
mailing list