[Gluster-devel] bug-1421590-brick-mux-reuse-ports.t failure

Amar Tumballi atumball at redhat.com
Wed Apr 12 06:58:35 UTC 2017


On Wed, Apr 12, 2017 at 12:07 PM, Atin Mukherjee <amukherj at redhat.com>
wrote:

> As per http://fstat.rht.gluster.org/weeks/1 the test in $Subject has
> failed multiple times and is now blocking most of the patches to pass the
> regression. I have a patch https://review.gluster.org/#/c/17033/ to
> remove this test entirely and I have the reason in the commit message.
>
> Can this patch be reviewed and merged asap?
>

+1. This is stopping us from many patches which could have passed otherwise.

Regards,
Amar


>
> ---------- Forwarded message ----------
> From: Atin Mukherjee <amukherj at redhat.com>
> Date: Mon, Apr 10, 2017 at 2:57 PM
> Subject: Re: [Gluster-Maintainers] Build failed in Jenkins:
> regression-test-burn-in #2932
> To: GlusterFS Maintainers <maintainers at gluster.org>, Jeff Darcy <
> jeff at pl.atyp.us>
>
>
> bug-1421590-brick-mux-reuse-ports.t seems to be a bad test to me and here
> is my reasoning:
>
> This test tries to check if the ports are reused or not. When a volume is
> restarted, by the time glusterd tries to allocate a new port to the one of
> the brick processes of the volume there is no guarantee that the older port
> will be allocated given the kernel might take some extra time to free up
> the port between this time frame. From https://build.gluster.org/job/
> regression-test-burn-in/2932/console we can clearly see that post restart
> of the volume, glusterd allocated port 49153 & 49155 for brick1 & brick2
> respectively but the test was expecting the ports to be matched with 49155
> & 49156 which were allocated before the volume was restarted.
>
> @Jeff - Is there any specific reason we want to keep this test running?
>
>
> On Sat, Apr 8, 2017 at 8:12 AM, Atin Mukherjee <amukherj at redhat.com>
> wrote:
>
>>
>> On Sat, 8 Apr 2017 at 08:06, <jenkins at build.gluster.org> wrote:
>>
>>> See <http://build.gluster.org/job/regression-test-burn-in/2932/d
>>> isplay/redirect>
>>>
>>> ------------------------------------------
>>> [...truncated 12020 lines...]
>>> ok 5, LINENUM:32
>>> ok 6, LINENUM:33
>>> ok 7, LINENUM:35
>>> not ok 8 , LINENUM:37
>>> FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
>>> ok 9, LINENUM:38
>>> not ok 10 , LINENUM:40
>>> FAILED COMMAND: gluster --mode=script --wignore volume start patchy
>>> not ok 11 Got "" instead of "49152", LINENUM:42
>>> FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
>>> not ok 12 , LINENUM:47
>>> FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
>>> not ok 13 , LINENUM:48
>>> FAILED COMMAND: gluster --mode=script --wignore volume start patchy
>>> not ok 14 Got "" instead of "49152", LINENUM:50
>>> FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
>>> not ok 15 Got "" instead of "get_nth_brick_port_for_volume", LINENUM:51
>>> FAILED COMMAND: get_nth_brick_port_for_volume patchy 2
>>> not ok 16 , LINENUM:53
>>> FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
>>> ok 17, LINENUM:55
>>> not ok 18 , LINENUM:57
>>> FAILED COMMAND: gluster --mode=script --wignore volume start patchy
>>> not ok 19 Got "" instead of "49152", LINENUM:59
>>> FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
>>> volume set: success
>>> Failed 10/19 subtests
>>>
>>> Test Summary Report
>>> -------------------
>>> ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t (Wstat: 0 Tests:
>>> 19 Failed: 10)
>>>   Failed tests:  8, 10-16, 18-19
>>> Files=1, Tests=19, 249 wallclock secs ( 0.03 usr  0.01 sys + 13.40 cusr
>>> 3.36 csys = 16.80 CPU)
>>> Result: FAIL
>>> End of test ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t
>>
>>
>> Something is wrong with this test, have seen it failing in many
>> regression test burns. I'll take a look at it.
>>
>>
>>> ============================================================
>>> ====================
>>>
>>>
>>> Run complete
>>> ============================================================
>>> ====================
>>> Number of tests found:                             199
>>> Number of tests selected for run based on pattern: 199
>>> Number of tests skipped as they were marked bad:   7
>>> Number of tests skipped because of known_issues:   4
>>> Number of tests that were run:                     188
>>>
>>> 1 test(s) failed
>>> ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t
>>>
>>> 0 test(s) generated core
>>>
>>>
>>> Tests ordered by time taken, slowest to fastest:
>>> ============================================================
>>> ====================
>>> ./tests/basic/ec/ec-12-4.t  -  336 second
>>> ./tests/basic/ec/ec-7-3.t  -  199 second
>>> ./tests/basic/ec/ec-6-2.t  -  178 second
>>> ./tests/basic/ec/self-heal.t  -  158 second
>>> ./tests/basic/afr/split-brain-favorite-child-policy.t  -  151 second
>>> ./tests/basic/ec/ec-5-2.t  -  150 second
>>> ./tests/basic/ec/ec-5-1.t  -  150 second
>>> ./tests/basic/afr/entry-self-heal.t  -  150 second
>>> ./tests/basic/afr/self-heal.t  -  137 second
>>> ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t  -  131 second
>>> ./tests/basic/tier/legacy-many.t  -  127 second
>>> ./tests/basic/tier/tier.t  -  126 second
>>> ./tests/basic/ec/ec-4-1.t  -  123 second
>>> ./tests/basic/ec/ec-optimistic-changelog.t  -  111 second
>>> ./tests/basic/afr/self-heald.t  -  109 second
>>> ./tests/basic/ec/ec-3-1.t  -  95 second
>>> ./tests/basic/volume-snapshot-clone.t  -  88 second
>>> ./tests/bugs/core/bug-1110917.t  -  85 second
>>> ./tests/basic/tier/new-tier-cmds.t  -  83 second
>>> ./tests/basic/afr/split-brain-heal-info.t  -  82 second
>>> ./tests/bugs/cli/bug-1320388.t  -  81 second
>>> ./tests/basic/ec/heal-info.t  -  80 second
>>> ./tests/basic/ec/ec-new-entry.t  -  79 second
>>> ./tests/basic/ec/ec-background-heals.t  -  77 second
>>> ./tests/basic/afr/split-brain-healing.t  -  76 second
>>> ./tests/basic/afr/metadata-self-heal.t  -  74 second
>>> ./tests/basic/ec/ec-notify.t  -  73 second
>>> ./tests/basic/afr/sparse-file-self-heal.t  -  73 second
>>> ./tests/basic/afr/granular-esh/cli.t  -  69 second
>>> ./tests/basic/quota.t  -  67 second
>>> ./tests/bugs/bug-1368312.t  -  63 second
>>> ./tests/basic/uss.t  -  52 second
>>> ./tests/basic/tier/frequency-counters.t  -  52 second
>>> ./tests/bugs/cli/bug-770655.t  -  50 second
>>> ./tests/basic/volume-snapshot.t  -  49 second
>>> ./tests/basic/afr/quorum.t  -  48 second
>>> ./tests/basic/tier/fops-during-migration-pause.t  -  47 second
>>> ./tests/basic/mount-nfs-auth.t  -  46 second
>>> ./tests/basic/ec/ec.t  -  46 second
>>> ./tests/basic/ec/ec-readdir.t  -  43 second
>>> ./tests/basic/afr/inodelk.t  -  40 second
>>> ./tests/basic/mpx-compat.t  -  39 second
>>> ./tests/basic/afr/arbiter.t  -  38 second
>>> ./tests/basic/ec/ec-cpu-extensions.t  -  37 second
>>> ./tests/basic/tier/locked_file_migration.t  -  36 second
>>> ./tests/bitrot/br-state-check.t  -  35 second
>>> ./tests/basic/tier/tier-heald.t  -  35 second
>>> ./tests/bitrot/bug-1294786.t  -  34 second
>>> ./tests/basic/volume-snapshot-xml.t  -  33 second
>>> ./tests/basic/mgmt_v3-locks.t  -  33 second
>>> ./tests/basic/tier/unlink-during-migration.t  -  32 second
>>> ./tests/basic/afr/granular-esh/conservative-merge.t  -  32 second
>>> ./tests/basic/afr/gfid-self-heal.t  -  31 second
>>> ./tests/basic/geo-replication/marker-xattrs.t  -  29 second
>>> ./tests/basic/quota-ancestry-building.t  -  27 second
>>> ./tests/bugs/cli/bug-1353156-get-state-cli-validations.t  -  26 second
>>> ./tests/basic/afr/arbiter-add-brick.t  -  26 second
>>> ./tests/bugs/core/bug-1402841.t-mt-dir-scan-race.t  -  25 second
>>> ./tests/basic/afr/heal-quota.t  -  25 second
>>> ./tests/basic/afr/data-self-heal.t  -  25 second
>>> ./tests/basic/afr/arbiter-mount.t  -  24 second
>>> ./tests/bugs/bitrot/bug-1227996.t  -  23 second
>>> ./tests/bitrot/bug-1373520.t  -  23 second
>>> ./tests/basic/nufa.t  -  23 second
>>> ./tests/basic/afr/durability-off.t  -  23 second
>>> ./tests/basic/mount.t  -  22 second
>>> ./tests/basic/gfapi/gfapi-ssl-test.t  -  22 second
>>> ./tests/basic/ec/quota.t  -  22 second
>>> ./tests/basic/afr/replace-brick-self-heal.t  -  22 second
>>> ./tests/basic/afr/granular-esh/replace-brick.t  -  22 second
>>> ./tests/basic/op_errnos.t  -  21 second
>>> ./tests/basic/glusterd/volfile_server_switch.t  -  21 second
>>> ./tests/bugs/bitrot/bug-1245981.t  -  20 second
>>> ./tests/basic/afr/granular-esh/granular-indices-but-non-granular-heal.t
>>> -  20 second
>>> ./tests/basic/afr/granular-esh/granular-esh.t  -  20 second
>>> ./tests/basic/0symbol-check.t  -  20 second
>>> ./tests/bugs/cli/bug-1113476.t  -  19 second
>>> ./tests/bugs/changelog/bug-1225542.t  -  19 second
>>> ./tests/bugs/bitrot/bug-1288490.t  -  19 second
>>> ./tests/basic/tier/file_with_spaces.t  -  19 second
>>> ./tests/basic/glusterd/heald.t  -  19 second
>>> ./tests/basic/afr/client-side-heal.t  -  19 second
>>> ./tests/basic/afr/add-brick-self-heal.t  -  19 second
>>> ./tests/bugs/cli/bug-1077682.t  -  18 second
>>> ./tests/bugs/bitrot/1209751-bitrot-scrub-tunable-reset.t  -  18 second
>>> ./tests/basic/ec/statedump.t  -  18 second
>>> ./tests/basic/afr/split-brain-resolution.t  -  18 second
>>> ./tests/basic/afr/granular-esh/add-brick.t  -  18 second
>>> ./tests/bugs/cli/bug-1169302.t  -  17 second
>>> ./tests/bugs/bug-1110262.t  -  17 second
>>> ./tests/bugs/bitrot/bug-1228680.t  -  17 second
>>> ./tests/basic/tier/readdir-during-migration.t  -  17 second
>>> ./tests/basic/volume.t  -  16 second
>>> ./tests/basic/afr/resolve.t  -  16 second
>>> ./tests/basic/afr/read-subvol-data.t  -  16 second
>>> ./tests/bugs/cli/bug-1047416.t  -  15 second
>>> ./tests/bugs/changelog/bug-1321955.t  -  15 second
>>> ./tests/bugs/bitrot/1209752-volume-status-should-show-bitrot-scrub-info.t
>>> -  15 second
>>> ./tests/basic/quota-anon-fd-nfs.t  -  15 second
>>> ./tests/basic/pump.t  -  15 second
>>> ./tests/basic/glusterd/arbiter-volume-probe.t  -  15 second
>>> ./tests/basic/gfid-access.t  -  15 second
>>> ./tests/bugs/changelog/bug-1211327.t  -  14 second
>>> ./tests/bitrot/bug-1207627-bitrot-scrub-status.t  -  14 second
>>> ./tests/basic/rpc-coverage.t  -  14 second
>>> ./tests/basic/multiplex.t  -  14 second
>>> ./tests/basic/glusterd/disperse-create.t  -  14 second
>>> ./tests/basic/cdc.t  -  14 second
>>> ./tests/basic/bd.t  -  14 second
>>> ./tests/basic/afr/stale-file-lookup.t  -  14 second
>>> ./tests/basic/afr/root-squash-self-heal.t  -  14 second
>>> ./tests/basic/afr/read-subvol-entry.t  -  14 second
>>> ./tests/bugs/cli/bug-822830.t  -  13 second
>>> ./tests/bugs/cli/bug-1030580.t  -  13 second
>>> ./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t
>>> -  13 second
>>> ./tests/basic/volume-status.t  -  13 second
>>> ./tests/basic/quota-nfs.t  -  13 second
>>> ./tests/basic/inode-quota-enforcing.t  -  13 second
>>> ./tests/basic/ec/ec-root-heal.t  -  13 second
>>> ./tests/basic/stats-dump.t  -  12 second
>>> ./tests/basic/glusterd/arbiter-volume.t  -  12 second
>>> ./tests/basic/ec/ec-read-policy.t  -  12 second
>>> ./tests/bugs/cli/bug-1087487.t  -  11 second
>>> ./tests/bugs/changelog/bug-1208470.t  -  11 second
>>> ./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t  -
>>> 11 second
>>> ./tests/bugs/access-control/bug-887098-gmount-crash.t  -  11 second
>>> ./tests/bitrot/bug-1244613.t  -  11 second
>>> ./tests/basic/fop-sampling.t  -  11 second
>>> ./tests/basic/ec/ec-anonymous-fd.t  -  11 second
>>> ./tests/bugs/core/bug-1119582.t  -  10 second
>>> ./tests/bugs/bug-1258069.t  -  10 second
>>> ./tests/bugs/bitrot/bug-1229134-bitd-not-support-vol-set.t  -  10 second
>>> ./tests/bugs/access-control/bug-958691.t  -  10 second
>>> ./tests/basic/tier/ctr-rename-overwrite.t  -  10 second
>>> ./tests/basic/meta.t  -  10 second
>>> ./tests/basic/afr/arbiter-remove-brick.t  -  10 second
>>> ./tests/bugs/cli/bug-1022905.t  -  9 second
>>> ./tests/bugs/bitrot/bug-1210684-scrub-pause-resume-error-handling.t  -
>>> 9 second
>>> ./tests/basic/ec/nfs.t  -  9 second
>>> ./tests/basic/distribute/bug-1265677-use-readdirp.t  -  9 second
>>> ./tests/basic/afr/gfid-mismatch.t  -  9 second
>>> ./tests/basic/afr/gfid-heal.t  -  9 second
>>> ./tests/basic/afr/compounded-write-txns.t  -  9 second
>>> ./tests/basic/afr/arbiter-statfs.t  -  9 second
>>> ./tests/bugs/access-control/bug-1051896.t  -  8 second
>>> ./tests/bitrot/br-stub.t  -  8 second
>>> ./tests/basic/pgfid-feat.t  -  8 second
>>> ./tests/basic/md-cache/bug-1317785.t  -  8 second
>>> ./tests/basic/gfapi/upcall-cache-invalidate.t  -  8 second
>>> ./tests/basic/gfapi/gfapi-dup.t  -  8 second
>>> ./tests/basic/gfapi/anonymous_fd.t  -  8 second
>>> ./tests/basic/ec/ec-internal-xattrs.t  -  8 second
>>> ./tests/basic/ec/dht-rename.t  -  8 second
>>> ./tests/basic/afr/heal-info.t  -  8 second
>>> ./tests/bugs/core/bug-1168803-snapd-option-validation-fix.t  -  7 second
>>> ./tests/bugs/core/949327.t  -  7 second
>>> ./tests/bugs/cli/bug-983317-volume-get.t  -  7 second
>>> ./tests/bugs/cli/bug-982174.t  -  7 second
>>> ./tests/bugs/cli/bug-961307.t  -  7 second
>>> ./tests/bitrot/bug-1221914.t  -  7 second
>>> ./tests/basic/quota-rename.t  -  7 second
>>> ./tests/basic/md-cache/bug-1418249.t  -  7 second
>>> ./tests/basic/gfapi/glfs_sysrq.t  -  7 second
>>> ./tests/basic/gfapi/gfapi-trunc.t  -  7 second
>>> ./tests/basic/gfapi/gfapi-async-calls-test.t  -  7 second
>>> ./tests/basic/gfapi/bug-1241104.t  -  7 second
>>> ./tests/basic/fops-sanity.t  -  7 second
>>> ./tests/basic/distribute/throttle-rebal.t  -  7 second
>>> ./tests/bugs/core/bug-1135514-allow-setxattr-with-null-value.t  -  6
>>> second
>>> ./tests/bugs/core/bug-1111557.t  -  6 second
>>> ./tests/bugs/cli/bug-977246.t  -  6 second
>>> ./tests/bugs/cli/bug-921215.t  -  6 second
>>> ./tests/bugs/cli/bug-867252.t  -  6 second
>>> ./tests/bugs/cli/bug-1378842-volume-get-all.t  -  6 second
>>> ./tests/bugs/cli/bug-1004218.t  -  6 second
>>> ./tests/bugs/access-control/bug-1387241.t  -  6 second
>>> ./tests/bitrot/bug-internal-xattrs-check-1243391.t  -  6 second
>>> ./tests/basic/gfapi/libgfapi-fini-hang.t  -  6 second
>>> ./tests/basic/afr/arbiter-cli.t  -  6 second
>>> ./tests/bugs/core/bug-1117951.t  -  5 second
>>> ./tests/bugs/cli/bug-969193.t  -  5 second
>>> ./tests/bugs/cli/bug-949298.t  -  5 second
>>> ./tests/bugs/cli/bug-764638.t  -  5 second
>>> ./tests/bugs/cli/bug-1047378.t  -  5 second
>>> ./tests/basic/posixonly.t  -  1 second
>>> ./tests/basic/netgroup_parsing.t  -  1 second
>>> ./tests/basic/exports_parsing.t  -  1 second
>>> ./tests/basic/first-test.t  -  0 second
>>>
>>> Result is 1
>>>
>>> tar: Removing leading `/' from member names
>>> Logs archived in http://slave28.cloud.gluster.o
>>> rg/logs/glusterfs-logs-regression-test-burn-in-2932.tgz
>>> kernel.core_pattern = /%e-%p.core
>>> Build step 'Execute shell' marked build as failure
>>> _______________________________________________
>>> maintainers mailing list
>>> maintainers at gluster.org
>>> http://lists.gluster.org/mailman/listinfo/maintainers
>>>
>> --
>> - Atin (atinm)
>>
>
>
>
> --
>
> ATin Mukherjee
>
> Associate Manager, RHGS Development
>
> Red Hat
>
> <https://www.redhat.com>
>
> amukherj at redhat.com    M: +919739491377
> <http://redhatemailsignature-marketing.itos.redhat.com/>     IM: IRC:
> atinm, twitter: @mukherjee_atin
> <https://red.ht/sig>
>
>
>
> --
>
> ATin Mukherjee
>
> Associate Manager, RHGS Development
>
> Red Hat
>
> <https://www.redhat.com>
>
> amukherj at redhat.com    M: +919739491377
> <http://redhatemailsignature-marketing.itos.redhat.com/>     IM: IRC:
> atinm, twitter: @mukherjee_atin
> <https://red.ht/sig>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20170412/b7e1f92a/attachment-0001.html>


More information about the Gluster-devel mailing list