<div><div dir="auto">C7 nightly has a crash too.</div></div><div><br><div class="gmail_quote"><div dir="ltr">---------- Forwarded message ---------<br>From: <span dir="ltr">&lt;<a href="mailto:jenkins@build.gluster.org">jenkins@build.gluster.org</a>&gt;</span><br>Date: Sat, 18 Aug 2018 at 00:01<br>Subject: [Gluster-Maintainers] Build failed in Jenkins: regression-test-burn-in #4067<br>To:  &lt;<a href="mailto:maintainers@gluster.org">maintainers@gluster.org</a>&gt;<br></div><br><br>See &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/4067/display/redirect?page=changes" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/4067/display/redirect?page=changes</a>&gt;<br>
<br>
Changes:<br>
<br>
[Amar Tumballi] jbr : fix coverity issues in jbr<br>
<br>
[Amar Tumballi] statedump : fix coverity issues<br>
<br>
[Amar Tumballi] glusterd: coverity defects fix introduced by commit 1f3bfe7<br>
<br>
[Amar Tumballi] features/acl: Fix a possible null dereference<br>
<br>
[Amar Tumballi] meta : fix coverity in meta-helpers.c<br>
<br>
[Amar Tumballi] nfs-server-mount : fix coverity issues in mount3.c<br>
<br>
[Amar Tumballi] posix: FORWARD_NULL coverity fix<br>
<br>
[Amar Tumballi] locks: FORWARD_NULL coverity fix<br>
<br>
[Amar Tumballi] doc: Add details around xlator categories<br>
<br>
[Amar Tumballi] features/changelog: Fix missing unlocks<br>
<br>
------------------------------------------<br>
[...truncated 981.94 KB...]<br>
        this = 0x7f6f2c007e50<br>
        priv = 0x7f6f2c0ab440<br>
        stub = 0x0<br>
        tmp = 0x0<br>
        list = {next = 0x7f6f0f7fde90, prev = 0x7f6f0f7fde90}<br>
        count = 0<br>
        do_fsync = true<br>
        __FUNCTION__ = &quot;posix_fsyncer&quot;<br>
#3  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#4  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 10 (Thread 0x7f6f24233700 (LWP 28244)):<br>
#0  0x00007f6f3d565d42 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#1  0x00007f6f2a810737 in iot_worker (data=0x7f6f2c068f40) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/xlators/performance/io-threads/src/io-threads.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/xlators/performance/io-threads/src/io-threads.c</a>&gt;:195<br>
        conf = 0x7f6f2c068f40<br>
        this = 0x7f6f2c020020<br>
        stub = 0x0<br>
        sleep_till = {tv_sec = 1534527479, tv_nsec = 542650613}<br>
        ret = 0<br>
        pri = -1<br>
        bye = false<br>
        __FUNCTION__ = &quot;iot_worker&quot;<br>
#2  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#3  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 9 (Thread 0x7f6f3ea2a780 (LWP 28136)):<br>
#0  0x00007f6f3d562f97 in pthread_join () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#1  0x00007f6f3e589159 in event_dispatch_epoll (event_pool=0x770c30) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/event-epoll.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/event-epoll.c</a>&gt;:750<br>
        i = 1<br>
        t_id = 140115546580736<br>
        pollercount = 1<br>
        ret = 0<br>
        ev_data = 0x7bc6c0<br>
        thread_name = &quot;epoll000\000\000&quot;<br>
        __FUNCTION__ = &quot;event_dispatch_epoll&quot;<br>
#2  0x00007f6f3e546b4a in event_dispatch (event_pool=0x770c30) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/event.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/event.c</a>&gt;:124<br>
        ret = -1<br>
        __FUNCTION__ = &quot;event_dispatch&quot;<br>
#3  0x000000000040b539 in ?? ()<br>
No symbol table info available.<br>
#4  0x0000000000000000 in ?? ()<br>
No symbol table info available.<br>
<br>
Thread 8 (Thread 0x7f6f35afe700 (LWP 28137)):<br>
#0  0x00007f6f3d568f3d in nanosleep () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#1  0x00007f6f3e520ef4 in gf_timer_proc (data=0x7796f0) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/timer.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/timer.c</a>&gt;:202<br>
        now = 516473032615399<br>
        now_ts = {tv_sec = 516473, tv_nsec = 32615399}<br>
        reg = 0x7796f0<br>
        sleepts = {tv_sec = 1, tv_nsec = 0}<br>
        event = 0x779700<br>
        tmp = 0x779700<br>
        old_THIS = 0x7f6f3e81a2c0 &lt;global_xlator&gt;<br>
#2  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#3  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 7 (Thread 0x7f6f24274700 (LWP 28202)):<br>
#0  0x00007f6f3d565d42 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#1  0x00007f6f2a810737 in iot_worker (data=0x7f6f2c068f40) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/xlators/performance/io-threads/src/io-threads.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/xlators/performance/io-threads/src/io-threads.c</a>&gt;:195<br>
        conf = 0x7f6f2c068f40<br>
        this = 0x7f6f2c020020<br>
        stub = 0x0<br>
        sleep_till = {tv_sec = 1534527479, tv_nsec = 542650613}<br>
        ret = 0<br>
        pri = -1<br>
        bye = false<br>
        __FUNCTION__ = &quot;iot_worker&quot;<br>
#2  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#3  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 6 (Thread 0x7f6f2647a700 (LWP 28150)):<br>
#0  0x00007f6f3cc1dc73 in select () from /lib64/libc.so.6<br>
No symbol table info available.<br>
#1  0x00007f6f2b92aace in changelog_ev_dispatch (data=0x7f6f2c08c0a8) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/xlators/features/changelog/src/changelog-ev-handle.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/xlators/features/changelog/src/changelog-ev-handle.c</a>&gt;:350<br>
        ret = 3<br>
        opaque = 0x0<br>
        this = 0x7f6f2c0116b0<br>
        c_clnt = 0x7f6f2c08c0a8<br>
        tv = {tv_sec = 0, tv_usec = 128007}<br>
        __FUNCTION__ = &quot;changelog_ev_dispatch&quot;<br>
#2  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#3  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 5 (Thread 0x7f6f3e8c9700 (LWP 28146)):<br>
#0  0x00007f6f3d565d42 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#1  0x00007f6f2a810737 in iot_worker (data=0x7f6f2c068f40) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/xlators/performance/io-threads/src/io-threads.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/xlators/performance/io-threads/src/io-threads.c</a>&gt;:195<br>
        conf = 0x7f6f2c068f40<br>
        this = 0x7f6f2c020020<br>
        stub = 0x0<br>
        sleep_till = {tv_sec = 1534527479, tv_nsec = 543650641}<br>
        ret = 0<br>
        pri = -1<br>
        bye = false<br>
        __FUNCTION__ = &quot;iot_worker&quot;<br>
#2  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#3  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 4 (Thread 0x7f6f33afa700 (LWP 28141)):<br>
#0  0x00007f6f3d565d42 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#1  0x00007f6f3e55f8a6 in syncenv_task (proc=0x77a2f0) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c</a>&gt;:603<br>
        env = 0x779f30<br>
        task = 0x0<br>
        sleep_till = {tv_sec = 1534527889, tv_nsec = 0}<br>
        ret = 0<br>
#2  0x00007f6f3e55fb41 in syncenv_processor (thdata=0x77a2f0) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c</a>&gt;:695<br>
        env = 0x779f30<br>
        proc = 0x77a2f0<br>
        task = 0x0<br>
#3  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#4  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 3 (Thread 0x7f6f34afc700 (LWP 28139)):<br>
#0  0x00007f6f3cbed56d in nanosleep () from /lib64/libc.so.6<br>
No symbol table info available.<br>
#1  0x00007f6f3cbed404 in sleep () from /lib64/libc.so.6<br>
No symbol table info available.<br>
#2  0x00007f6f3e54830e in pool_sweeper (arg=0x0) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/mem-pool.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/mem-pool.c</a>&gt;:470<br>
        state = {death_row = {next = 0x7f6f34af9e90, prev = 0x7f6f34af9e90}, cold_lists = {0x7f6f18002030, 0x7f6f2c002000, 0x7f6f2c000d70, 0x7ba510, 0x0 &lt;repeats 1020 times&gt;}, n_cold_lists = 4}<br>
        pool_list = 0x7f6f34af9e90<br>
        next_pl = 0x7f6f34af9e90<br>
        pt_pool = 0x0<br>
        i = 4<br>
        begin_time = {tv_sec = 1534527349, tv_usec = 820624}<br>
        end_time = {tv_sec = 1534527349, tv_usec = 820631}<br>
        elapsed = {tv_sec = 0, tv_usec = 7}<br>
#3  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#4  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 2 (Thread 0x7f6f342fb700 (LWP 28140)):<br>
#0  0x00007f6f3d565d42 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#1  0x00007f6f3e55f8a6 in syncenv_task (proc=0x779f30) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c</a>&gt;:603<br>
        env = 0x779f30<br>
        task = 0x0<br>
        sleep_till = {tv_sec = 1534527889, tv_nsec = 0}<br>
        ret = 0<br>
#2  0x00007f6f3e55fb41 in syncenv_processor (thdata=0x779f30) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/syncop.c</a>&gt;:695<br>
        env = 0x779f30<br>
        proc = 0x779f30<br>
        task = 0x0<br>
#3  0x00007f6f3d561e25 in start_thread () from /lib64/libpthread.so.0<br>
No symbol table info available.<br>
#4  0x00007f6f3cc26bad in clone () from /lib64/libc.so.6<br>
No symbol table info available.<br>
<br>
Thread 1 (Thread 0x7f6f352fd700 (LWP 28138)):<br>
#0  0x00007f6f3e549039 in mem_get (mem_pool=0x73a4d000) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/mem-pool.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/libglusterfs/src/mem-pool.c</a>&gt;:826<br>
        pool_list = 0x7f6ee8000980<br>
        pt_pool = 0x29383878302b7469<br>
        retval = 0x0<br>
        __FUNCTION__ = &quot;mem_get&quot;<br>
#1  0x00007f6f3e2d5d83 in rpc_clnt_submit (rpc=0x7ba7f0, prog=0x7f6f3e4e39e0 &lt;clnt_pmap_signout_prog&gt;, procnum=5, cbkfn=0x7f6f3e2dadef &lt;mgmt_pmap_signout_cbk&gt;, proghdr=0x7f6f352fbd40, proghdrcount=1, progpayload=0x0, progpayloadcount=0, iobref=0x7f6ee8000930, frame=0x7f6ee8001328, rsphdr=0x0, rsphdr_count=0, rsp_payload=0x0, rsp_payload_count=0, rsp_iobref=0x0) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-lib/src/rpc-clnt.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-lib/src/rpc-clnt.c</a>&gt;:1654<br>
        conn = 0x7ba820<br>
        request_iob = 0x0<br>
        rpchdr = {iov_base = 0x0, iov_len = 0}<br>
        rpcreq = 0x0<br>
        req = {msg = {rpchdr = 0x0, rpchdrcount = 0, proghdr = 0x0, proghdrcount = 0, progpayload = 0x0, progpayloadcount = 0, iobref = 0x0}, rsp = {rsphdr = 0x0, rsphdr_count = 892320976, rsp_payload = 0x7f6f3e0a20aa &lt;xdr_serialize_generic+133&gt;, rsp_payload_count = 1040953334, rsp_iobref = 0x7f6f352fcd50}, rpc_req = 0x7f6f3ea29f00}<br>
        ret = -1<br>
        proglen = 0<br>
        new_iobref = 0 &#39;\000&#39;<br>
        callid = 0<br>
        need_unref = false<br>
        __FUNCTION__ = &quot;rpc_clnt_submit&quot;<br>
#2  0x00007f6f3e2db23b in rpc_clnt_mgmt_pmap_signout (ctx=0x739010, brickname=0x7f6f2c02eff0 &quot;/d/backends/2/patchy&quot;) at &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-lib/src/mgmt-pmap.c" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/rpc/rpc-lib/src/mgmt-pmap.c</a>&gt;:127<br>
        ret = 32<br>
        req = {brick = 0x7f6f2c02eff0 &quot;/d/backends/2/patchy&quot;, port = 49153, rdma_port = 0}<br>
        frame = 0x7f6ee8001328<br>
        cmd_args = 0x739010<br>
        brick_name = &#39;\000&#39; &lt;repeats 4095 times&gt;<br>
        iov = {iov_base = 0x7f6f3ea29f00, iov_len = 32}<br>
        iobuf = 0x754c10<br>
        iobref = 0x7f6ee8000930<br>
        xdr_size = 32<br>
        __FUNCTION__ = &quot;rpc_clnt_mgmt_pmap_signout&quot;<br>
#3  0x0000000000408b61 in ?? ()<br>
No symbol table info available.<br>
#4  0x0000000000000001 in ?? ()<br>
No symbol table info available.<br>
#5  0x00000000000186c0 in ?? ()<br>
No symbol table info available.<br>
#6  0x0000000000414880 in ?? ()<br>
No symbol table info available.<br>
#7  0x000000000000000f in ?? ()<br>
No symbol table info available.<br>
#8  0x0000000000000000 in ?? ()<br>
No symbol table info available.<br>
=========================================================<br>
              Finish backtrace<br>
         program name : /build/install/sbin/glusterfsd<br>
         corefile     : /glfs_sigwait-28136.core<br>
=========================================================<br>
<br>
+ rm -f /build/install/cores/gdbout.txt<br>
+ sort /build/install/cores/liblist.txt<br>
+ uniq<br>
+ cat /build/install/cores/liblist.txt.tmp<br>
+ grep -v /build/install<br>
+ tar -cf /archives/archived_builds/build-install-regression-test-burn-in-4067.tar /build/install/sbin /build/install/bin /build/install/lib /build/install/libexec /build/install/cores<br>
tar: Removing leading `/&#39; from member names<br>
+ tar -rhf /archives/archived_builds/build-install-regression-test-burn-in-4067.tar -T /build/install/cores/liblist.txt<br>
tar: Removing leading `/&#39; from member names<br>
+ bzip2 /archives/archived_builds/build-install-regression-test-burn-in-4067.tar<br>
+ rm -f /build/install/cores/liblist.txt<br>
+ rm -f /build/install/cores/liblist.txt.tmp<br>
+ find /archives -size +1G -delete -type f<br>
+ echo &#39;Cores and build archived in <a href="http://builder103.cloud.gluster.org/archived_builds/build-install-regression-test-burn-in-4067.tar.bz2" rel="noreferrer" target="_blank">http://builder103.cloud.gluster.org/archived_builds/build-install-regression-test-burn-in-4067.tar.bz2</a>&#39;<br>
Cores and build archived in <a href="http://builder103.cloud.gluster.org/archived_builds/build-install-regression-test-burn-in-4067.tar.bz2" rel="noreferrer" target="_blank">http://builder103.cloud.gluster.org/archived_builds/build-install-regression-test-burn-in-4067.tar.bz2</a><br>
+ echo &#39;Open core using the following command to get a proper stack&#39;<br>
Open core using the following command to get a proper stack<br>
+ echo &#39;Example: From root of extracted tarball&#39;<br>
Example: From root of extracted tarball<br>
+ echo &#39;\t\tgdb -ex &#39;\&#39;&#39;set sysroot ./&#39;\&#39;&#39; -ex &#39;\&#39;&#39;core-file ./build/install/cores/xxx.core&#39;\&#39;&#39; &lt;target, say ./build/install/sbin/glusterd&gt;&#39;<br>
\t\tgdb -ex &#39;set sysroot ./&#39; -ex &#39;core-file ./build/install/cores/xxx.core&#39; &lt;target, say ./build/install/sbin/glusterd&gt;<br>
+ RET=1<br>
+ &#39;[&#39; 1 -ne 0 &#39;]&#39;<br>
+ tar -czf &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/4067/artifact/glusterfs-logs.tgz" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/4067/artifact/glusterfs-logs.tgz</a>&gt; /var/log/glusterfs /var/log/messages /var/log/messages-20180722 /var/log/messages-20180729 /var/log/messages-20180805 /var/log/messages-20180812<br>
tar: Removing leading `/&#39; from member names<br>
+ scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i &lt;<a href="https://build.gluster.org/job/regression-test-burn-in/ws/" rel="noreferrer" target="_blank">https://build.gluster.org/job/regression-test-burn-in/ws/</a>&gt; glusterfs-logs.tgz _logs_collector@http.int.rht.gluster.org:/var/www/glusterfs-logs/regression-test-burn-in-4067.tgz<br>
ssh: connect to host <a href="http://http.int.rht.gluster.org" rel="noreferrer" target="_blank">http.int.rht.gluster.org</a> port 22: Connection timed out<br>
lost connection<br>
+ true<br>
+ case $(uname -s) in<br>
++ uname -s<br>
+ /sbin/sysctl -w kernel.core_pattern=/%e-%p.core<br>
kernel.core_pattern = /%e-%p.core<br>
+ exit 1<br>
Build step &#39;Execute shell&#39; marked build as failure<br>
_______________________________________________<br>
maintainers mailing list<br>
<a href="mailto:maintainers@gluster.org" target="_blank">maintainers@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/maintainers" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/maintainers</a><br>
</div></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">- Atin (atinm)</div>