<div dir="ltr"><div dir="ltr">Hi Mohit,<div><br></div><div>The regression run in the subject has failed because a brick has crashed in <pre class="gmail-console-output" style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;box-sizing:border-box;white-space:pre-wrap;word-wrap:break-word;margin:0px;color:rgb(51,51,51)">bug-1432542-mpx-restart-crash.t</pre><pre class="gmail-console-output" style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;box-sizing:border-box;white-space:pre-wrap;word-wrap:break-word;margin:0px;color:rgb(51,51,51)"><br></pre></div><div><pre class="gmail-console-output" style="box-sizing:border-box;white-space:pre-wrap;word-wrap:break-word;margin:0px;color:rgb(51,51,51);text-decoration-style:initial;text-decoration-color:initial"><span class="gmail-timestamp" style="box-sizing:border-box"><b style="box-sizing:border-box">06:03:38</b> </span>1 test(s) generated core
<span class="gmail-timestamp" style="box-sizing:border-box"><b style="box-sizing:border-box">06:03:38</b> </span>./tests/bugs/core/bug-1432542-mpx-restart-crash.t
<span class="gmail-timestamp" style="box-sizing:border-box"><b style="box-sizing:border-box">06:03:38</b> </span></pre><br></div><div>The brick process has crashed in <span style="font-family:monospace,monospace;font-size:x-small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">posix_fs_health_check </span><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif" style="">as</font></span><span style="font-family:monospace,monospace;font-size:x-small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"> </span> this->priv contains garbage. It looks like it might have been freed already. Can you take a look at it?</div><div><br></div><div><br></div><div><br></div><div><div><font face="monospace, monospace" size="1">(gdb) bt</font></div><div><font face="monospace, monospace" size="1">#0 0x00007f4019ea1f19 in vfprintf () from ./lib64/libc.so.6</font></div><div><font face="monospace, monospace" size="1">#1 0x00007f4019eccf49 in vsnprintf () from ./lib64/libc.so.6</font></div><div><font face="monospace, monospace" size="1">#2 0x00007f401b87705a in gf_vasprintf (string_ptr=0x7f3e81ff99f0, format=0x7f400df32f40 "op=%s;path=%s;error=%s;brick=%s:%s timeout is %d", arg=0x7f3e81ff99f8)</font></div><div><font face="monospace, monospace" size="1"> at /home/jenkins/root/workspace/centos7-regression/libglusterfs/src/mem-pool.c:234</font></div><div><font face="monospace, monospace" size="1">#3 0x00007f401b8de6e2 in _gf_event (event=EVENT_POSIX_HEALTH_CHECK_FAILED, fmt=0x7f400df32f40 "op=%s;path=%s;error=%s;brick=%s:%s timeout is %d")</font></div><div><font face="monospace, monospace" size="1"> at /home/jenkins/root/workspace/centos7-regression/libglusterfs/src/events.c:89</font></div><div><font face="monospace, monospace" size="1">#4 0x00007f400def07f9 in posix_fs_health_check (this=0x7f3fd78b7840) at /home/jenkins/root/workspace/centos7-regression/xlators/storage/posix/src/posix-helpers.c:1960</font></div><div><font face="monospace, monospace" size="1">#5 0x00007f400def0926 in posix_health_check_thread_proc (data=0x7f3fd78b7840)</font></div><div><font face="monospace, monospace" size="1"> at /home/jenkins/root/workspace/centos7-regression/xlators/storage/posix/src/posix-helpers.c:2005</font></div><div><font face="monospace, monospace" size="1">#6 0x00007f401a68ae25 in start_thread () from ./lib64/libpthread.so.0</font></div><div><font face="monospace, monospace" size="1">#7 0x00007f4019f53bad in clone () from ./lib64/libc.so.6</font></div><div><font face="monospace, monospace" size="1">(gdb) f 4</font></div><div><font face="monospace, monospace" size="1">#4 0x00007f400def07f9 in posix_fs_health_check (this=0x7f3fd78b7840) at /home/jenkins/root/workspace/centos7-regression/xlators/storage/posix/src/posix-helpers.c:1960</font></div><div><font face="monospace, monospace" size="1">1960<span style="white-space:pre"> </span> gf_event(EVENT_POSIX_HEALTH_CHECK_FAILED,</font></div><div><font face="monospace, monospace" size="1">(gdb) l</font></div><div><font face="monospace, monospace" size="1">1955<span style="white-space:pre"> </span> sys_close(fd);</font></div><div><font face="monospace, monospace" size="1">1956<span style="white-space:pre"> </span> }</font></div><div><font face="monospace, monospace" size="1">1957<span style="white-space:pre"> </span> if (ret && file_path[0]) {</font></div><div><font face="monospace, monospace" size="1">1958<span style="white-space:pre"> </span> gf_msg(this->name, GF_LOG_WARNING, errno, P_MSG_HEALTHCHECK_FAILED,</font></div><div><font face="monospace, monospace" size="1">1959<span style="white-space:pre"> </span> "%s() on %s returned", op, file_path);</font></div><div><font face="monospace, monospace" size="1">1960<span style="white-space:pre"> </span> gf_event(EVENT_POSIX_HEALTH_CHECK_FAILED,</font></div><div><font face="monospace, monospace" size="1">1961<span style="white-space:pre"> </span> "op=%s;path=%s;error=%s;brick=%s:%s timeout is %d", op,</font></div><div><font face="monospace, monospace" size="1">1962<span style="white-space:pre"> </span> file_path, strerror(op_errno), priv->hostname, priv->base_path,</font></div><div><font face="monospace, monospace" size="1">1963<span style="white-space:pre"> </span> timeout);</font></div><div><font face="monospace, monospace" size="1">1964<span style="white-space:pre"> </span> }</font></div><div><font face="monospace, monospace" size="1">(gdb) p pri->hostname</font></div><div><font face="monospace, monospace" size="1">No symbol "pri" in current context.</font></div><div><font face="monospace, monospace" size="1"><b>(gdb) p priv->hostname</b></font></div><div><font face="monospace, monospace" size="1"><b>$14 = 0xa200 <error: Cannot access memory at address 0xa200></b></font></div><div><font face="monospace, monospace" size="1"><b>(gdb) p priv->base_path</b></font></div><div><font face="monospace, monospace" size="1"><b>$15 = 0x7f3ddeadc0de00 <error: Cannot access memory at address 0x7f3ddeadc0de00></b></font></div><div><font face="monospace, monospace" size="1">(gdb) </font></div></div><div><br></div><div><br></div><div><br></div><div>Thanks,</div><div>Nithya</div></div></div>