<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 08/01/20 7:56 pm, lejeczek wrote:<br>
</div>
<blockquote type="cite"
cite="mid:bb701c63-d628-8439-6fc2-97d28a2f658b@yahoo.co.uk">
<pre class="moz-quote-pre" wrap="">On 08/01/2020 11:28, Ravishankar N wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">
On 08/01/20 3:55 pm, lejeczek wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">On 08/01/2020 02:08, Ravishankar N wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">On 07/01/20 8:07 pm, lejeczek wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">Which process should I be gdbing, selfheal's?
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">No the brick process on one of the nodes where file is missing.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">okey, would you mind showing exec/cmd for debug? I want to be sure you
get exactly what you need.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">Is it possible to give me temporary ssh access to your servers to
debug? If it is, then you can email me off-list with the login details
:-)
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">many thanks, L.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">oh no, I'm afraid not, no access from outside, it's public institution
here with very limited outside access, sorry.
But if you can share a all the CMDs you want me to do I'll try to get
you all the things.
many thanks, L.
</pre>
</blockquote>
<p>Okay, here goes:</p>
<tt>#gdb -p $brick_pid</tt><br>
<tt>(gdb) break server4_mknod_cbk</tt><br>
<tt>continue</tt>
<p>Once the break point is hit, <br>
</p>
<p><tt>(gdb) backtrace</tt><tt><br>
</tt><tt>#0 server4_mknod_cbk (frame=0x7f1ca40016a8,
cookie=0x7f1ca4006268, this=0x7f1cb402d7f0, <font
color="#ff0011">op_ret=0, op_errno=2</font>,
inode=0x7f1ca4006098, stbuf=0x7f1cc006e860,
preparent=0x7f1cc006e7c0, postparent=0x7f1cc006e720, xdata=0x0)</tt><tt><br>
</tt><tt> at server-rpc-fops_v2.c:538</tt><tt><br>
</tt><tt>#1 0x00007f1cc1aef27b in io_stats_mknod_cbk
(frame=0x7f1ca4006268, cookie=0x7f1ca4006638,
this=0x7f1cb402b170, op_ret=0, op_errno=2, inode=0x7f1ca4006098,
buf=0x7f1cc006e860, preparent=0x7f1cc006e7c0,
postparent=0x7f1cc006e720,</tt><tt><br>
</tt><tt> xdata=0x0) at io-stats.c:2251</tt><tt><br>
</tt><tt>#2 0x00007f1cc1b89641 in marker_mknod_cbk
(frame=0x7f1ca4006638, cookie=0x7f1ca4006748,
this=0x7f1cb4022520, op_ret=0, op_errno=2, inode=0x7f1ca4006098,
buf=0x7f1cc006e860, preparent=0x7f1cc006e7c0,
postparent=0x7f1cc006e720,</tt><tt><br>
</tt><tt> xdata=0x0) at marker.c:2061</tt><tt><br>
</tt><tt>#3 0x00007f1cd3d6f091 in default_mknod_cbk
(frame=0x7f1ca4006748, cookie=0x7f1cac001b28,
this=0x7f1cb401eba0, op_ret=0, op_errno=2, inode=0x7f1ca4006098,
buf=0x7f1cc006e860, preparent=0x7f1cc006e7c0,
postparent=0x7f1cc006e720,</tt><tt><br>
</tt><tt> xdata=0x0) at defaults.c:1327</tt><tt><br>
</tt><tt>#4 0x00007f1cc1bd267c in up_mknod_cbk
(frame=0x7f1cac001b28, cookie=0x7f1cac003f18,
this=0x7f1cb401ceb0, op_ret=0, op_errno=2, inode=0x7f1ca4006098,
buf=0x7f1cc006e860, preparent=0x7f1cc006e7c0,
postparent=0x7f1cc006e720, xdata=0x0)</tt><tt><br>
</tt><tt> at upcall.c:1021</tt><tt><br>
</tt><tt>#5 0x00007f1cc1c56285 in pl_mknod_cbk
(frame=0x7f1cac003f18, cookie=0x7f1cac002108,
this=0x7f1cb4015530, op_ret=0, op_errno=2, inode=0x7f1ca4006098,
buf=0x7f1cc006e860, preparent=0x7f1cc006e7c0,
postparent=0x7f1cc006e720, xdata=0x0)</tt><tt><br>
</tt><tt> at posix.c:4345</tt><tt><br>
</tt><tt>#6 0x00007f1cc1c8b8fb in posix_acl_mknod_cbk
(frame=0x7f1cac002108, cookie=0x7f1cac001678,
this=0x7f1cb4013950, op_ret=0, op_errno=2, inode=0x7f1ca4006098,
buf=0x7f1cc006e860, preparent=0x7f1cc006e7c0,
postparent=0x7f1cc006e720,</tt><tt><br>
</tt><tt> xdata=0x0) at posix-acl.c:1323</tt><tt><br>
</tt><tt>#7 0x00007f1cc1cb19ee in br_stub_mknod_cbk
(frame=0x7f1cac001678, cookie=0x7f1cac001ff8,
this=0x7f1cb4011990, op_ret=0, op_errno=2, inode=0x7f1ca4006098,
stbuf=0x7f1cc006e860, preparent=0x7f1cc006e7c0,
postparent=0x7f1cc006e720,</tt><tt><br>
</tt><tt> xdata=0x0) at bit-rot-stub.c:2629</tt><tt><br>
</tt><tt>#8 0x00007f1cc1ccb05f in changelog_mknod_cbk
(frame=0x7f1cac001ff8, cookie=0x7f1cac0023e8,
this=0x7f1cb400f7d0, op_ret=0, op_errno=2, inode=0x7f1ca4006098,
buf=0x7f1cc006e860, preparent=0x7f1cc006e7c0,
postparent=0x7f1cc006e720,</tt><tt><br>
</tt><tt> xdata=0x0) at changelog.c:818</tt><tt><br>
</tt><tt>#9 0x00007f1cc1d37a77 in posix_mknod
(frame=0x7f1cac0023e8, this=0x7f1cb400a360, loc=0x7f1ca4002ab0,
mode=33188, dev=0, umask=0, xdata=0x7f1ca40022e8) at
posix-entry-ops.c:557</tt><tt><br>
</tt><tt>#10 0x00007f1cd3d85a5f in default_mknod
(frame=0x7f1cac0023e8, this=0x7f1cb400d600, loc=0x7f1ca4002ab0,
mode=33188, rdev=0, umask=0, xdata=0x7f1ca40022e8) at
defaults.c:2710</tt><tt><br>
</tt><tt>#11 0x00007f1cc1ccbfdf in changelog_mknod
(frame=0x7f1cac001ff8, this=0x7f1cb400f7d0, loc=0x7f1ca4002ab0,
mode=33188, dev=0, umask=0, xdata=0x7f1ca40022e8) at
changelog.c:939</tt><tt><br>
</tt><tt>#12 0x00007f1cc1cb1ef9 in br_stub_mknod
(frame=0x7f1cac001678, this=0x7f1cb4011990, loc=0x7f1ca4002ab0,
mode=33188, dev=0, umask=0, xdata=0x7f1ca40022e8) at
bit-rot-stub.c:2642</tt><tt><br>
</tt><tt>#13 0x00007f1cc1c8bd28 in posix_acl_mknod
(frame=0x7f1cac002108, this=0x7f1cb4013950, loc=0x7f1ca4002ab0,
mode=33188, rdev=0, umask=0, xdata=0x7f1ca40022e8) at
posix-acl.c:1342</tt><tt><br>
</tt><tt>#14 0x00007f1cc1c567c7 in pl_mknod (frame=0x7f1cac003f18,
this=0x7f1cb4015530, loc=0x7f1ca4002ab0, mode=33188, rdev=0,
umask=0, xdata=0x7f1ca40022e8) at posix.c:4355</tt><tt><br>
</tt><tt>#15 0x00007f1cd3d85a5f in default_mknod
(frame=0x7f1cac003f18, this=0x7f1cb40172a0, loc=0x7f1ca4002ab0,
mode=33188, rdev=0, umask=0, xdata=0x7f1ca40022e8) at
defaults.c:2710</tt><tt><br>
</tt><tt>#16 0x00007f1cc1c0d857 in ro_mknod (frame=0x7f1cac003f18,
this=0x7f1cb4019430, loc=0x7f1ca4002ab0, mode=33188, rdev=0,
umask=0, xdata=0x7f1ca40022e8) at read-only-common.c:209</tt><tt><br>
</tt><tt>#17 0x00007f1cd3d85a5f in default_mknod
(frame=0x7f1cac003f18, this=0x7f1cb401b170, loc=0x7f1ca4002ab0,
mode=33188, rdev=0, umask=0, xdata=0x7f1ca40022e8) at
defaults.c:2710</tt><tt><br>
</tt><tt>#18 0x00007f1cc1bd2ae7 in up_mknod (frame=0x7f1cac001b28,
this=0x7f1cb401ceb0, loc=0x7f1ca4002ab0, mode=33188, rdev=0,
umask=0, xdata=0x7f1ca40022e8) at upcall.c:1042</tt><tt><br>
</tt><tt>#19 0x00007f1cd3d7b1a8 in default_mknod_resume
(frame=0x7f1ca4006748, this=0x7f1cb401eba0, loc=0x7f1ca4002ab0,
mode=33188, rdev=0, umask=0, xdata=0x7f1ca40022e8) at
defaults.c:1961</tt><tt><br>
</tt><tt>#20 0x00007f1cd3cc8046 in call_resume_wind
(stub=0x7f1ca4002a68) at call-stub.c:2046</tt><tt><br>
</tt><tt>#21 0x00007f1cd3cda128 in call_resume
(stub=0x7f1ca4002a68) at call-stub.c:2555</tt><tt><br>
</tt><tt>#22 0x00007f1cc1bb7372 in iot_worker
(data=0x7f1cb405a7a0) at io-threads.c:232</tt><tt><br>
</tt><tt>#23 0x00007f1cd3a114e2 in start_thread () from
/lib64/libpthread.so.0</tt><tt><br>
</tt><tt>#24 0x00007f1cd3660693 in clone () from /lib64/libc.so.6</tt></p>
<p>Note that in the backtrace above, <font color="#ff0011">op_ret</font>=0.
i.e. the mknod was successful. In your case you should see <font
color="#ff0011">op_ret</font>=-1 and <font color="#ff0011">op_errno</font>=13
probably when the file default.sock tries to get created.<br>
</p>
<p>Thanks,</p>
Ravi<br>
</body>
</html>