<div dir="ltr"><div>[2017-12-01 14:10:13.060193] I [MSGID: 100030] [glusterfsd.c:2348:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd       version 3.7.6 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)<br><br></div>From the above I can see you&#39;re running a way too old gluster version which has been EOLed. Please upgrade your cluster to the latest supported versions.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 6, 2017 at 5:39 PM, ABHISHEK PALIWAL <span dir="ltr">&lt;<a href="mailto:abhishpaliwal@gmail.com" target="_blank">abhishpaliwal@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I hope these logs were sufficient... please let me know if you require more logs.</div><div class="gmail_extra"><div><div class="h5"><br><div class="gmail_quote">On Wed, Dec 6, 2017 at 3:26 PM, ABHISHEK PALIWAL <span dir="ltr">&lt;<a href="mailto:abhishpaliwal@gmail.com" target="_blank">abhishpaliwal@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Atin,<div><br></div><div>Please find the backtrace and logs files attached here.</div><div><br></div><div>Also, below are the BT from core.</div><div><br></div><div><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">(gdb) bt</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#0  0x00003fff8834b898 in __GI_raise (sig=&lt;optimized out&gt;) at ../sysdeps/unix/sysv/linux/rai<wbr>se.c:55</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#1  0x00003fff88350fd0 in __GI_abort () at abort.c:89</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"> </p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">[**ALERT: The abort() might not be exactly invoked from the following function line.</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">                If the trail function contains multiple abort() calls, then you should cross check by other means to get correct abort() call location.</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">                This is due to the optimized compilation which hides the debug info for multiple abort() calls in a given function.</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">                Refer TR HU16995 for more information]</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"> </p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#2  0x00003fff8838be04 in __libc_message (do_abort=&lt;optimized out&gt;, fmt=&lt;optimized out&gt;) at ../sysdeps/posix/libc_fatal.c:<wbr>175</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#3  0x00003fff8839aba8 in malloc_printerr (action=&lt;optimized out&gt;, str=0x3fff8847e498 &quot;double free or corruption (!prev)&quot;, ptr=&lt;optimized out&gt;, ar_ptr=&lt;optimized out&gt;) at malloc.c:5007</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#4  0x00003fff8839ba40 in _int_free (av=0x3fff6c000020, p=&lt;optimized out&gt;, have_lock=&lt;optimized out&gt;) at malloc.c:3868</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#5  0x00003fff885e0814 in __gf_free (free_ptr=0x3fff6c045da0) at mem-pool.c:336</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#6  0x00003fff849093c4 in glusterd_friend_sm () at glusterd-sm.c:1295</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#7  0x00003fff84901a58 in __glusterd_handle_incoming_unf<wbr>riend_req (req=0x3fff8481c06c) at glusterd-handler.c:2606</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#8  0x00003fff848fb870 in glusterd_big_locked_handler (req=0x3fff8481c06c, actor_fn=@0x3fff84a43e70: 0x3fff84901830 &lt;__glusterd_handle_incoming_un<wbr>friend_req&gt;) at glusterd-handler.c:83</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#9  0x00003fff848fbd08 in glusterd_handle_incoming_unfri<wbr>end_req (req=&lt;optimized out&gt;) at glusterd-handler.c:2615</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#10 0x00003fff8854e87c in rpcsvc_handle_rpc_call (svc=0x10062fd0 &lt;_GLOBAL__sub_I__ZN27UehChSwit<wbr>chFachToDchC_ActorC2EP12RTCont<wbr>rollerP10RTActorRef()+1148&gt;, trans=&lt;optimized out&gt;, msg=0x3fff6c000920) at rpcsvc.c:705</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#11 0x00003fff8854eb7c in rpcsvc_notify (trans=0x3fff74002210, mydata=&lt;optimized out&gt;, event=&lt;optimized out&gt;, data=&lt;optimized out&gt;) at rpcsvc.c:799</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#12 0x00003fff885514fc in rpc_transport_notify (this=&lt;optimized out&gt;, event=&lt;optimized out&gt;, data=&lt;optimized out&gt;) at rpc-transport.c:546</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#13 0x00003fff847fcd44 in socket_event_poll_in (this=this@entry=0x3fff7400221<wbr>0) at socket.c:2236</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#14 0x00003fff847ff89c in socket_event_handler (fd=&lt;optimized out&gt;, idx=&lt;optimized out&gt;, data=0x3fff74002210, poll_in=&lt;optimized out&gt;, poll_out=&lt;optimized out&gt;, poll_err=&lt;optimized out&gt;) at socket.c:2349</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#15 0x00003fff88616874 in event_dispatch_epoll_handler (event=0x3fff83d9d6a0, event_pool=0x10045bc0 &lt;_GLOBAL__sub_I__ZN29DrhIfRhCo<wbr>ntrolPdrProxyC_ActorC2EP12RTCo<wbr>ntrollerP10RTActorRef()+116&gt;) at event-epoll.c:575</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#16 event_dispatch_epoll_worker (data=0x100bb4a0 &lt;main_thread_func__()+1756&gt;) at event-epoll.c:678</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#17 0x00003fff884cfb10 in start_thread (arg=0x3fff83d9e160) at pthread_create.c:339</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#18 0x00003fff88419c0c in .__clone () at ../sysdeps/unix/sysv/linux/pow<wbr>erpc/powerpc64/clone.S:96</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"> </p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">(gdb) bt full</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#0  0x00003fff8834b898 in __GI_raise (sig=&lt;optimized out&gt;) at ../sysdeps/unix/sysv/linux/rai<wbr>se.c:55</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        <span lang="SV">r4 = 1560</span></p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span lang="SV">        r7 = 16</span></p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span lang="SV">        arg2 = 1560</span></p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span lang="SV">        r5 = 6</span></p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span lang="SV">        r8 = 0</span></p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span lang="SV">        arg3 = 6</span></p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span lang="SV">        </span>r0 = 250</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        r3 = 0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        r6 = 8</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        arg1 = 0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        sc_err = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        sc_ret = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        pd = 0x3fff83d9e160</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        pid = 0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        selftid = 1560</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#1  0x00003fff88350fd0 in __GI_abort () at abort.c:89</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        save_stage = 2</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        <span lang="SV">act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = {0 &lt;repeats 16 times&gt;}}, sa_flags = 0, sa_restorer = 0x0}</span></p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span lang="SV">        </span>sigs = {__val = {32, 0 &lt;repeats 15 times&gt;}}</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"> </p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">[**ALERT: The abort() might not be exactly invoked from the following function line.</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">                If the trail function contains multiple abort() calls, then you should cross check by other means to get correct abort() call location.</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">                This is due to the optimized compilation which hides the debug info for multiple abort() calls in a given function.</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">                Refer TR HU16995 for more information]</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"> </p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#2  0x00003fff8838be04 in __libc_message (do_abort=&lt;optimized out&gt;, fmt=&lt;optimized out&gt;) at ../sysdeps/posix/libc_fatal.c:<wbr>175</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ap = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        fd = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        on_2 = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        list = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        nlist = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        cp = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        written = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#3  0x00003fff8839aba8 in malloc_printerr (action=&lt;optimized out&gt;, str=0x3fff8847e498 &quot;<b>double free or corruption (!prev)</b>&quot;, ptr=&lt;optimized out&gt;, ar_ptr=&lt;optimized out&gt;) at malloc.c:5007</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        buf = &quot;00003fff6c045d60&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        cp = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ar_ptr = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ptr = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        str = 0x3fff8847e498 &quot;double free or corruption (!prev)&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        action = 3</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#4  0x00003fff8839ba40 in _int_free (av=0x3fff6c000020, p=&lt;optimized out&gt;, have_lock=&lt;optimized out&gt;) at malloc.c:3868</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        size = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        fb = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        nextchunk = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        nextsize = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        nextinuse = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        prevsize = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        bck = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        fwd = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        errstr = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        locked = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __func__ = &quot;_int_free&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#5  0x00003fff885e0814 in __gf_free (free_ptr=0x3fff6c045da0) at mem-pool.c:336</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ptr = 0x3fff6c045d60</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        mem_acct = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        header = 0x3fff6c045d60</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        free_ptr = 0x3fff6c045da0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#6  0x00003fff849093c4 in glusterd_friend_sm () at glusterd-sm.c:1295</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        event = 0x3fff6c045da0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        tmp = 0x3fff6c045da0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        handler = @0x3fff84a44038: 0x3fff84906750 &lt;glusterd_ac_friend_remove&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        state = 0x3fff84a390c0 &lt;glusterd_state_befriended&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        peerinfo = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        event_type = GD_FRIEND_EVENT_REMOVE_FRIEND</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        is_await_conn = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        quorum_action = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        old_state = GD_FRIEND_STATE_BEFRIENDED</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        this = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        priv = 0x3fff84748050</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __FUNCTION__ = &quot;glusterd_friend_sm&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#7  0x00003fff84901a58 in __glusterd_handle_incoming_unf<wbr>riend_req (req=0x3fff8481c06c) at glusterd-handler.c:2606</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = 0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        friend_req = {uuid = &quot;\231\214R¦\177\223I\216\236Õ\<wbr>214dÎöy¡&quot;, hostname = 0x3fff6c028ef0 &quot;&quot;, port = 0, vols = {vols_len = 0, vols_val = 0x0}}</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        remote_hostname = &quot;10.32.0.48&quot;, &#39;\000&#39; &lt;repeats 98 times&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __FUNCTION__ = &quot;__glusterd_handle_incoming_un<wbr>friend_req&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#8  0x00003fff848fb870 in glusterd_big_locked_handler (req=0x3fff8481c06c, actor_fn=@0x3fff84a43e70: 0x3fff84901830 &lt;__glusterd_handle_incoming_un<wbr>friend_req&gt;) at glusterd-handler.c:83</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        priv = 0x3fff84748050</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = -1</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#9  0x00003fff848fbd08 in glusterd_handle_incoming_unfri<wbr>end_req (req=&lt;optimized out&gt;) at glusterd-handler.c:2615</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">No locals.</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#10 0x00003fff8854e87c in rpcsvc_handle_rpc_call (svc=0x10062fd0 &lt;_GLOBAL__sub_I__ZN27UehChSwit<wbr>chFachToDchC_ActorC2EP12RTCont<wbr>rollerP10RTActorRef()+1148&gt;, trans=&lt;optimized out&gt;, msg=0x3fff6c000920) at rpcsvc.c:705</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        actor = 0x3fff84a38860 &lt;gd_svc_peer_actors+192&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        actor_fn = @0x3fff84a43ab0: 0x3fff848fbcf0 &lt;glusterd_handle_incoming_unfr<wbr>iend_req&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        req = 0x3fff8481c06c</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = -1</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        port = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        unprivileged = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        reply = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        drc = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __FUNCTION__ = &quot;rpcsvc_handle_rpc_call&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#11 0x00003fff8854eb7c in rpcsvc_notify (trans=0x3fff74002210, mydata=&lt;optimized out&gt;, event=&lt;optimized out&gt;, data=&lt;optimized out&gt;) at rpcsvc.c:799</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = -1</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        msg = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        new_trans = 0x0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        svc = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        listener = 0x0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __FUNCTION__ = &quot;rpcsvc_notify&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#12 0x00003fff885514fc in rpc_transport_notify (this=&lt;optimized out&gt;, event=&lt;optimized out&gt;, data=&lt;optimized out&gt;) at rpc-transport.c:546</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = -1</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __FUNCTION__ = &quot;rpc_transport_notify&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#13 0x00003fff847fcd44 in socket_event_poll_in (this=this@entry=0x3fff7400221<wbr>0) at socket.c:2236</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        pollin = 0x3fff6c000920</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        priv = 0x3fff74002d50</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#14 0x00003fff847ff89c in socket_event_handler (fd=&lt;optimized out&gt;, idx=&lt;optimized out&gt;, data=0x3fff74002210, poll_in=&lt;optimized out&gt;, poll_out=&lt;optimized out&gt;, poll_err=&lt;optimized out&gt;) at socket.c:2349</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        this = 0x3fff74002210</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        priv = 0x3fff74002d50</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __FUNCTION__ = &quot;socket_event_handler&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#15 0x00003fff88616874 in event_dispatch_epoll_handler (event=0x3fff83d9d6a0, event_pool=0x10045bc0 &lt;_GLOBAL__sub_I__ZN29DrhIfRhCo<wbr>ntrolPdrProxyC_ActorC2EP12RTCo<wbr>ntrollerP10RTActorRef()+116&gt;) at event-epoll.c:575</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        <span lang="SV">handler = @0x3fff8481a620: 0x3fff847ff6f0 &lt;socket_event_handler&gt;</span></p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span lang="SV">        </span>gen = 1</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        slot = 0x100803f0 &lt;_GLOBAL__sub_I__ZN24RoamIfFro<wbr>RrcRoExtAttribDC2Ev()+232&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        data = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = -1</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        fd = 8</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ev_data = 0x3fff83d9d6a8</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        idx = 7</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#16 event_dispatch_epoll_worker (data=0x100bb4a0 &lt;main_thread_func__()+1756&gt;) at event-epoll.c:678</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        event = {events = 1, data = {ptr = 0x700000001, fd = 7, u32 = 7, u64 = 30064771073}}</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ret = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        ev_data = 0x100bb4a0 &lt;main_thread_func__()+1756&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        event_pool = 0x10045bc0 &lt;_GLOBAL__sub_I__ZN29DrhIfRhCo<wbr>ntrolPdrProxyC_ActorC2EP12RTCo<wbr>ntrollerP10RTActorRef()+116&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        myindex = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        timetodie = 0</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __FUNCTION__ = &quot;event_dispatch_epoll_worker&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#17 0x00003fff884cfb10 in start_thread (arg=0x3fff83d9e160) at pthread_create.c:339</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        pd = 0x3fff83d9e160</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        now = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-6868946778599096053, 70366736145408, -6868946778421678961, 0, 0, 70366652919808, 70366661304864, 8388608, 70366736105504, 269202592, 70367897957648, 70366736131032, 70366737568040, 3, 0, 70366736131048, 70367897957296, 70367897957352, 4001536, 70366736106520, 70366661302080, -3187654076, 0 &lt;repeats 42 times&gt;}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        not_first_call = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        pagesize_m1 = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        sp = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        freesize = &lt;optimized out&gt;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">        __PRETTY_FUNCTION__ = &quot;start_thread&quot;</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">#18 0x00003fff88419c0c in .__clone () at ../sysdeps/unix/sysv/linux/pow<wbr>erpc/powerpc64/clone.S:96</p><p class="m_164272196664377004m_-4261569791629226590gmail-x_MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">No locals.</p></div><div><br></div><div>Regards,</div><div>Abhishek</div></div><div class="gmail_extra"><div><div class="m_164272196664377004h5"><br><div class="gmail_quote">On Wed, Dec 6, 2017 at 3:21 PM, Atin Mukherjee <span dir="ltr">&lt;<a href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Without the glusterd log file and the core file or the backtrace I can&#39;t comment anything.<br></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_164272196664377004m_-4261569791629226590h5">On Wed, Dec 6, 2017 at 3:09 PM, ABHISHEK PALIWAL <span dir="ltr">&lt;<a href="mailto:abhishpaliwal@gmail.com" target="_blank">abhishpaliwal@gmail.com</a>&gt;</span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_164272196664377004m_-4261569791629226590h5"><div dir="auto">Any suggestion....</div><div class="gmail_extra"><br><div class="gmail_quote">On Dec 6, 2017 11:51, &quot;ABHISHEK PALIWAL&quot; &lt;<a href="mailto:abhishpaliwal@gmail.com" target="_blank">abhishpaliwal@gmail.com</a>&gt; wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Team,<div><br></div><div>We are getting the crash in glusterd after start of it. When I tried to debug in brick logs we are getting below errors:</div><div><br></div><div>[2017-12-01 14:10:14.684122] E [MSGID: 100018] [glusterfsd.c:1960:glusterfs_p<wbr>idfile_update] 0-glusterfsd: pidfile /system/glusterd/vols/c_gluste<wbr>rfs/run/10.32.1.144-opt-lvmdir<wbr>-c2-brick.pid lock failed [Resource temporarily unavailable]<br></div><div>:</div><div>:</div><div><div>:</div><div><div>[2017-12-01 14:10:16.862903] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=18: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:16.862985] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 92: FTRUNCATE 1 (934f08b7-e3b5-4690-84fc-742a4<wbr>b1fb78b)==&gt; (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:16.907037] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=17: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:16.907108] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 35: FTRUNCATE 0 (109d6537-a1ec-4556-8ce1-04c36<wbr>5c451eb)==&gt; (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:16.947541] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=17: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:16.947623] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 70: FTRUNCATE 0 (8f9c8054-b0d7-4b93-a95b-cd3ab<wbr>249c56d)==&gt; (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:16.968515] E [MSGID: 113001] [posix.c:4616:_posix_remove_xa<wbr>ttr] 0-c_glusterfs-posix: removexattr failed on /opt/lvmdir/c2/brick/.glusterf<wbr>s/00/00/00000000-0000-0000-000<wbr>0-000000000001/configuration (for trusted.glusterfs.dht) [No space left on device]</div><div>[2017-12-01 14:10:16.968589] I [MSGID: 115058] [server-rpc-fops.c:740:server_<wbr>removexattr_cbk] 0-c_glusterfs-server: 90: REMOVEXATTR &lt;gfid:a240d2fd-869c-408d-9b95-<wbr>62ee1bff074e&gt; (a240d2fd-869c-408d-9b95-62ee1<wbr>bff074e) of key  ==&gt; (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:17.039815] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=17: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:17.039900] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 152: FTRUNCATE 0 (d67bcfcd-ff19-4b58-9823-46d6c<wbr>ce9ace3)==&gt; (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:17.048767] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=17: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:17.048874] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 163: FTRUNCATE 0 (0e3ee6ad-408b-4fcf-a1a7-4262e<wbr>c113316)==&gt; (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:17.075007] E [MSGID: 113001] [posix.c:4616:_posix_remove_xa<wbr>ttr] 0-c_glusterfs-posix: removexattr failed on /opt/lvmdir/c2/brick/.glusterf<wbr>s/00/00/00000000-0000-0000-000<wbr>0-000000000001/java (for trusted.glusterfs.dht) [No space left on device]</div></div><div><br></div><div>Also, we are having the lack disk space.</div><div><br></div><div>Could any one please explain me what glusterd is doing in brick so that it is causing of its crash.</div><div><br></div><div>Please find the brick logs in attachment.</div><div><br></div><div>Thanks in advance!!!</div><span class="m_164272196664377004m_-4261569791629226590m_2113983986184265794HOEnZb"><font color="#888888">-- <br><div class="m_164272196664377004m_-4261569791629226590m_2113983986184265794m_7987781935775733261m_3854133520433708890gmail_signature"><div dir="ltr">Regards<br>
Abhishek Paliwal<br>
</div></div>
</font></span></div></div>
</blockquote></div></div>
<br></div></div>______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-devel</a><br></blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span class="m_164272196664377004HOEnZb"><font color="#888888">-- <br><div class="m_164272196664377004m_-4261569791629226590gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><br><br><br><br>Regards<br>
Abhishek Paliwal<br>
</div></div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span class="HOEnZb"><font color="#888888">-- <br><div class="m_164272196664377004gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><br><br><br><br>Regards<br>
Abhishek Paliwal<br>
</div></div>
</font></span></div>
</blockquote></div><br></div>