<div dir="ltr">Without the glusterd log file and the core file or the backtrace I can't comment anything.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 6, 2017 at 3:09 PM, ABHISHEK PALIWAL <span dir="ltr"><<a href="mailto:abhishpaliwal@gmail.com" target="_blank">abhishpaliwal@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Any suggestion....</div><div class="gmail_extra"><br><div class="gmail_quote">On Dec 6, 2017 11:51, "ABHISHEK PALIWAL" <<a href="mailto:abhishpaliwal@gmail.com" target="_blank">abhishpaliwal@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Team,<div><br></div><div>We are getting the crash in glusterd after start of it. When I tried to debug in brick logs we are getting below errors:</div><div><br></div><div>[2017-12-01 14:10:14.684122] E [MSGID: 100018] [glusterfsd.c:1960:glusterfs_p<wbr>idfile_update] 0-glusterfsd: pidfile /system/glusterd/vols/c_gluste<wbr>rfs/run/10.32.1.144-opt-lvmdir<wbr>-c2-brick.pid lock failed [Resource temporarily unavailable]<br></div><div>:</div><div>:</div><div><div>:</div><div><div>[2017-12-01 14:10:16.862903] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=18: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:16.862985] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 92: FTRUNCATE 1 (934f08b7-e3b5-4690-84fc-742a4<wbr>b1fb78b)==> (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:16.907037] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=17: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:16.907108] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 35: FTRUNCATE 0 (109d6537-a1ec-4556-8ce1-04c36<wbr>5c451eb)==> (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:16.947541] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=17: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:16.947623] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 70: FTRUNCATE 0 (8f9c8054-b0d7-4b93-a95b-cd3ab<wbr>249c56d)==> (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:16.968515] E [MSGID: 113001] [posix.c:4616:_posix_remove_xa<wbr>ttr] 0-c_glusterfs-posix: removexattr failed on /opt/lvmdir/c2/brick/.glusterf<wbr>s/00/00/00000000-0000-0000-<wbr>0000-000000000001/configuratio<wbr>n (for trusted.glusterfs.dht) [No space left on device]</div><div>[2017-12-01 14:10:16.968589] I [MSGID: 115058] [server-rpc-fops.c:740:server_<wbr>removexattr_cbk] 0-c_glusterfs-server: 90: REMOVEXATTR <gfid:a240d2fd-869c-408d-9b95-<wbr>62ee1bff074e> (a240d2fd-869c-408d-9b95-62ee1<wbr>bff074e) of key ==> (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:17.039815] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=17: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:17.039900] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 152: FTRUNCATE 0 (d67bcfcd-ff19-4b58-9823-46d6c<wbr>ce9ace3)==> (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:17.048767] E [MSGID: 113001] [posix-helpers.c:1228:posix_fh<wbr>andle_pair] 0-c_glusterfs-posix: fd=17: key:trusted.bit-rot.version [No space left on device]</div><div>[2017-12-01 14:10:17.048874] I [MSGID: 115063] [server-rpc-fops.c:1317:server<wbr>_ftruncate_cbk] 0-c_glusterfs-server: 163: FTRUNCATE 0 (0e3ee6ad-408b-4fcf-a1a7-4262e<wbr>c113316)==> (No space left on device) [No space left on device]</div><div>[2017-12-01 14:10:17.075007] E [MSGID: 113001] [posix.c:4616:_posix_remove_xa<wbr>ttr] 0-c_glusterfs-posix: removexattr failed on /opt/lvmdir/c2/brick/.glusterf<wbr>s/00/00/00000000-0000-0000-<wbr>0000-000000000001/java (for trusted.glusterfs.dht) [No space left on device]</div></div><div><br></div><div>Also, we are having the lack disk space.</div><div><br></div><div>Could any one please explain me what glusterd is doing in brick so that it is causing of its crash.</div><div><br></div><div>Please find the brick logs in attachment.</div><div><br></div><div>Thanks in advance!!!</div><span class="HOEnZb"><font color="#888888">-- <br><div class="m_7987781935775733261m_3854133520433708890gmail_signature"><div dir="ltr">Regards<br>
Abhishek Paliwal<br>
</div></div>
</font></span></div></div>
</blockquote></div></div>
<br>______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-devel</a><br></blockquote></div><br></div>