[Gluster-users] GlusterFS 3.7.11 crash issue

Atin Mukherjee amukherj at redhat.com
Mon Jun 27 08:24:01 UTC 2016 

+Anoop, Jiffin

On Mon, Jun 27, 2016 at 1:50 PM, Yann LEMARIE <ylemarie at iraiser.eu> wrote:

> Thanks Atin,
>
> Here is the log with "backtrace"
>
> time of crash:
> 2016-06-26 09:27:44
> configuration details:
> argp 1
> backtrace 1
> dlfcn 1
> libpthread 1
> llistxattr 1
> setfsid 1
> spinlock 1
> epoll.h 1
> xattr.h 1
> st_atim.tv_nsec 1
> package-string: glusterfs 3.7.11
>
> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_msg_backtrace_nomem+0x92)[0x7f2626e7a5e2]
>
> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_print_trace+0x31d)[0x7f2626e966cd]
> /lib/x86_64-linux-gnu/libc.so.6(+0x36cb0)[0x7f2626278cb0]
> /lib/x86_64-linux-gnu/libc.so.6(strlen+0x2a)[0x7f26262ca9da]
>
> /usr/lib/x86_64-linux-gnu/glusterfs/3.7.11/xlator/features/trash.so(trash_truncate_stat_cbk+0x279)[0x7f2620c7a909]
>
>
> I have lot of message about [File exist] or [No such file or directory]
> within the .trash directory, is ther any link with the crash ?
>
> [2016-06-26 09:27:37.965461] E [MSGID: 113027] [posix.c:1427:posix_mkdir]
> 0-cdn-posix: mkdir of /srv/cdn/ failed [File exists]
> [2016-06-26 09:27:37.965479] E [MSGID: 113027] [posix.c:1427:posix_mkdir]
> 0-cdn-posix: mkdir of /srv/cdn/.trashcan failed [File exists]
> [2016-06-26 09:27:37.965495] E [MSGID: 113027] [posix.c:1427:posix_mkdir]
> 0-cdn-posix: mkdir of /srv/cdn/.trashcan/ failed [File exists]
> [2016-06-26 09:27:37.965512] E [MSGID: 113027] [posix.c:1427:posix_mkdir]
> 0-cdn-posix: mkdir of /srv/cdn/.trashcan//medias failed [File exists]
> [2016-06-26 09:27:37.965570] E [MSGID: 113001] [posix.c:1451:posix_mkdir]
> 0-cdn-posix: setting xattrs on
> /srv/cdn/.trashcan//medias/fjDtE6cAz80xweuAdoIcWCDYxo9FdPqIVL88Yrl8ZiGVIvdiyBD5u1pJUXBzYJ5b
> failed [No such file or directory]
> [2016-06-26 09:27:37.965615] E [MSGID: 113001] [posix.c:1451:posix_mkdir]
> 0-cdn-posix: setting xattrs on
> /srv/cdn/.trashcan//medias/fjDtE6cAz80xweuAdoIcWCDYxo9FdPqIVL88Yrl8ZiGVIvdiyBD5u1pJUXBzYJ5b/Dante_Olive
> failed [No such file or directory]
> [2016-06-26 09:27:37.965659] E [MSGID: 113001] [posix.c:1451:posix_mkdir]
> 0-cdn-posix: setting xattrs on
> /srv/cdn/.trashcan//medias/fjDtE6cAz80xweuAdoIcWCDYxo9FdPqIVL88Yrl8ZiGVIvdiyBD5u1pJUXBzYJ5b/Dante_Olive/origin
> failed [No such file or directory]
> [2016-06-26 09:27:37.965683] E [MSGID: 113027] [posix.c:1427:posix_mkdir]
> 0-cdn-posix: mkdir of
> /srv/cdn/.trashcan//medias/fjDtE6cAz80xweuAdoIcWCDYxo9FdPqIVL88Yrl8ZiGVIvdiyBD5u1pJUXBzYJ5b/Dante_Olive/origin
> failed [File exists]
> [2016-06-26 09:27:37.965720] E [MSGID: 113001] [posix.c:1451:posix_mkdir]
> 0-cdn-posix: setting xattrs on
> /srv/cdn/.trashcan//medias/fjDtE6cAz80xweuAdoIcWCDYxo9FdPqIVL88Yrl8ZiGVIvdiyBD5u1pJUXBzYJ5b/Dante_Olive/origin/tmp
> failed [No such file or directory]
> [2016-06-26 09:27:37.965767] E [MSGID: 113001] [posix.c:2644:posix_create]
> 0-cdn-posix: setting xattrs on
> /srv/cdn/.trashcan//medias/fjDtE6cAz80xweuAdoIcWCDYxo9FdPqIVL88Yrl8ZiGVIvdiyBD5u1pJUXBzYJ5b/Dante_Olive/origin/tmp/IMG3920.JPG_2016-06-26_092737
> failed  [No such file or directory]
> [2016-06-26 09:27:37.965955] E [posix.c:2996:_fill_writev_xdata]
> (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.11/xlator/features/trash.so(trash_truncate_readv_cbk+0x16c)
> [0x7f2620c6fc4c]
> -->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.11/xlator/storage/posix.so(posix_writev+0x1dc)
> [0x7f26214ae37c]
> -->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.11/xlator/storage/posix.so(_fill_writev_xdata+0x1ff)
> [0x7f26214ae16f] ) 0-cdn-posix: fd: 0x7f26140057fc inode:
> 0x7f2600932484gfid:00000000-0000-0000-0000-000000000000 [Invalid argument]
> [2016-06-26 09:27:37.966096] E [posix.c:2996:_fill_writev_xdata]
> (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.11/xlator/features/trash.so(trash_truncate_readv_cbk+0x16c)
> [0x7f2620c6fc4c]
> -->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.11/xlator/storage/posix.so(posix_writev+0x1dc)
> [0x7f26214ae37c]
> -->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.11/xlator/storage/posix.so(_fill_writev_xdata+0x1ff)
> [0x7f26214ae16f] ) 0-cdn-posix: fd: 0x7f26140057fc inode:
> 0x7f2600932484gfid:00000000-0000-0000-0000-000000000000 [Invalid argument]
>
>
> Yann
>
>
> Le 27/06/2016 10:02, Atin Mukherjee a écrit :
>
> Grep for "backtrace" in /var/log/glusterfs, that should give you a
> particular log file name by which you can determine which process crashed.
> You should also be able to see a coredump. Why don't you attach the same
> for further analysis?
>
> ~Atin
>
> On Mon, Jun 27, 2016 at 1:17 PM, Yann LEMARIE <ylemarie at iraiser.eu> wrote:
>
>> Hi,
>>
>> I'm using GlusterFS since many years and never see this problem, but this
>> is the second time in one week ...
>>
>> I have 3 volumes with 2 bricks and 1 volume crash with no reason, I just
>> have to stop/start the volume to make it up again.
>> The only logs I can find are in syslog :
>>
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: pending frames:
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: frame : type(0) op(10)
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: patchset: git://
>> git.gluster.com/glusterfs.git
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: signal received: 11
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: time of crash:
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: 2016-06-26 09:27:44
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: configuration details:
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: argp 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: backtrace 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: dlfcn 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: libpthread 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: llistxattr 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: setfsid 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: spinlock 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: epoll.h 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: xattr.h 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: st_atim.tv_nsec 1
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: package-string: glusterfs 3.7.11
>> Jun 26 11:27:44 nfs05 srv-cdn[7879]: ---------
>>
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: pending frames:
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: frame : type(0) op(10)
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: patchset: git://
>> git.gluster.com/glusterfs.git
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: signal received: 11
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: time of crash:
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: 2016-06-26 09:27:44
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: configuration details:
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: argp 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: backtrace 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: dlfcn 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: libpthread 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: llistxattr 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: setfsid 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: spinlock 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: epoll.h 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: xattr.h 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: st_atim.tv_nsec 1
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: package-string: glusterfs 3.7.11
>> Jun 26 11:27:44 nfs06 srv-cdn[1787]: ---------
>>
>>
>> Thanks for your help
>>
>>
>> Regards
>> --
>> Yann Lemarié
>> iRaiser - Support Technique
>> [image: iRaiser Logotype] <http://www.iraiser.eu>
>> <ylemarie at iraiser.eu>ylemarie at iraiser.eu
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-users
>>
>
>
> --
> Yann Lemarié
> iRaiser - Support Technique
> [image: iRaiser Logotype] <http://www.iraiser.eu>
> <ylemarie at iraiser.eu>ylemarie at iraiser.eu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160627/7dbf81f1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dcfhgbch.png
Type: image/png
Size: 1842 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160627/7dbf81f1/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 1842 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160627/7dbf81f1/attachment-0001.png>

More information about the Gluster-users mailing list