[Gluster-devel] [Gluster-users] Glusterfsd crashed with SIGSEGV
ABHISHEK PALIWAL
abhishpaliwal at gmail.com
Wed Mar 13 04:00:08 UTC 2019
Hi Amar,
did you get time to check the logs?
Regards,
Abhishek
On Tue, Mar 12, 2019 at 10:58 AM ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
wrote:
> Hi Amar,
>
> Below are the requested logs
>
> pabhishe at arn-build3$ldd ./sysroot-destdir/usr/lib64/libglusterfs.so
> not a dynamic executable
>
> pabhishe at arn-build3$ldd ./sysroot-destdir/usr/lib64/libgfrpc.so
> not a dynamic executable
>
> root at 128:/# gdb /usr/sbin/glusterd core.1099
> GNU gdb (GDB) 7.10.1
> Copyright (C) 2015 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "powerpc64-wrs-linux".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from /usr/sbin/glusterd...(no debugging symbols
> found)...done.
> [New LWP 1109]
> [New LWP 1101]
> [New LWP 1105]
> [New LWP 1110]
> [New LWP 1099]
> [New LWP 1107]
> [New LWP 1119]
> [New LWP 1103]
> [New LWP 1112]
> [New LWP 1116]
> [New LWP 1104]
> [New LWP 1239]
> [New LWP 1106]
> [New LWP 1111]
> [New LWP 1108]
> [New LWP 1117]
> [New LWP 1102]
> [New LWP 1118]
> [New LWP 1100]
> [New LWP 1114]
> [New LWP 1113]
> [New LWP 1115]
>
> warning: Could not load shared library symbols for linux-vdso64.so.1.
> Do you need "set solib-search-path" or "set sysroot"?
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> Core was generated by `/usr/sbin/glusterfsd -s 128.224.95.140 --volfile-id
> gv0.128.224.95.140.tmp-bric'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0 0x00003fffb76a6d48 in _int_malloc (av=av at entry=0x3fffa8000020,
> bytes=bytes at entry=36) at malloc.c:3327
> 3327 {
> [Current thread is 1 (Thread 0x3fffb1689160 (LWP 1109))]
> (gdb) bt full
> #0 0x00003fffb76a6d48 in _int_malloc (av=av at entry=0x3fffa8000020,
> bytes=bytes at entry=36) at malloc.c:3327
> nb = <optimized out>
> idx = <optimized out>
> bin = <optimized out>
> victim = <optimized out>
> size = <optimized out>
> victim_index = <optimized out>
> remainder = <optimized out>
> remainder_size = <optimized out>
> block = <optimized out>
> bit = <optimized out>
> map = <optimized out>
> fwd = <optimized out>
> bck = <optimized out>
> errstr = 0x0
> __func__ = "_int_malloc"
> #1 0x00003fffb76a93dc in __GI___libc_malloc (bytes=36) at malloc.c:2921
> ar_ptr = 0x3fffa8000020
> victim = <optimized out>
> hook = <optimized out>
> __func__ = "__libc_malloc"
> #2 0x00003fffb7764fd0 in x_inline (xdrs=0x3fffb1686d20, len=<optimized
> out>) at xdr_sizeof.c:89
> len = 36
> xdrs = 0x3fffb1686d20
> #3 0x00003fffb7842488 in .xdr_gfx_iattx () from /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #4 0x00003fffb7842e84 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #5 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa81099f0, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa8109aa0 "\265\256\373\200\f\206\361j"
> stat = <optimized out>
> #6 0x00003fffb7764e04 in __GI_xdr_pointer (xdrs=0x3fffb1686d20,
> objpp=0x3fffa81099f0, obj_size=<optimized out>,
> xdr_obj=@0x3fffb785f4b0: 0x3fffb7842dc0 <.xdr_gfx_dirplist>) at
> xdr_ref.c:135
> more_data = 1
> #7 0x00003fffb7842ec0 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #8 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa8109870, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa8109920 "\232\373\377\315\352\325\005\271"
> stat = <optimized out>
> #9 0x00003fffb7764e04 in __GI_xdr_pointer (xdrs=0x3fffb1686d20,
> objpp=0x3fffa8109870, obj_size=<optimized out>,
> xdr_obj=@0x3fffb785f4b0: 0x3fffb7842dc0 <.xdr_gfx_dirplist>) at
> xdr_ref.c:135
> more_data = 1
> #10 0x00003fffb7842ec0 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #11 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa81096f0, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa81097a0 "\241X\372!\216\256=\342"
> stat = <optimized out>
> ---Type <return> to continue, or q <return> to quit---
> #12 0x00003fffb7764e04 in __GI_xdr_pointer (xdrs=0x3fffb1686d20,
> objpp=0x3fffa81096f0, obj_size=<optimized out>,
> xdr_obj=@0x3fffb785f4b0: 0x3fffb7842dc0 <.xdr_gfx_dirplist>) at
> xdr_ref.c:135
> more_data = 1
> #13 0x00003fffb7842ec0 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #14 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa8109570, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa8109620 "\265\205\003Vu'\002L"
> stat = <optimized out>
> #15 0x00003fffb7764e04 in __GI_xdr_pointer (xdrs=0x3fffb1686d20,
> objpp=0x3fffa8109570, obj_size=<optimized out>,
> xdr_obj=@0x3fffb785f4b0: 0x3fffb7842dc0 <.xdr_gfx_dirplist>) at
> xdr_ref.c:135
> more_data = 1
> #16 0x00003fffb7842ec0 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #17 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa81093f0, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa81094a0 "\200L\027F'\177\366D"
> stat = <optimized out>
> #18 0x00003fffb7764e04 in __GI_xdr_pointer (xdrs=0x3fffb1686d20,
> objpp=0x3fffa81093f0, obj_size=<optimized out>,
> xdr_obj=@0x3fffb785f4b0: 0x3fffb7842dc0 <.xdr_gfx_dirplist>) at
> xdr_ref.c:135
> more_data = 1
> #19 0x00003fffb7842ec0 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #20 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa8109270, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa8109320 "\217{dK(\001E\220"
> stat = <optimized out>
> #21 0x00003fffb7764e04 in __GI_xdr_pointer (xdrs=0x3fffb1686d20,
> objpp=0x3fffa8109270, obj_size=<optimized out>,
> xdr_obj=@0x3fffb785f4b0: 0x3fffb7842dc0 <.xdr_gfx_dirplist>) at
> xdr_ref.c:135
> more_data = 1
> #22 0x00003fffb7842ec0 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #23 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa81090f0, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa81091a0 "\217\275\067\336\232\300(\005"
> stat = <optimized out>
> #24 0x00003fffb7764e04 in __GI_xdr_pointer (xdrs=0x3fffb1686d20,
> objpp=0x3fffa81090f0, obj_size=<optimized out>,
> xdr_obj=@0x3fffb785f4b0: 0x3fffb7842dc0 <.xdr_gfx_dirplist>) at
> xdr_ref.c:135
> more_data = 1
> #25 0x00003fffb7842ec0 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #26 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa8108f70, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa8109020 "\260.\025\b\244\352IT"
> stat = <optimized out>
> #27 0x00003fffb7764e04 in __GI_xdr_pointer (xdrs=0x3fffb1686d20,
> objpp=0x3fffa8108f70, obj_size=<optimized out>,
> xdr_obj=@0x3fffb785f4b0: 0x3fffb7842dc0 <.xdr_gfx_dirplist>) at
> xdr_ref.c:135
> more_data = 1
> #28 0x00003fffb7842ec0 in .xdr_gfx_dirplist () from
> /usr/lib64/libgfxdr.so.0
> No symbol table info available.
> #29 0x00003fffb7764c28 in __GI_xdr_reference (xdrs=0x3fffb1686d20,
> pp=0x3fffa8108df0, size=<optimized out>, proc=<optimized out>) at
> xdr_ref.c:84
> loc = 0x3fffa8108ea0 "\212GS\203l\035\n\\"
> ---Type <return> to continue, or q <return> to quit---
>
>
> Regards,
> Abhishek
>
> On Mon, Mar 11, 2019 at 7:10 PM Amar Tumballi Suryanarayan <
> atumball at redhat.com> wrote:
>
>> Hi Abhishek,
>>
>> Can you check and get back to us?
>>
>> ```
>> bash# ldd /usr/lib64/libglusterfs.so
>> bash# ldd /usr/lib64/libgfrpc.so
>>
>> ```
>>
>> Also considering you have the core, can you do `(gdb) thr apply all bt
>> full` and pass it on?
>>
>> Thanks & Regards,
>> Amar
>>
>> On Mon, Mar 11, 2019 at 3:41 PM ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
>> wrote:
>>
>>> Hi Team,
>>>
>>> COuld you please provide some pointer to debug it further.
>>>
>>> Regards,
>>> Abhishek
>>>
>>> On Fri, Mar 8, 2019 at 2:19 PM ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
>>> wrote:
>>>
>>>> Hi Team,
>>>>
>>>> I am using Glusterfs 5.4, where after setting the gluster mount point
>>>> when trying to access it, glusterfsd is getting crashed and mount point
>>>> through the "Transport endpoint is not connected error.
>>>>
>>>> Here I are the gdb log for the core file
>>>>
>>>> warning: Could not load shared library symbols for linux-vdso64.so.1.
>>>> Do you need "set solib-search-path" or "set sysroot"?
>>>> [Thread debugging using libthread_db enabled]
>>>> Using host libthread_db library "/lib64/libthread_db.so.1".
>>>> Core was generated by `/usr/sbin/glusterfsd -s 128.224.95.140
>>>> --volfile-id gv0.128.224.95.140.tmp-bric'.
>>>> Program terminated with signal SIGSEGV, Segmentation fault.
>>>> #0 0x00003fff95ab1d48 in _int_malloc (av=av at entry=0x3fff7c000020,
>>>> bytes=bytes at entry=36) at malloc.c:3327
>>>> 3327 {
>>>> [Current thread is 1 (Thread 0x3fff90394160 (LWP 811))]
>>>> (gdb)
>>>> (gdb)
>>>> (gdb) bt
>>>> #0 0x00003fff95ab1d48 in _int_malloc (av=av at entry=0x3fff7c000020,
>>>> bytes=bytes at entry=36) at malloc.c:3327
>>>> #1 0x00003fff95ab43dc in __GI___libc_malloc (bytes=36) at malloc.c:2921
>>>> #2 0x00003fff95b6ffd0 in x_inline (xdrs=0x3fff90391d20, len=<optimized
>>>> out>) at xdr_sizeof.c:89
>>>> #3 0x00003fff95c4d488 in .xdr_gfx_iattx () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #4 0x00003fff95c4de84 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #5 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c132020, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #6 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c132020, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #7 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #8 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c131ea0, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #9 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c131ea0, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #10 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #11 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c131d20, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #12 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c131d20, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #13 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #14 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c131ba0, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #15 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c131ba0, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #16 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #17 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c131a20, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #18 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c131a20, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #19 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #20 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c1318a0, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #21 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c1318a0, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #22 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #23 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c131720, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #24 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c131720, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #25 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #26 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c1315a0, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #27 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c1315a0, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #28 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #29 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c131420, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #30 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c131420, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>> #31 0x00003fff95c4dec0 in .xdr_gfx_dirplist () from
>>>> /usr/lib64/libgfxdr.so.0
>>>> #32 0x00003fff95b6fc28 in __GI_xdr_reference (xdrs=0x3fff90391d20,
>>>> pp=0x3fff7c1312a0, size=<optimized out>, proc=<optimized out>) at
>>>> xdr_ref.c:84
>>>> #33 0x00003fff95b6fe04 in __GI_xdr_pointer (xdrs=0x3fff90391d20,
>>>> objpp=0x3fff7c1312a0, obj_size=<optimized out>,
>>>> xdr_obj=@0x3fff95c6a4b0: 0x3fff95c4ddc0 <.xdr_gfx_dirplist>) at
>>>> xdr_ref.c:135
>>>>
>>>> Frames are getting repeated, could any one please me.
>>>> --
>>>> Regards
>>>> Abhishek Paliwal
>>>>
>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>> Regards
>>> Abhishek Paliwal
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>
>>
>>
>> --
>> Amar Tumballi (amarts)
>>
>
>
> --
>
>
>
>
> Regards
> Abhishek Paliwal
>
--
Regards
Abhishek Paliwal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20190313/dc78d1c1/attachment-0001.html>
More information about the Gluster-devel
mailing list