[Gluster-users] core dump when do "gluster volume quota foo list --xml"

Raghavendra Talur raghavendra.talur at gmail.com
Sat Jun 13 08:10:35 UTC 2015


Sorry for top posting but I think this might be a different issue.

Dang,

If it was working well for you and started crashing recently after doing a
system update,
it might be because of glibc checks.

This is the line from your output which gives the hint
>*** glibc detected *** gluster: double free or corruption (out):

As Niels suggests 3.4.5 is too old and you should upgrade to supported
version.
However, if you want a quick workaround for now, you should disable glibc
checks
for double free.

I found this stack overflow link for doing that.
http://stackoverflow.com/questions/6750815/how-to-turn-off-glibc-run-time-protections

Let us know if it works.
Thanks,
Raghavendra Talur

On Sat, Jun 13, 2015 at 1:30 PM, Niels de Vos <ndevos at redhat.com> wrote:

> On Sat, Jun 13, 2015 at 10:24:34AM +0530, Vijaikumar M wrote:
> > Hi Dang,
> >
> > This issue has been fixed in 3.7.0:
> > bug: https://bugzilla.redhat.com/show_bug.cgi?id=1185259
> > patch: http://review.gluster.org/#/c/9481/
>
> I found https://bugzilla.redhat.com/1200297 which is doing a backport to
> 3.6.x. Should this not get fixed in 3.5.x too? If it applies there,
> please send a backport for it.
>
> Dang, 3.4.x is not getting any updates anymore. We are providing updated
> releases for the most current version (3.7) and the two previous ones
> (3.6 and 3.5).
>
> HTH,
> Niels
>
> >
> >
> > Thanks,
> > Vijay
> >
> >
> > On Thursday 11 June 2015 12:48 PM, Dang Zhiqiang wrote:
> > >Hi,
> > >
> > >    core dump when do "gluster volume quota foo list --xml", how to fix
> > >it?
> > >
> > >env:
> > >glusterfs 3.4.5
> > >centos 6.5
> > >
> > >log:
> > ># gluster volume quota foo list --xml
> > ><?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> > ><cliOutput>
> > >  <opRet>0</opRet>
> > >  <opErrno>115</opErrno>
> > >  <opErrstr/>
> > >  <volQuota>
> > >    <quota>
> > >      <path>/music</path>
> > >      <limit>50MB</limit>
> > >      <size>0Bytes</size>
> > >    </quota>
> > >    <quota>
> > >      <path>/film</path>
> > >      <limit>60MB</limit>
> > >      <size>9.6MB</size>
> > >    </quota>
> > >    <quota>
> > >      <path>/</path>
> > >      <limit>200MB</limit>
> > >      <size>19.3MB</size>
> > >    </quota>
> > >  </volQuota>
> > ></cliOutput>
> > >*** glibc detected *** gluster: double free or corruption (out):
> > >0x000000000203c0c0 ***
> > >======= Backtrace: =========
> > >/lib64/libc.so.6[0x3d53a75e76]
> > >/lib64/libc.so.6[0x3d53a789b3]
> > >gluster(cli_xml_output_vol_quota_limit_list+0x141)[0x427161]
> > >gluster(gf_cli_quota_cbk+0x3f3)[0x415f93]
> > >/usr/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5)[0x347f80d6f5]
> > >/usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x11f)[0x347f80ec6f]
> > >/usr/lib64/libgfrpc.so.0(rpc_transport_notify+0x28)[0x347f80a4e8]
> >
> >/usr/lib64/glusterfs/3.4.5/rpc-transport/socket.so(+0x9216)[0x7f0d425df216]
> >
> >/usr/lib64/glusterfs/3.4.5/rpc-transport/socket.so(+0xac3d)[0x7f0d425e0c3d]
> > >/usr/lib64/libglusterfs.so.0[0x347f45e907]
> > >gluster(main+0x355)[0x408f35]
> > >/lib64/libc.so.6(__libc_start_main+0xfd)[0x3d53a1ed5d]
> > >gluster[0x4083c9]
> > >======= Memory map: ========
> > >00400000-0043b000 r-xp 00000000 fd:00 40468
> > >/usr/sbin/gluster
> > >0063b000-0063f000 rw-p 0003b000 fd:00 40468
> > >/usr/sbin/gluster
> > >01fe0000-0204d000 rw-p 00000000 00:00 0                [heap]
> > >347f400000-347f47a000 r-xp 00000000 fd:00 40465
> > >/usr/lib64/libglusterfs.so.0.0.0
> > >347f47a000-347f67a000 ---p 0007a000 fd:00 40465
> > >/usr/lib64/libglusterfs.so.0.0.0
> > >347f67a000-347f67c000 rw-p 0007a000 fd:00 40465
> > >/usr/lib64/libglusterfs.so.0.0.0
> > >347f67c000-347f67f000 rw-p 00000000 00:00 0
> > >347f800000-347f814000 r-xp 00000000 fd:00 40466
> > >/usr/lib64/libgfrpc.so.0.0.0
> > >347f814000-347fa14000 ---p 00014000 fd:00 40466
> > >/usr/lib64/libgfrpc.so.0.0.0
> > >347fa14000-347fa15000 rw-p 00014000 fd:00 40466
> > >/usr/lib64/libgfrpc.so.0.0.0
> > >347fc00000-347fc15000 r-xp 00000000 fd:00 40467
> > >/usr/lib64/libgfxdr.so.0.0.0
> > >347fc15000-347fe15000 ---p 00015000 fd:00 40467
> > >/usr/lib64/libgfxdr.so.0.0.0
> > >347fe15000-347fe16000 rw-p 00015000 fd:00 40467
> > >/usr/lib64/libgfxdr.so.0.0.0
> > >3d53200000-3d53220000 r-xp 00000000 fd:00 40276
> > >/lib64/ld-2.12.so
> > >3d5341f000-3d53420000 r--p 0001f000 fd:00 40276
> > >/lib64/ld-2.12.so
> > >3d53420000-3d53421000 rw-p 00020000 fd:00 40276
> > >/lib64/ld-2.12.so
> > >3d53421000-3d53422000 rw-p 00000000 00:00 0
> > >3d53600000-3d53602000 r-xp 00000000 fd:00 40285
> > >/lib64/libdl-2.12.so
> > >3d53602000-3d53802000 ---p 00002000 fd:00 40285
> > >/lib64/libdl-2.12.so
> > >3d53802000-3d53803000 r--p 00002000 fd:00 40285
> > >/lib64/libdl-2.12.so
> > >3d53803000-3d53804000 rw-p 00003000 fd:00 40285
> > >/lib64/libdl-2.12.so
> > >3d53a00000-3d53b8a000 r-xp 00000000 fd:00 40277
> > >/lib64/libc-2.12.so
> > >3d53b8a000-3d53d8a000 ---p 0018a000 fd:00 40277
> > >/lib64/libc-2.12.so
> > >3d53d8a000-3d53d8e000 r--p 0018a000 fd:00 40277
> > >/lib64/libc-2.12.so
> > >3d53d8e000-3d53d8f000 rw-p 0018e000 fd:00 40277
> > >/lib64/libc-2.12.so
> > >3d53d8f000-3d53d94000 rw-p 00000000 00:00 0
> > >3d53e00000-3d53e17000 r-xp 00000000 fd:00 11764
> > >/lib64/libpthread-2.12.so
> > >3d53e17000-3d54017000 ---p 00017000 fd:00 11764
> > >/lib64/libpthread-2.12.so
> > >3d54017000-3d54018000 r--p 00017000 fd:00 11764
> > >/lib64/libpthread-2.12.so
> > >3d54018000-3d54019000 rw-p 00018000 fd:00 11764
> > >/lib64/libpthread-2.12.so
> > >3d54019000-3d5401d000 rw-p 00000000 00:00 0
> > >3d54200000-3d54222000 r-xp 00000000 fd:00 5114
> > >/lib64/libncurses.so.5.7
> > >3d54222000-3d54421000 ---p 00022000 fd:00 5114
> > >/lib64/libncurses.so.5.7
> > >3d54421000-3d54422000 rw-p 00021000 fd:00 5114
> > >/lib64/libncurses.so.5.7
> > >3d54600000-3d54615000 r-xp 00000000 fd:00 40286
> > >/lib64/libz.so.1.2.3
> > >3d54615000-3d54814000 ---p 00015000 fd:00 40286
> > >/lib64/libz.so.1.2.3
> > >3d54814000-3d54815000 r--p 00014000 fd:00 40286
> > >/lib64/libz.so.1.2.3
> > >3d54815000-3d54816000 rw-p 00015000 fd:00 40286
> > >/lib64/libz.so.1.2.3
> > >3d54a00000-3d54a83000 r-xp 00000000 fd:00 40287
> > >/lib64/libm-2.12.so
> > >3d54a83000-3d54c82000 ---p 00083000 fd:00 40287
> > >/lib64/libm-2.12.so
> > >3d54c82000-3d54c83000 r--p 00082000 fd:00 40287
> > >/lib64/libm-2.12.so
> > >3d54c83000-3d54c84000 rw-p 00083000 fd:00 40287
> > >/lib64/libm-2.12.so
> > >3d55200000-3d5521d000 r-xp 00000000 fd:00 40290
> > >/lib64/libselinux.so.1
> > >3d5521d000-3d5541c000 ---p 0001d000 fd:00 40290
> > >/lib64/libselinux.so.1
> > >3d5541c000-3d5541d000 r--p 0001c000 fd:00 40290
> > >/lib64/libselinux.so.1
> > >3d5541d000-3d5541e000 rw-p 0001d000 fd:00 40290
> > >/lib64/libselinux.so.1
> > >3d5541e000-3d5541f000 rw-p 00000000 00:00 0
> > >3d55600000-3d55616000 r-xp 00000000 fd:00 12401
> > >/lib64/libresolv-2.12.so
> > >3d55616000-3d55816000 ---p 00016000 fd:00 12401
> > >/lib64/libresolv-2.12.so
> > >3d55816000-3d55817000 r--p 00016000 fd:00 12401
> > >/lib64/libresolv-2.12.so
> > >3d55817000-3d55818000 rw-p 00017000 fd:00 12401
> > >/lib64/libresolv-2.12.so
> > >3d55818000-3d5581a000 rw-p 00000000 00:00 0
> > >3d55a00000-3d55a16000 r-xp 00000000 fd:00 40318
> > >/lib64/libgcc_s-4.4.7-20120601.so.1
> > >3d55a16000-3d55c15000 ---p 00016000 fd:00 40318
> > >/lib64/libgcc_s-4.4.7-20120601.so.1
> > >3d55c15000-3d55c16000 rw-p 00015000 fd:00 40318
> > >/lib64/libgcc_s-4.4.7-20120601.so.1
> > >3d55e00000-3d55e1d000 r-xp 00000000 fd:00 24839
> > >/lib64/libtinfo.so.5.7
> > >3d55e1d000-3d5601d000 ---p 0001d000 fd:00 24839
> > >/lib64/libtinfo.so.5.7
> > >3d5601d000-3d56021000 rw-p 0001d000 fd:00 24839
> > >/lib64/libtinfo.so.5.7Aborted (core dumped
> > >
> > >debug:
> > >Core was generated by `gluster --mode=script v quota foo list --xml'.
> > >Program terminated with signal 6, Aborted.
> > >#0  0x0000003d53a32635 in raise () from /lib64/libc.so.6
> > >Missing separate debuginfos, use: debuginfo-install
> > >glibc-2.12-1.132.el6_5.4.x86_64 keyutils-libs-1.4-4.el6.x86_64
> > >krb5-libs-1.10.3-15.el6_5.1.x86_64 libcom_err-1.41.12-18.el6_5.1.x86_64
> > >libgcc-4.4.7-4.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64
> > >libxml2-2.7.6-14.el6_5.2.x86_64 ncurses-libs-5.7-3.20090208.el6.x86_64
> > >openssl-1.0.1e-16.el6_5.15.x86_64 readline-6.0-4.el6.x86_64
> > >zlib-1.2.3-29.el6.x86_64
> > >(gdb) bt
> > >#0  0x0000003d53a32635 in raise () from /lib64/libc.so.6
> > >#1  0x0000003d53a33e15 in abort () from /lib64/libc.so.6
> > >#2  0x0000003d53a70547 in __libc_message () from /lib64/libc.so.6
> > >#3  0x0000003d53a75e76 in malloc_printerr () from /lib64/libc.so.6
> > >#4  0x0000003d53a789b3 in _int_free () from /lib64/libc.so.6
> > >#5  0x0000000000427161 in cli_xml_output_vol_quota_limit_list
> > >(volname=<value optimized out>, limit_list=0xcc3bb0
> > >"/music:50MB,/film:60MB,/:200MB", op_ret=30,
> > >    op_errno=115, op_errstr=<value optimized out>) at
> > >cli-xml-output.c:2843
> > >#6  0x0000000000415f93 in gf_cli_quota_cbk (req=<value optimized out>,
> > >iov=<value optimized out>, count=<value optimized out>, myframe=<value
> > >optimized out>)
> > >    at cli-rpc-ops.c:2487
> > >#7  0x000000347f80d6f5 in rpc_clnt_handle_reply (clnt=0xcb3b10,
> > >pollin=0xcc3d00) at rpc-clnt.c:771
> > >#8  0x000000347f80ec6f in rpc_clnt_notify (trans=<value optimized out>,
> > >mydata=0xcb3b40, event=<value optimized out>, data=<value optimized
> out>)
> > >at rpc-clnt.c:891
> > >#9  0x000000347f80a4e8 in rpc_transport_notify (this=<value optimized
> > >out>, event=<value optimized out>, data=<value optimized out>) at
> > >rpc-transport.c:497
> > >#10 0x00007fa4f14fe216 in socket_event_poll_in (this=0xcbd130) at
> > >socket.c:2118
> > >#11 0x00007fa4f14ffc3d in socket_event_handler (fd=<value optimized
> out>,
> > >idx=<value optimized out>, data=0xcbd130, poll_in=1, poll_out=0,
> > >poll_err=0) at socket.c:2230
> > >#12 0x000000347f45e907 in event_dispatch_epoll_handler
> > >(event_pool=0xc8be90) at event-epoll.c:384
> > >#13 event_dispatch_epoll (event_pool=0xc8be90) at event-epoll.c:445
> > >#14 0x0000000000408f35 in main (argc=<value optimized out>, argv=<value
> > >optimized out>) at cli.c:620
> > >(gdb) l *0x0000000000427161
> > >0x427161 is in cli_xml_output_vol_quota_limit_list
> > >(cli-xml-output.c:2844).
> > >2839
> > >2840       ret = cli_end_xml_output (writer, doc);
> > >2841
> > >2842out:
> > >*2843        GF_FREE (size_str);*
> > >2844       gf_log ("cli", GF_LOG_DEBUG, "Returning %d", ret);
> > >2845       return ret;
> > >2846#else
> > >2847       return 0;
> > >2848#endif
> > >
> > >
> > >
> > >_______________________________________________
> > >Gluster-users mailing list
> > >Gluster-users at gluster.org
> > >http://www.gluster.org/mailman/listinfo/gluster-users
> >
>
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>



-- 
*Raghavendra Talur *
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150613/4b9d4966/attachment.html>


More information about the Gluster-users mailing list