[Gluster-devel] gluster crashes in dht_getxattr_cbk() due to null pointer dereference.

[Niels de Vos]

On Fri, May 08, 2015 at 05:55:01PM +0800, Paul Guo wrote:
> Hi,
> 
> gdb debugging shows the rootcause seems to be quite straightforward. The
> gluster version is 3.4.5 and the stack:

As others have mentioned already, GlusterFS 3.4 is on its way out. When
3.7.0 has been released (any day now), there will be no more updates for
3.4.x. That said, the latest 3.4 release is 3.4.7, swhich contains many
bugfixes compared to the release you are running:

    https://github.com/gluster/glusterfs/compare/v3.4.5...v3.4.7

I suggest you go through those changes, and see if there is something
that looks like a fix. If that is the case, you should be able to safely
update to 3.4.7.

But, in the near future, you really should look into upgrading to a
newer version.

HTH,
Niels

> 
> #0  0x00007eff735fe354 in dht_getxattr_cbk (frame=0x7eff775b6360,
> cookie=<value optimized out>, this=<value optimized out>, op_ret=<value
> optimized out>, op_errno=0,
>     xattr=<value optimized out>, xdata=0x0) at dht-common.c:2043
> 2043	                DHT_STACK_UNWIND (getxattr, frame, local->op_ret,
> op_errno,
> Missing separate debuginfos, use: debuginfo-install
> glibc-2.12-1.80.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64
> krb5-libs-1.9-33.el6.x86_64 libcom_err-1.41.12-12.el6.x86_64
> libgcc-4.4.6-4.el6.x86_64 libselinux-2.0.94-5.3.el6.x86_64
> openssl-1.0.1e-16.el6_5.14.x86_64 zlib-1.2.3-27.el6.x86_64
> (gdb) bt
> #0  0x00007eff735fe354 in dht_getxattr_cbk (frame=0x7eff775b6360,
> cookie=<value optimized out>, this=<value optimized out>, op_ret=<value
> optimized out>, op_errno=0,
>     xattr=<value optimized out>, xdata=0x0) at dht-common.c:2043
> #1  0x00007eff7383c168 in afr_getxattr_cbk (frame=0x7eff7756ab58,
> cookie=<value optimized out>, this=<value optimized out>, op_ret=0,
> op_errno=0, dict=0x7eff76f21dc8, xdata=0x0)
>     at afr-inode-read.c:618
> #2  0x00007eff73aaaad8 in client3_3_getxattr_cbk (req=<value optimized out>,
> iov=<value optimized out>, count=<value optimized out>,
> myframe=0x7eff77554d4c) at client-rpc-fops.c:1115
> #3  0x0000003de700d6f5 in rpc_clnt_handle_reply (clnt=0xc36ad0,
> pollin=0x14b21560) at rpc-clnt.c:771
> #4  0x0000003de700ec6f in rpc_clnt_notify (trans=<value optimized out>,
> mydata=0xc36b00, event=<value optimized out>, data=<value optimized out>) at
> rpc-clnt.c:891
> #5  0x0000003de700a4e8 in rpc_transport_notify (this=<value optimized out>,
> event=<value optimized out>, data=<value optimized out>) at
> rpc-transport.c:497
> #6  0x00007eff74af6216 in socket_event_poll_in (this=0xc46530) at
> socket.c:2118
> #7  0x00007eff74af7c3d in socket_event_handler (fd=<value optimized out>,
> idx=<value optimized out>, data=0xc46530, poll_in=1, poll_out=0, poll_err=0)
> at socket.c:2230
> #8  0x0000003de785e907 in event_dispatch_epoll_handler (event_pool=0xb70e90)
> at event-epoll.c:384
> #9  event_dispatch_epoll (event_pool=0xb70e90) at event-epoll.c:445
> #10 0x0000000000406818 in main (argc=4, argv=0x7fff24878238) at
> glusterfsd.c:1934
> 
> See dht_getxattr_cbk() (below). When frame->local is equal to 0, gluster
> jumps to the label "out" where when it accesses local->xattr (i.e.
> 0->xattr), it crashes. Note in DHT_STACK_UNWIND()->STACK_UNWIND_STRICT(), fn
> looks fine.
> 
> (gdb) p __local
> $11 = (dht_local_t *) 0x0
> (gdb) p frame->local
> $12 = (void *) 0x0
> (gdb) p fn
> $1 = (fop_getxattr_cbk_t) 0x7eff7298c940 <mdc_readv_cbk>
> 
> I did not read the dht code much so I have not idea whether zero
> frame->local is normal or not, but from the code's perspective this is an
> obvious bug and it still exists in latest glusterfs workspace.
> 
> The following code change is a simple fix, but maybe there's a better one.
> -        if (is_last_call (this_call_cnt)) {
> +        if (is_last_call (this_call_cnt) && local != NULL) {
> 
> Similar issues exist in other functions also, e.g. stripe_getxattr_cbk() (I
> did not check all code).
> 
> int
> dht_getxattr_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
>                   int op_ret, int op_errno, dict_t *xattr, dict_t *xdata)
> {
>         int             this_call_cnt = 0;
>         dht_local_t     *local = NULL;
> 
>         VALIDATE_OR_GOTO (frame, out);
>         VALIDATE_OR_GOTO (frame->local, out);
> 
>         ......
> 
> out:
>         if (is_last_call (this_call_cnt)) {
>                 DHT_STACK_UNWIND (getxattr, frame, local->op_ret, op_errno,
>                                   local->xattr, NULL);
>         }
>         return 0;
> }
> 
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150509/084dc3d9/attachment.sig>