[Bugs] [Bug 1246809] New: glusterd crashed when a client which doesn't support SSL tries to mount a SSL enabled gluster volume

Sun Jul 26 01:52:33 UTC 2015

https://bugzilla.redhat.com/show_bug.cgi?id=1246809

            Bug ID: 1246809
           Summary: glusterd crashed when a client which doesn't support
                    SSL tries to mount a SSL enabled gluster volume
           Product: GlusterFS
           Version: mainline
         Component: glusterd
          Severity: medium
          Assignee: kaushal at redhat.com
          Reporter: kaushal at redhat.com
                CC: bugs at gluster.org, gluster-bugs at redhat.com,
                    nlevinki at redhat.com, rgowdapp at redhat.com,
                    rhs-bugs at redhat.com, storage-qa-internal at redhat.com,
                    vbellur at redhat.com, vbhat at redhat.com
        Depends On: 1243774
            Blocks: 1233025 (glusterfs-3.7.3)

+++ This bug was initially created as a clone of Bug #1243774 +++

+++ This bug was initially created as a clone of Bug #1243722 +++

Description of problem:
I was running rebalance automation and I had a gluster volume with SSL network
encryption enabled in both data and management path. But when a client with a
gluster version which does not support SSL tries to mount the this gluster
volume, the glusterd in the volfile-server crashed.

Version-Release number of selected component (if applicable):
gluster server: glusterfs-3.7.1-10.el6rhs.x86_64
gluster client: glusterfs-3.6.0.53-1.el6.x86_64

How reproducible:
Hit only once in as many tries

Steps to Reproduce:
1. Run rebalance regression tests with SSL enabled with old client version

Actual results:
glusterd crashed with below backtrace

#0  list_del (rpc=<value optimized out>, xl=0x7fb94bfe1050, event=<value
optimized out>, data=0x7fb92c000be0) at ../../../../libglusterfs/src/list.h:76
#1  glusterd_rpcsvc_notify (rpc=<value optimized out>, xl=0x7fb94bfe1050,
event=<value optimized out>, data=0x7fb92c000be0) at glusterd.c:347
#2  0x00007fb94a9df665 in rpcsvc_handle_disconnect (svc=0x7fb94bfea380,
trans=0x7fb92c000be0) at rpcsvc.c:754
#3  0x00007fb94a9e11c0 in rpcsvc_notify (trans=0x7fb92c000be0, mydata=<value
optimized out>, event=<value optimized out>, data=0x7fb92c000be0) at
rpcsvc.c:792
#4  0x00007fb94a9e2ad8 in rpc_transport_notify (this=<value optimized out>,
event=<value optimized out>, data=<value optimized out>) at rpc-transport.c:543
#5  0x00007fb93dca9ba3 in socket_poller (ctx=0x7fb92c000be0) at socket.c:2582
#6  0x00007fb949d02a51 in start_thread () from /lib64/libpthread.so.0
#7  0x00007fb94966c96d in clone () from /lib64/libc.so.6

Expected results:
There should be no glusterd crash even if a client which does not support the
SSL tries to mount the SSL enabled volume.

Additional info:

I will upload the sosreport from the crashed machine.

--- Additional comment from Kaushal on 2015-07-16 15:25:13 IST ---

Change has been posted for review at http://review.gluster.org/11692

--- Additional comment from Anand Avati on 2015-07-18 20:24:44 IST ---

REVIEW: http://review.gluster.org/11692 (rpc,server,glusterd: Init transport
list for accepted transport) posted (#2) for review on master by Kaushal M
(kaushal at redhat.com)

--- Additional comment from Anand Avati on 2015-07-23 18:09:34 IST ---

REVIEW: http://review.gluster.org/11692 (rpc,server,glusterd: Init transport
list for accepted transport) posted (#3) for review on master by Kaushal M
(kaushal at redhat.com)

--- Additional comment from Anand Avati on 2015-07-24 14:24:07 IST ---

COMMIT: http://review.gluster.org/11692 committed in master by Raghavendra G
(rgowdapp at redhat.com) 
------
commit a909ccfa1b4cbf656c4608ef2124347851c492cb
Author: Kaushal M <kaushal at redhat.com>
Date:   Thu Jul 16 14:52:36 2015 +0530

    rpc,server,glusterd: Init transport list for accepted transport

    GlusterD or a brick would crash when encrypted transport was enabled and
    an unencrypted client tried to connect to them. The crash occured when
    GlusterD/server tried to remove the transport from their xprt_list due
    to a DISCONNECT event. But as the client transport's list head wasn't
    inited, the process would crash when list_del was performed.

    Initing the client transports list head during acceptence, prevents this
    crash.

    Also, an extra check has been added to the GlusterD and Server
    notification handlers for client DISCONNECT events. The handlers will
    now first check if the client transport is a member of any list.
    GlusterD and Server DISCONNECT event handlers could be called without
    the ACCEPT handler, which adds the transport to the list, being called.
    This situation also occurs when an unencrypted client tries to establish
    a connection with an encrypted server.

    Change-Id: Icc24a08d60e978aaa1d3322e0cbed680dcbda2b4
    BUG: 1243774
    Signed-off-by: Kaushal M <kaushal at redhat.com>
    Reviewed-on: http://review.gluster.org/11692
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Tested-by: NetBSD Build System <jenkins at build.gluster.org>
    Reviewed-by: Raghavendra G <rgowdapp at redhat.com>

Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1233025
[Bug 1233025] GlusterFS 3.7.3 tracker
https://bugzilla.redhat.com/show_bug.cgi?id=1243774
[Bug 1243774] glusterd crashed when a client which doesn't support SSL
tries to mount a SSL enabled gluster volume
-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=XQG9ZWpY66&a=cc_unsubscribe