[Bugs] [Bug 1228093] New: Glusterd crash

bugzilla at redhat.com bugzilla at redhat.com
Thu Jun 4 08:01:28 UTC 2015


https://bugzilla.redhat.com/show_bug.cgi?id=1228093

            Bug ID: 1228093
           Summary: Glusterd crash
           Product: GlusterFS
           Version: mainline
         Component: glusterd
          Assignee: bugs at gluster.org
          Reporter: soumplis at ekt.gr
                CC: bugs at gluster.org, gluster-bugs at redhat.com



Description of problem:

On a two node gluster with 4 distributed volumes after upgrade of glusterfs
server to version 3.7.1 from 3.7.0 on one of the two nodes glusterd crashes
with a buffer overflow error. This happens on node0 when it tries to
communicate with node1. 

The nodes are exactly the same. Same hardware, same OS, same packages
installed.

Version-Release number of selected component (if applicable): 3.7.1


How reproducible:
Always on our environment

Steps to Reproduce:
1. On node0:

[root at gnode00 ~]# systemctl start glusterd
[root at gnode00 ~]# systemctl status glusterd.service 
glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled)
   Active: active (running) since Thu 2015-06-04 10:38:48 EEST; 7min ago
  Process: 12501 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid
(code=exited, status=0/SUCCESS)
 Main PID: 12502 (glusterd)
   CGroup: /system.slice/glusterd.service
           └─12502 /usr/sbin/glusterd -p /var/run/glusterd.pid

Jun 04 10:38:48 gnode00 systemd[1]: Starting GlusterFS, a clustered file-system
server...
Jun 04 10:38:48 gnode00 systemd[1]: Started GlusterFS, a clustered file-system
server.

2. On node1:
[root at gnode01 ~]# systemctl start glusterd
[root at gnode01 ~]# systemctl status glusterd
glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled)
   Active: active (running) since Thu 2015-06-04 10:54:55 EEST; 6s ago
  Process: 8179 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid
(code=exited, status=0/SUCCESS)
 Main PID: 8180 (glusterd)
   CGroup: /system.slice/glusterd.service
           ├─8180 /usr/sbin/glusterd -p /var/run/glusterd.pid
           ├─8202 /usr/sbin/glusterfsd -s gnode01 --volfile-id
mmphoto.gnode01.BRICKS-lv02a-mmphoto -p
/var/lib/glusterd/vols/mmphoto/run/gnode01...
           ├─8209 /usr/sbin/glusterfsd -s gnode01 --volfile-id
mmphoto.gnode01.BRICKS-lv02b-mmphoto -p
/var/lib/glusterd/vols/mmphoto/run/gnode01...
           ├─8215 /usr/sbin/glusterfsd -s gnode01 --volfile-id
phdms.gnode01.BRICKS-lv00-phdms -p
/var/lib/glusterd/vols/phdms/run/gnode01-BRICKS...
           ├─8220 /usr/sbin/glusterfsd -s gnode01 --volfile-id
restore.gnode01.BRICKS-lv03a-restore -p
/var/lib/glusterd/vols/restore/run/gnode01...
           └─8225 /usr/sbin/glusterfsd -s gnode01 --volfile-id
thesis.gnode01.BRICKS-lv01-thesis -p
/var/lib/glusterd/vols/thesis/run/gnode01-BRI...

Jun 04 10:54:55 gnode01 systemd[1]: Starting GlusterFS, a clustered file-system
server...
Jun 04 10:54:55 gnode01 systemd[1]: Started GlusterFS, a clustered file-system
server.


3. On node0:

[root at gnode00 ~]# systemctl status glusterd.service 
glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled)
   Active: failed (Result: signal) since Thu 2015-06-04 10:54:57 EEST; 1min 51s
ago
  Process: 12501 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid
(code=exited, status=0/SUCCESS)
 Main PID: 12502 (code=killed, signal=ABRT)

Jun 04 10:54:57 gnode00 etc-glusterfs-glusterd.vol[12502]: llistxattr 1
Jun 04 10:54:57 gnode00 etc-glusterfs-glusterd.vol[12502]: setfsid 1
Jun 04 10:54:57 gnode00 etc-glusterfs-glusterd.vol[12502]: spinlock 1
Jun 04 10:54:57 gnode00 etc-glusterfs-glusterd.vol[12502]: epoll.h 1
Jun 04 10:54:57 gnode00 etc-glusterfs-glusterd.vol[12502]: xattr.h 1
Jun 04 10:54:57 gnode00 etc-glusterfs-glusterd.vol[12502]: st_atim.tv_nsec 1
Jun 04 10:54:57 gnode00 etc-glusterfs-glusterd.vol[12502]: package-string:
glusterfs 3.7.1
Jun 04 10:54:57 gnode00 etc-glusterfs-glusterd.vol[12502]: ---------
Jun 04 10:54:57 gnode00 systemd[1]: glusterd.service: main process exited,
code=killed, status=6/ABRT
Jun 04 10:54:57 gnode00 systemd[1]: Unit glusterd.service entered failed state.


The log files has the following entry:
[2015-06-04 07:24:07.098158] D [rpc-transport.c:287:rpc_transport_load]
0-rpc-transport: attempt to load file
/usr/lib64/glusterfs/3.7.1/rpc-transport/socket.so
[2015-06-04 07:24:07.098247] D [socket.c:3816:socket_init] 0-management:
disabling nodelay
[2015-06-04 07:24:07.098273] D [socket.c:3867:socket_init] 0-management:
Configued transport.tcp-user-timeout=0
[2015-06-04 07:24:07.098294] D [socket.c:3938:socket_init] 0-management: SSL
support on the I/O path is NOT enabled
[2015-06-04 07:24:07.098313] D [socket.c:3941:socket_init] 0-management: SSL
support for glusterd is NOT enabled
[2015-06-04 07:24:07.098329] D [socket.c:3958:socket_init] 0-management: using
system polling thread
[2015-06-04 07:24:07.098382] D [glusterd-handler.c:3153:glusterd_rpc_create]
0-management: returning 0
[2015-06-04 07:24:07.098408] D [glusterd-utils.c:1493:glusterd_brick_connect]
0-: Returning 0
[2015-06-04 07:24:07.098428] D [glusterd-utils.c:4387:glusterd_brick_start]
0-management: returning 0 
[2015-06-04 07:24:07.098453] D [glusterd-utils.c:4387:glusterd_brick_start]
0-management: returning 0 
*** buffer overflow detected ***: glusterd terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f0493709a57]
/lib64/libc.so.6(+0x10bc10)[0x7f0493707c10]
/lib64/libc.so.6(+0x10b32b)[0x7f049370732b]
/lib64/libc.so.6(__snprintf_chk+0x78)[0x7f0493707248]
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_volume_defrag_restart+0x191)[0x7f0489ad2931]
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_restart_rebalance+0x82)[0x7f0489ad8aa2]
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_spawn_daemons+0x4f)[0x7f0489ad8b1f]
/lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x7f0494c7a482]
/lib64/libc.so.6(+0x470f0)[0x7f04936430f0]
======= Memory map: ========
7f0478000000-7f0478021000 rw-p 00000000 00:00 0 
7f0478021000-7f047c000000 ---p 00000000 00:00 0 
7f047c000000-7f047c023000 rw-p 00000000 00:00 0 
7f047c023000-7f0480000000 ---p 00000000 00:00 0 
7f0480000000-7f0480021000 rw-p 00000000 00:00 0 
7f0480021000-7f0484000000 ---p 00000000 00:00 0 
7f0484b35000-7f0484d36000 rw-p 00000000 00:00 0                         
[stack:12401]
7f0484d36000-7f0484d37000 ---p 00000000 00:00 0 
7f0484d37000-7f0485537000 rw-p 00000000 00:00 0                         
[stack:12405]
7f0485537000-7f0485538000 ---p 00000000 00:00 0 
7f0485538000-7f0485d38000 rw-p 00000000 00:00 0                         
[stack:12404]
7f0485d38000-7f0485d3d000 r-xp 00000000 08:04 3016                      
/usr/lib64/libnss_dns-2.17.so
7f0485d3d000-7f0485f3c000 ---p 00005000 08:04 3016                      
/usr/lib64/libnss_dns-2.17.so
7f0485f3c000-7f0485f3d000 r--p 00004000 08:04 3016                      
/usr/lib64/libnss_dns-2.17.so
7f0485f3d000-7f0485f3e000 rw-p 00005000 08:04 3016                      
/usr/lib64/libnss_dns-2.17.so
7f0485f3e000-7f0485f49000 r-xp 00000000 08:04 3018                      
/usr/lib64/libnss_files-2.17.so
7f0485f49000-7f0486148000 ---p 0000b000 08:04 3018                      
/usr/lib64/libnss_files-2.17.so
7f0486148000-7f0486149000 r--p 0000a000 08:04 3018                      
/usr/lib64/libnss_files-2.17.so
7f0486149000-7f048614a000 rw-p 0000b000 08:04 3018                      
/usr/lib64/libnss_files-2.17.so
7f048614a000-7f04862f5000 rw-p 00000000 00:00 0 
7f04862f5000-7f048630b000 r-xp 00000000 08:04 3028                      
/usr/lib64/libresolv-2.17.so
7f048630b000-7f048650b000 ---p 00016000 08:04 3028                      
/usr/lib64/libresolv-2.17.so
7f048650b000-7f048650c000 r--p 00016000 08:04 3028                      
/usr/lib64/libresolv-2.17.so
7f048650c000-7f048650d000 rw-p 00017000 08:04 3028                      
/usr/lib64/libresolv-2.17.so
7f048650d000-7f048650f000 rw-p 00000000 00:00 0 
7f048650f000-7f0486512000 r-xp 00000000 08:04 3374                      
/usr/lib64/libkeyutils.so.1.5
7f0486512000-7f0486711000 ---p 00003000 08:04 3374                      
/usr/lib64/libkeyutils.so.1.5
7f0486711000-7f0486712000 r--p 00002000 08:04 3374                      
/usr/lib64/libkeyutils.so.1.5
7f0486712000-7f0486713000 rw-p 00003000 08:04 3374                      
/usr/lib64/libkeyutils.so.1.5
7f0486713000-7f0486720000 r-xp 00000000 08:04 35693                     
/usr/lib64/libkrb5support.so.0.1
7f0486720000-7f0486920000 ---p 0000d000 08:04 35693                     
/usr/lib64/libkrb5support.so.0.1
7f0486920000-7f0486921000 r--p 0000d000 08:04 35693                     
/usr/lib64/libkrb5support.so.0.1
7f0486921000-7f0486922000 rw-p 0000e000 08:04 35693                     
/usr/lib64/libkrb5support.so.0.1
7f0486922000-7f0486951000 r-xp 00000000 08:04 35681                     
/usr/lib64/libk5crypto.so.3.1
7f0486951000-7f0486b50000 ---p 0002f000 08:04 35681                     
/usr/lib64/libk5crypto.so.3.1
7f0486b50000-7f0486b52000 r--p 0002e000 08:04 35681                     
/usr/lib64/libk5crypto.so.3.1
7f0486b52000-7f0486b53000 rw-p 00030000 08:04 35681                     
/usr/lib64/libk5crypto.so.3.1
7f0486b53000-7f0486b54000 rw-p 00000000 00:00 0 
7f0486b54000-7f0486b57000 r-xp 00000000 08:04 3157                      
/usr/lib64/libcom_err.so.2.1
7f0486b57000-7f0486d56000 ---p 00003000 08:04 3157                      
/usr/lib64/libcom_err.so.2.1
7f0486d56000-7f0486d57000 r--p 00002000 08:04 3157                      
/usr/lib64/libcom_err.so.2.1
7f0486d57000-7f0486d58000 rw-p 00003000 08:04 3157                      
/usr/lib64/libcom_err.so.2.1
7f0486d58000-7f0486e2b000 r-xp 00000000 08:04 35691                     
/usr/lib64/libkrb5.so.3.3
7f0486e2b000-7f048702b000 ---p 000d3000 08:04 35691                     
/usr/lib64/libkrb5.so.3.3
7f048702b000-7f0487038000 r--p 000d3000 08:04 35691                     
/usr/lib64/libkrb5.so.3.3
7f0487038000-7f048703b000 rw-p 000e0000 08:04 35691                     
/usr/lib64/libkrb5.so.3.3
7f048703b000-7f0487084000 r-xp 00000000 08:04 4350                      
/usr/lib64/libgssapi_krb5.so.2.2
7f0487084000-7f0487284000 ---p 00049000 08:04 4350                      
/usr/lib64/libgssapi_krb5.so.2.2
7f0487284000-7f0487285000 r--p 00049000 08:04 4350                      
/usr/lib64/libgssapi_krb5.so.2.2
7f0487285000-7f0487287000 rw-p 0004a000 08:04 4350                      
/usr/lib64/libgssapi_krb5.so.2.2
7f0487287000-7f04872e9000 r-xp 00000000 08:04 35705                     
/usr/lib64/libssl.so.1.0.1e
7f04872e9000-7f04874e9000 ---p 00062000 08:04 35705                     
/usr/lib64/libssl.so.1.0.1e
7f04874e9000-7f04874ed000 r--p 00062000 08:04 35705                     
/usr/lib64/libssl.so.1.0.1e
7f04874ed000-7f04874f4000 rw-p 00066000 08:04 35705                     
/usr/lib64/libssl.so.1.0.1e
7f04874f4000-7f0487505000 r-xp 00000000 08:04 135881318                 
/usr/lib64/glusterfs/3.7.1/rpc-transport/socket.so
7f0487505000-7f0487704000 ---p 00011000 08:04 135881318                 
/usr/lib64/glusterfs/3.7.1/rpc-transport/socket.so
7f0487704000-7f0487705000 r--p 00010000 08:04 135881318                 
/usr/lib64/glusterfs/3.7.1/rpc-transport/socket.so
7f0487705000-7f0487709000 rw-p 00011000 08:04 135881318                 
/usr/lib64/glusterfs/3.7.1/rpc-transport/socket.so
7f0487709000-7f0487736000 rw-p 00000000 00:00 0 
7f0487736000-7f0487795000 r-xp 00000000 08:04 3080                      
/usr/lib64/libpcre.so.1.2.0
7f0487795000-7f0487995000 ---p 0005f000 08:04 3080                      
/usr/lib64/libpcre.so.1.2.0
7f0487995000-7f0487996000 r--p 0005f000 08:04 3080                      
/usr/lib64/libpcre.so.1.2.0
7f0487996000-7f0487997000 rw-p 00060000 08:04 3080                      
/usr/lib64/libpcre.so.1.2.0
7f0487997000-7f04879db000 r-xp 00000000 08:04 3054                      
/usr/lib64/libsepol.so.1
7f04879db000-7f0487bda000 ---p 00044000 08:04 3054                      
/usr/lib64/libsepol.so.1
7f0487bda000-7f0487bdb000 r--p 00043000 08:04 3054                      
/usr/lib64/libsepol.so.1
7f0487bdb000-7f0487bdc000 rw-p 00044000 08:04 3054                      
/usr/lib64/libsepol.so.1
7f0487bdc000-7f0487bf1000 r-xp 00000000 08:04 209                       
/usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f0487bf1000-7f0487df0000 ---p 00015000 08:04 209                       
/usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f0487df0000-7f0487df1000 r--p 00014000 08:04 209                       
/usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f0487df1000-7f0487df2000 rw-p 00015000 08:04 209                       
/usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f0487df2000-7f0487e13000 r-xp 00000000 08:04 3091                      
/usr/lib64/libselinux.so.1
7f0487e13000-7f0488013000 ---p 00021000 08:04 3091                      
/usr/lib64/libselinux.so.1
7f0488013000-7f0488014000 r--p 00021000 08:04 3091                      
/usr/lib64/libselinux.so.1
7f0488014000-7f0488015000 rw-p 00022000 08:04 3091                      
/usr/lib64/libselinux.so.1
7f0488015000-7f0488017000 rw-p 00000000 00:00 0 
7f0488017000-7f048801c000 r-xp 00000000 08:04 1647807                   
/usr/lib64/libdevmapper-event.so.1.02
7f048801c000-7f048821c000 ---p 00005000 08:04 1647807                   
/usr/lib64/libdevmapper-event.so.1.02
7f048821c000-7f048821d000 r--p 00005000 08:04 1647807                   
/usr/lib64/libdevmapper-event.so.1.02
7f048821d000-7f048821e000 rw-p 00006000 08:04 1647807                   
/usr/lib64/libdevmapper-event.so.1.02
7f048821e000-7f0488263000 r-xp 00000000 08:04 206103                    
/usr/lib64/libdevmapper.so.1.02
7f0488263000-7f0488462000 ---p 00045000 08:04 206103                    
/usr/lib64/libdevmapper.so.1.02
7f0488462000-7f0488463000 r--p 00044000 08:04 206103                    
/usr/lib64/libdevmapper.so.1.02
7f0488463000-7f0488466000 rw-p 00045000 08:04 206103                    
/usr/lib64/libdevmapper.so.1.02
7f0488466000-7f0488467000 rw-p 00000000 00:00 0 
7f0488467000-7f048849f000 r-xp 00000000 08:04 131775                    
/usr/lib64/libblkid.so.1.1.0
7f048849f000-7f048869e000 ---p 00038000 08:04 131775                    
/usr/lib64/libblkid.so.1.1.0
7f048869e000-7f04886a1000 r--p 00037000 08:04 131775                    
/usr/lib64/libblkid.so.1.1.0
7f04886a1000-7f04886a2000 rw-p 0003a000 08:04 131775                    
/usr/lib64/libblkid.so.1.1.0
7f04886a2000-7f04886a3000 rw-p 00000000 00:00 0 
7f04886a3000-7f04886b4000 r-xp 00000000 08:04 35842                     
/usr/lib64/libudev.so.1.4.0
7f04886b4000-7f04888b3000 ---p 00011000 08:04 35842                     
/usr/lib64/libudev.so.1.4.0
7f04888b3000-7f04888b4000 r--p 00010000 08:04 35842                     
/usr/lib64/libudev.so.1.4.0
7f04888b4000-7f04888b5000 rw-p 00011000 08:04 35842                     
/usr/lib64/libudev.so.1.4.0
7f04888b5000-7f04888b7000 r-xp 00000000 08:04 35806                     
/usr/lib64/liburcu-common.so.1.0.0
7f04888b7000-7f0488ab6000 ---p 00002000 08:04 35806                     
/usr/lib64/liburcu-common.so.1.0.0
7f0488ab6000-7f0488ab7000 r--p 00001000 08:04 35806                     
/usr/lib64/liburcu-common.so.1.0.0
7f0488ab7000-7f0488ab8000 rw-p 00002000 08:04 35806                     
/usr/lib64/liburcu-common.so.1.0.0
7f0488ab8000-7f0488adc000 r-xp 00000000 08:04 3047                      
/usr/lib64/liblzma.so.5.0.99
7f0488adc000-7f0488cdb000 ---p 00024000 08:04 3047                      
/usr/lib64/liblzma.so.5.0.99
7f0488cdb000-7f0488cdc000 r--p 00023000 08:04 3047                      
/usr/lib64/liblzma.so.5.0.99
7f0488cdc000-7f0488cdd000 rw-p 00024000 08:04 3047                      
/usr/lib64/liblzma.so.5.0.99
7f0488cdd000-7f0488dba000 r-xp 00000000 08:04 275744                    
/usr/lib64/liblvm2app.so.2.2
7f0488dba000-7f0488fb9000 ---p 000dd000 08:04 275744                    
/usr/lib64/liblvm2app.so.2.2
7f0488fb9000-7f0488fba000 r--p 000dc000 08:04 275744                    
/usr/lib64/liblvm2app.so.2.2
7f0488fba000-7f0488fc0000 rw-p 000dd000 08:04 275744                    
/usr/lib64/liblvm2app.so.2.2
7f0488fc0000-7f0488fdc000 rw-p 00000000 00:00 0 
7f0488fdc000-7f0488fe2000 r-xp 00000000 08:04 249077                    
/usr/lib64/liburcu-cds.so.1.0.0
7f0488fe2000-7f04891e1000 ---p 00006000 08:04 249077                    
/usr/lib64/liburcu-cds.so.1.0.0
7f04891e1000-7f04891e2000 r--p 00005000 08:04 249077                    
/usr/lib64/liburcu-cds.so.1.0.0
7f04891e2000-7f04891e3000 rw-p 00006000 08:04 249077                    
/usr/lib64/liburcu-cds.so.1.0.0
7f04891e3000-7f04891e9000 r-xp 00000000 08:04 249075                    
/usr/lib64/liburcu-bp.so.1.0.0
7f04891e9000-7f04893e8000 ---p 00006000 08:04 249075                    
/usr/lib64/liburcu-bp.so.1.0.0
7f04893e8000-7f04893e9000 r--p 00005000 08:04 249075                    
/usr/lib64/liburcu-bp.so.1.0.0
7f04893e9000-7f04893ea000 rw-p 00006000 08:04 249075                    
/usr/lib64/liburcu-bp.so.1.0.0
7f04893ea000-7f04894eb000 r-xp 00000000 08:04 3008                      
/usr/lib64/libm-2.17.so
7f04894eb000-7f04896ea000 ---p 00101000 08:04 3008                      
/usr/lib64/libm-2.17.so
7f04896ea000-7f04896eb000 r--p 00100000 08:04 3008                      
/usr/lib64/libm-2.17.so
7f04896eb000-7f04896ec000 rw-p 00101000 08:04 3008                      
/usr/lib64/libm-2.17.so
7f04896ec000-7f048984a000 r-xp 00000000 08:04 1434624                   
/usr/lib64/libxml2.so.2.9.1
7f048984a000-7f0489a49000 ---p 0015e000 08:04 1434624                   
/usr/lib64/libxml2.so.2.9.1
7f0489a49000-7f0489a51000 r--p 0015d000 08:04 1434624                   
/usr/lib64/libxml2.so.2.9.1
7f0489a51000-7f0489a53000 rw-p 00165000 08:04 1434624                   
/usr/lib64/libxml2.so.2.9.1
7f0489a53000-7f0489a55000 rw-p 00000000 00:00 0 
7f0489a5e000-7f0489bb5000 r-xp 00000000 08:04 135881320                 
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so
7f0489bb5000-7f0489db5000 ---p 00157000 08:04 135881320                 
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so
7f0489db5000-7f0489db6000 r--p 00157000 08:04 135881320                 
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so
7f0489db6000-7f0489dc7000 rw-p 00158000 08:04 135881320                 
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so
7f0489dc7000-7f0489dc8000 rw-p 00000000 00:00 0 
7f0489dc8000-7f0489dc9000 ---p 00000000 00:00 0 
7f0489dc9000-7f048a5c9000 rw-p 00000000 00:00 0                         
[stack:12402]
7f048a5c9000-7f048a5ca000 ---p 00000000 00:00 0 
7f048a5ca000-7f048adca000 rw-p 00000000 00:00 0 
7f048adca000-7f048adcb000 ---p 00000000 00:00 0 
7f048adcb000-7f048b5cb000 rw-p 00000000 00:00 0                         
[stack:12400]
7f048b5cb000-7f048b5cc000 ---p 00000000 00:00 0 
7f048b5cc000-7f048bdcc000 rw-p 00000000 00:00 0                         
[stack:12399]
7f048bdcc000-7f04922f3000 r--p 00000000 08:04 201329209                 
/usr/lib/locale/locale-archive
7f04922f3000-7f04935fc000 rw-p 00000000 00:00 0 
7f04935fc000-7f04937b2000 r-xp 00000000 08:04 3000                      
/usr/lib64/libc-2.17.so
7f04937b2000-7f04939b2000 ---p 001b6000 08:04 3000                      
/usr/lib64/libc-2.17.so
7f04939b2000-7f04939b6000 r--p 001b6000 08:04 3000                      
/usr/lib64/libc-2.17.so
7f04939b6000-7f04939b8000 rw-p 001ba000 08:04 3000                      
/usr/lib64/libc-2.17.so
7f04939b8000-7f04939bd000 rw-p 00000000 00:00 0 
7f04939bd000-7f0493b7a000 r-xp 00000000 08:04 4346                      
/usr/lib64/libcrypto.so.1.0.1e
7f0493b7a000-7f0493d7a000 ---p 001bd000 08:04 4346                      
/usr/lib64/libcrypto.so.1.0.1e
7f0493d7a000-7f0493d94000 r--p 001bd000 08:04 4346                      
/usr/lib64/libcrypto.so.1.0.1e
7f0493d94000-7f0493da0000 rw-p 001d7000 08:04 4346                      
/usr/lib64/libcrypto.so.1.0.1e
7f0493da0000-7f0493da4000 rw-p 00000000 00:00 0 
7f0493da4000-7f0493dba000 r-xp 00000000 08:04 3026                      
/usr/lib64/libpthread-2.17.so
7f0493dba000-7f0493fba000 ---p 00016000 08:04 3026                      
/usr/lib64/libpthread-2.17.so
7f0493fba000-7f0493fbb000 r--p 00016000 08:04 3026                      
/usr/lib64/libpthread-2.17.so
7f0493fbb000-7f0493fbc000 rw-p 00017000 08:04 3026                      
/usr/lib64/libpthread-2.17.so
7f0493fbc000-7f0493fc0000 rw-p 00000000 00:00 0 
7f0493fc0000-7f0493fc3000 r-xp 00000000 08:04 3006                      
/usr/lib64/libdl-2.17.so
7f0493fc3000-7f04941c2000 ---p 00003000 08:04 3006                      
/usr/lib64/libdl-2.17.so
7f04941c2000-7f04941c3000 r--p 00002000 08:04 3006                      
/usr/lib64/libdl-2.17.so
7f04941c3000-7f04941c4000 rw-p 00003000 08:04 3006                      
/usr/lib64/libdl-2.17.so
7f04941c4000-7f04941cb000 r-xp 00000000 08:04 3030                      
/usr/lib64/librt-2.17.so
7f04941cb000-7f04943ca000 ---p 00007000 08:04 3030                      
/usr/lib64/librt-2.17.so
7f04943ca000-7f04943cb000 r--p 00006000 08:04 3030                      
/usr/lib64/librt-2.17.so
7f04943cb000-7f04943cc000 rw-p 00007000 08:04 3030                      
/usr/lib64/librt-2.17.so
7f04943cc000-7f04943d0000 r-xp 00000000 08:04 131774                    
/usr/lib64/libuuid.so.1.3.0
7f04943d0000-7f04945cf000 ---p 00004000 08:04 131774                    
/usr/lib64/libuuid.so.1.3.0
7f04945cf000-7f04945d0000 r--p 00003000 08:04 131774                    
/usr/lib64/libuuid.so.1.3.0
7f04945d0000-7f04945d1000 rw-p 00004000 08:04 131774                    
/usr/lib64/libuuid.so.1.3.0
7f04945d1000-7f04945e6000 r-xp 00000000 08:04 3094                      
/usr/lib64/libz.so.1.2.7
7f04945e6000-7f04947e5000 ---p 00015000 08:04 3094                      
/usr/lib64/libz.so.1.2.7
7f04947e5000-7f04947e6000 r--p 00014000 08:04 3094                      
/usr/lib64/libz.so.1.2.7
7f04947e6000-7f04947e7000 rw-p 00015000 08:04 3094                      
/usr/lib64/libz.so.1.2.7
7f04947e7000-7f04947fe000 r-xp 00000000 08:04 35840                     
/usr/lib64/libgfxdr.so.0.0.1
7f04947fe000-7f04949fd000 ---p 00017000 08:04 35840                     
/usr/lib64/libgfxdr.so.0.0.1
7f04949fd000-7f04949fe000 r--p 00016000 08:04 35840                     
/usr/lib64/libgfxdr.so.0.0.1
7f04949fe000-7f04949ff000 rw-p 00017000 08:04 35840                     
/usr/lib64/libgfxdr.so.0.0.1
7f04949ff000-7f0494a19000 r-xp 00000000 08:04 127607                    
/usr/lib64/libgfrpc.so.0.0.1
7f0494a19000-7f0494c18000 ---p 0001a000 08:04 127607                    
/usr/lib64/libgfrpc.so.0.0.1
7f0494c18000-7f0494c19000 r--p 00019000 08:04 127607                    
/usr/lib64/libgfrpc.so.0.0.1
7f0494c19000-7f0494c1a000 rw-p 0001a000 08:04 127607                    
/usr/lib64/libgfrpc.so.0.0.1
7f0494c1a000-7f0494cd5000 r-xp 00000000 08:04 39467                     
/usr/lib64/libglusterfs.so.0.0.1
7f0494cd5000-7f0494ed5000 ---p 000bb000 08:04 39467                     
/usr/lib64/libglusterfs.so.0.0.1
7f0494ed5000-7f0494ed7000 r--p 000bb000 08:04 39467                     
/usr/lib64/libglusterfs.so.0.0.1
7f0494ed7000-7f0494ed8000 rw-p 000bd000 08:04 39467                     
/usr/lib64/libglusterfs.so.0.0.1
7f0494ed8000-7f0494edb000 rw-p 00000000 00:00 0 
7f0494edb000-7f0494efc000 r-xp 00000000 08:04 2993                      
/usr/lib64/ld-2.17.so
7f0494efd000-7f04950f2000 rw-p 00000000 00:00 0 
7f04950f2000-7f04950f9000 r--s 00000000 08:04 67111173                  
/usr/lib64/gconv/gconv-modules.cache
7f04950f9000-7f04950fc000 rw-p 00000000 00:00 0 
7f04950fc000-7f04950fd000 r--p 00021000 08:04 2993                      
/usr/lib64/ld-2.17.so
7f04950fd000-7f04950fe000 rw-p 00022000 08:04 2993                      
/usr/lib64/ld-2.17.so
7f04950fe000-7f04950ff000 rw-p 00000000 00:00 0 
7f04950ff000-7f0495113000 r-xp 00000000 08:04 35418                     
/usr/sbin/glusterfsd
7f0495313000-7f0495314000 r--p 00014000 08:04 35418                     
/usr/sbin/glusterfsd
7f0495314000-7f0495316000 rw-p 00015000 08:04 35418                     
/usr/sbin/glusterfsd
7f0496c8d000-7f0496da3000 rw-p 00000000 00:00 0                          [heap]
7fff0b610000-7fff0b631000 rw-p 00000000 00:00 0                         
[stack]
7fff0b714000-7fff0b716000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                 
[vsyscall]
[2015-06-04 07:24:07.101320] D [logging.c:1765:gf_log_flush_extra_msgs]
0-logging-infra: Log buffer size reduced. About to flush 5 extra log messages
The message "D [MSGID: 0] [glusterd-svc-mgmt.c:116:glusterd_svc_init_common]
0-management: Returning 0" repeated 4 times between [2015-06-04
07:24:04.537856] and [2015-06-04 07:24:04.539461]
[2015-06-04 07:24:07.101384] D [logging.c:1768:gf_log_flush_extra_msgs]
0-logging-infra: Just flushed 5 extra log messages
pending frames:
frame : type(0) op(0)
patchset: git://git.gluster.com/glusterfs.git
signal received: 6
time of crash: 
2015-06-04 07:24:07
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 3.7.1
/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xb2)[0x7f0494c3fd92]
/lib64/libglusterfs.so.0(gf_print_trace+0x32d)[0x7f0494c5a9ed]
/lib64/libc.so.6(+0x35650)[0x7f0493631650]
/lib64/libc.so.6(gsignal+0x37)[0x7f04936315d7]
/lib64/libc.so.6(abort+0x148)[0x7f0493632cc8]
/lib64/libc.so.6(+0x75e07)[0x7f0493671e07]
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f0493709a57]
/lib64/libc.so.6(+0x10bc10)[0x7f0493707c10]
/lib64/libc.so.6(+0x10b32b)[0x7f049370732b]
/lib64/libc.so.6(__snprintf_chk+0x78)[0x7f0493707248]
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_volume_defrag_restart+0x191)[0x7f0489ad2931]
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_restart_rebalance+0x82)[0x7f0489ad8aa2]
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_spawn_daemons+0x4f)[0x7f0489ad8b1f]
/lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x7f0494c7a482]
/lib64/libc.so.6(+0x470f0)[0x7f04936430f0]
---------
Aborted (core dumped)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.


More information about the Bugs mailing list