[Gluster-users] gfs vs nfs: abysmal performance difference

[Paolo Pisati]

Anand Avati wrote:
> Paolo,
>   Do you have a core dump on your system (in /)? Can you get us a backtrace from it?
>   
 > sudo gdb /usr/local/gfs/sbin/glusterfsd /glusterfsd.core
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
Core was generated by `glusterfsd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/local/gfs/lib/libglusterfs.so.0...done.
Loaded symbols for /usr/local/gfs/lib/libglusterfs.so.0
Reading symbols from /usr/local/lib/libiconv.so.3...done.
Loaded symbols for /usr/local/lib/libiconv.so.3
Reading symbols from /lib/libthr.so.3...done.
Loaded symbols for /lib/libthr.so.3
Reading symbols from /lib/libc.so.7...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/protocol/client.so...done.
Loaded symbols for 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/protocol/client.so
Reading symbols from 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/performance/read-ahead.so...done.
Loaded symbols for 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/performance/read-ahead.so
Reading symbols from 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/performance/io-cache.so...done.
Loaded symbols for 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/performance/io-cache.so
Reading symbols from 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/performance/write-behind.so...done.
Loaded symbols for 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/performance/write-behind.so
Reading symbols from 
/usr/local/gfs/lib/glusterfs/2.0.0/xlator/mount/fuse.so...done.
Loaded symbols for /usr/local/gfs/lib/glusterfs/2.0.0/xlator/mount/fuse.so
Reading symbols from /usr/local/lib/libfuse.so.2...done.
Loaded symbols for /usr/local/lib/libfuse.so.2
Reading symbols from 
/usr/local/gfs/lib/glusterfs/2.0.0/transport/socket.so...done.
Loaded symbols for /usr/local/gfs/lib/glusterfs/2.0.0/transport/socket.so
Reading symbols from /libexec/ld-elf.so.1...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x0000000800994152 in pthread_spin_lock () from /lib/libthr.so.3
[New Thread 0x800d02560 (LWP 100273)]
[New Thread 0x800d02240 (LWP 100266)]
[New Thread 0x800d020b0 (LWP 100338)]
(gdb) bt
#0  0x0000000800994152 in pthread_spin_lock () from /lib/libthr.so.3
#1  0x0000000801619c9f in client_forget (this=0x800d200c0, 
inode=0x801f0c040) at client-protocol.c:4131
#2  0x0000000800661874 in __inode_destroy (inode=0x801f0c040) at inode.c:283
#3  0x00000008006631cb in inode_table_prune (table=0x800da8110) at 
inode.c:996
#4  0x00000008006628ef in inode_link (inode=0x801f0c0d0, 
parent=0x800da81a0, name=0x80605f041 "Bonnie.94754", 
stbuf=0x7fffffffe150) at inode.c:689
#5  0x0000000801a4f198 in fuse_create_cbk (frame=0x801f12060, 
cookie=0x801f03ab0, this=0x800d20340, op_ret=0, op_errno=0, 
fd=0x801f02080, inode=0x801f0c0d0, buf=0x7fffffffe150)
    at fuse-bridge.c:1414
#6  0x0000000801941091 in wb_create_cbk (frame=0x801f03ab0, 
cookie=0x801f03b50, this=0x800d202a0, op_ret=0, op_errno=0, 
fd=0x801f02080, inode=0x801f0c0d0, buf=0x7fffffffe150)
    at write-behind.c:1028
#7  0x0000000801834bc8 in ioc_create_cbk (frame=0x801f03b50, 
cookie=0x801f03ba0, this=0x800d20200, op_ret=0, op_errno=0, 
fd=0x801f02080, inode=0x801f0c0d0, buf=0x7fffffffe150)
    at io-cache.c:732
#8  0x000000080172a236 in ra_create_cbk (frame=0x801f03ba0, 
cookie=0x801f03bf0, this=0x800d20160, op_ret=0, op_errno=0, 
fd=0x801f02080, inode=0x801f0c0d0, buf=0x7fffffffe150)
    at read-ahead.c:170
#9  0x000000080161b57c in client_create_cbk (frame=0x801f03bf0, 
hdr=0x800d8e800, hdrlen=116, iobuf=0x0) at client-protocol.c:4602
#10 0x000000080162056e in protocol_client_interpret (this=0x800d200c0, 
trans=0x800d0dc00, hdr_p=0x800d8e800 "", hdrlen=116, iobuf=0x0) at 
client-protocol.c:6652
#11 0x00000008016211f9 in protocol_client_pollin (this=0x800d200c0, 
trans=0x800d0dc00) at client-protocol.c:6942
#12 0x0000000801621387 in notify (this=0x800d200c0, event=2, 
data=0x800d0dc00) at client-protocol.c:6986
#13 0x0000000801c791f9 in socket_event_poll_in (this=0x800d0dc00) at 
socket.c:713
#14 0x0000000801c794f7 in socket_event_handler (fd=9, idx=2, 
data=0x800d0dc00, poll_in=1, poll_out=0, poll_err=0) at socket.c:812
#15 0x00000008006739d4 in event_dispatch_poll_handler 
(event_pool=0x800d1f0b0, ufds=0x800d0cdc0, i=2) at event.c:372
#16 0x0000000800673c36 in event_dispatch_poll (event_pool=0x800d1f0b0) 
at event.c:453
#17 0x0000000800673f03 in event_dispatch (event_pool=0x800d1f0b0) at 
event.c:975
#18 0x00000000004060e7 in main (argc=8, argv=0x7fffffffec40) at 
glusterfsd.c:1154
(gdb)

-- 

Paolo Pisati
Reparto Sistemi
---------
ContactLab
Via Natale Battaglia, 10
20127 Milano (Italy)
Tel. +39 02 28 31 181
Fax +39 02 70 03 02 69.
http://www.contactlab.com