Fwd: [Gluster-devel] GlusterFS clients getting disconnected from servers with signals 6 and 11

Ioannis Aslanidis iaslanidis at flumotion.com
Mon Apr 28 10:40:37 UTC 2008


Hello everyone,

Now I have the latest version on the servers and the clients. Exactly 
the same version. Last version available. I experimented the issue once 
again.

The error happened at 12:21 aprox.

In the logs of the server I only have the following:

2008-04-28 12:30:21 E [server-protocol.c:5190:mop_getspec] server: 
Unable to open /etc/glusterfs/glusterfs-client.vol.192.168.128.102 (No 
such file or directory)
2008-04-28 12:30:21 E [protocol.c:271:gf_block_unserialize_transport] 
server: EOF from peer (192.168.128.102:1022)

In the client, if I run ls on the mounted point I get the following:

ls: /mnt/file01: Transport endpoint is not connected

In the logs of the client I have the following:

TLA Repo Revision: glusterfs--mainline--2.5--patch-751
Time : 2008-04-28 12:21:09
Signal Number : 11

/usr/sbin/glusterfs -s file01.priv -l /var/log/glusterfs/glusterfs.log 
-L ERROR /mnt/file01
volume fuse
   type mount/fuse
   option direct-io-mode 1
   option entry-timeout 1
   option attr-timeout 1
   option mount-point /mnt/file01
   subvolumes iocache
end-volume

volume iocache
   type performance/io-cache
   option page-size 256KB
   option cache-size 64MB
   subvolumes readahead
end-volume

volume readahead
   type performance/read-ahead
   option page-count 2
   option page-size 256kB
   subvolumes filedata
end-volume

volume filedata
   type protocol/client
   option remote-subvolume filedata
   option remote-host file01.priv
   option transport-type tcp/client
end-volume

frame : type(1) op(13)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(1) op(13)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)

[0x879420]
/usr/lib/glusterfs/1.3.8pre6/xlator/performance/io-cache.so(ioc_frame_fill+0x53)[0x972bc3]
/usr/lib/glusterfs/1.3.8pre6/xlator/performance/io-cache.so(ioc_page_wakeup+0x67)[0x973097]
/usr/lib/glusterfs/1.3.8pre6/xlator/performance/io-cache.so(ioc_inode_wakeup+0xb2)[0x973a32]
/usr/lib/glusterfs/1.3.8pre6/xlator/performance/io-cache.so(ioc_cache_validate_cbk+0xae)[0x96ef9e]
/usr/lib/glusterfs/1.3.8pre6/xlator/performance/read-ahead.so(ra_fstat_cbk+0xa8)[0x712f78]
/usr/lib/glusterfs/1.3.8pre6/xlator/protocol/client.so(client_fstat_cbk+0xcc)[0x75df3c]
/usr/lib/glusterfs/1.3.8pre6/xlator/protocol/client.so(notify+0xb6e)[0x75c41e]
/usr/lib/libglusterfs.so.0(transport_notify+0x38)[0x466d8ed8]
/usr/lib/libglusterfs.so.0(sys_epoll_iteration+0xd6)[0x466d9e66]
/usr/lib/libglusterfs.so.0(poll_iteration+0x98)[0x466d9088]
[glusterfs](main+0x8a6)[0x804a3e6]
/lib/libc.so.6(__libc_start_main+0xdc)[0x4894a7e4]
[glusterfs][0x8049601]
---------


In order to unmount and remount, I had to use the lazy flag.

If you need any further details, please do not hesitate to ask for them.


TIA for your help.

Kind regards,

Ioannis Aslanidis


Raghavendra G wrote:
> Hi loannis,
> 
> On Fri, Apr 25, 2008 at 12:06 PM, Ioannis Aslanidis 
> <iaslanidis at flumotion.com <mailto:iaslanidis at flumotion.com>> wrote:
> 
>     Hi there,
> 
>     We have upgraded our servers and clients to the latest version of
>     glusterfs. I hope that the problem will not reproduce. If it does, I
>     will report back to the mailing list.
> 
>     I am still wondering though, on the new versions, what do the
>     following messages on the server mean:
> 
>     2008-04-24 05:26:17 E
>     [server-protocol.c:6021:server_protocol_interpret] server: bound_xl
>     is null
> 
> 
> bound_xl  null indicates failure of authentication of client from server.
>  
> 
> 
> 
> 
>     2008-04-24 18:17:35 E [server-protocol.c:2136:server_stub_cbk]
>     server: returning ENOENT: -1 (2)
> 
> 
>     Thanks for your help.
> 
>     Regards,
> 
>     Ioannis Aslanidis
> 
> 
>     Raghavendra G wrote:
> 
>         resending to list.
> 
>         ---------- Forwarded message ----------
>         From: Raghavendra G <raghavendra.hg at gmail.com
>         <mailto:raghavendra.hg at gmail.com>>
>         Date: Fri, Apr 25, 2008 at 10:58 AM
>         Subject: Re: [Gluster-devel] GlusterFS clients getting
>         disconnected from
>         servers with signals 6 and 11
>         To: Ioannis Aslanidis <iaslanidis at flumotion.com
>         <mailto:iaslanidis at flumotion.com>>
> 
> 
>         Hi loannis,
> 
>         Do the system on which client is running uses poll or epoll? you
>         can find it
>         in the summary printed at the end of ./configure. Also is it
>         possible to
>         attach client side log file.
> 
>         regards,
> 
>         On Tue, Apr 22, 2008 at 12:42 PM, Ioannis Aslanidis <
>         iaslanidis at flumotion.com <mailto:iaslanidis at flumotion.com>> wrote:
> 
>             Hello,
> 
>             We are currently using server version 1.3.8-0.6 and client
>             version
>             1.3.8-0.8. Do you think that updating to a newer version
>             could solve the
>             issues below? If you need any further information, please do
>             not hesitate to
>             ask for it.
> 
>             The log of the server is the following:
> 
>             2008-04-21 16:07:48 E
>             [protocol.c:254:gf_block_unserialize_transport]
>             server: EOF from peer (192.168.128.101:1021
>             <http://192.168.128.101:1021>)
>             2008-04-21 16:07:48 C [tcp.c:81:tcp_disconnect] server:
>             connection
>             disconnected
>             2008-04-21 16:13:15 E
>             [protocol.c:254:gf_block_unserialize_transport]
>             server: EOF from peer (192.168.128.101:1023
>             <http://192.168.128.101:1023>)
>             2008-04-21 16:13:15 C [tcp.c:81:tcp_disconnect] server:
>             connection
>             disconnected
>             2008-04-21 17:37:41 E
>             [protocol.c:254:gf_block_unserialize_transport]
>             server: EOF from peer (192.168.128.101:1022
>             <http://192.168.128.101:1022>)
>             2008-04-21 17:37:41 C [tcp.c:81:tcp_disconnect] server:
>             connection
>             disconnected
>             2008-04-21 17:49:53 E
>             [protocol.c:254:gf_block_unserialize_transport]
>             server: EOF from peer (192.168.128.101:1023
>             <http://192.168.128.101:1023>)
>             2008-04-21 17:49:53 C [tcp.c:81:tcp_disconnect] server:
>             connection
>             disconnected
>             2008-04-21 20:01:17 E
>             [protocol.c:254:gf_block_unserialize_transport]
>             server: EOF from peer (192.168.128.104:1019
>             <http://192.168.128.104:1019>)
>             2008-04-21 20:01:17 C [tcp.c:81:tcp_disconnect] server:
>             connection
>             disconnected
>             2008-04-21 20:15:30 E
>             [protocol.c:254:gf_block_unserialize_transport]
>             server: EOF from peer (192.168.128.104:1023
>             <http://192.168.128.104:1023>)
>             2008-04-21 20:15:30 C [tcp.c:81:tcp_disconnect] server:
>             connection
>             disconnected
>             2008-04-21 22:05:50 C [tcp.c:81:tcp_disconnect] server:
>             connection
>             disconnected
>             2008-04-21 22:16:42 E
>             [protocol.c:254:gf_block_unserialize_transport]
>             server: EOF from peer (192.168.128.102:1023
>             <http://192.168.128.102:1023>)
>             2008-04-21 22:16:42 C [tcp.c:81:tcp_disconnect] server:
>             connection
>             disconnected
> 
>             The log of the clients:
> 
>             ---------
>             got signal (11), printing backtrace
>             ---------
>             [0xd9f420]
>             /lib/libc.so.6[0x4ef12450]
>             /lib/libc.so.6(calloc+0x8e)[0x4ef139a2]
>             /usr/lib/glusterfs/1.3.8/xlator/performance/read-ahead.so[0xe20551]
>             /usr/lib/glusterfs/1.3.8/xlator/protocol/client.so[0x30718b]
>             /usr/lib/glusterfs/1.3.8/xlator/protocol/client.so(notify+0xb6e)[0x30ac0e]
>             /usr/lib/libglusterfs.so.0(transport_notify+0x37)[0x45c8eed7]
>             /usr/lib/libglusterfs.so.0(sys_epoll_iteration+0xd7)[0x45c8fc17]
>             /usr/lib/libglusterfs.so.0(poll_iteration+0x7c)[0x45c8f00c]
>             [glusterfs](main+0x454)[0x804a864]
>             /lib/libc.so.6(__libc_start_main+0xdc)[0x4eec37e4]
>             [glusterfs][0x804a381]
>             ---------
> 
>             ---------
>             got signal (6), printing backtrace
>             ---------
>             [0x480420]
>             [0x480410]
>             /lib/libc.so.6(gsignal+0x49)[0x4eed6159]
>             /lib/libc.so.6(abort+0x103)[0x4eed76e3]
>             /lib/libc.so.6[0x4ef0aa1b]
>             /lib/libc.so.6[0x4ef12660]
>             /lib/libc.so.6(malloc+0x74)[0x4ef13c7c]
>             /lib/libc.so.6(vasprintf+0x23)[0x4ef09a07]
>             /lib/libc.so.6(__asprintf+0x2e)[0x4eef135e]
>             /usr/lib/libglusterfs.so.0(dict_set+0x26b)[0x45c85b6b]
>             [glusterfs][0x804b3d8]
>             [glusterfs][0x804bfff]
>             /lib/libfuse.so.2[0x45c6e359]
>             /lib/libfuse.so.2[0x45c6f29d]
>             /lib/libfuse.so.2(fuse_session_process+0x26)[0x45c70d96]
>             [glusterfs][0x804fcc8]
>             /lib/libpthread.so.0[0x4f1f03b6]
>             /lib/libc.so.6(__clone+0x5e)[0x4ef7633e]
>             ---------
> 
> 
>             Regards,
> 
>             Ioannis
> 
> 
>             _______________________________________________
>             Gluster-devel mailing list
>             Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
>             http://lists.nongnu.org/mailman/listinfo/gluster-devel
> 
> 
> 
> 
> 
> 
> 
> -- 
> Raghavendra G
> 
> A centipede was happy quite, until a toad in fun,
> Said, "Prey, which leg comes after which?",
> This raised his doubts to such a pitch,
> He fell flat into the ditch,
> Not knowing how to run.
> -Anonymous





More information about the Gluster-devel mailing list