[Gluster-users] activating bail-out :frame sent = 2009-04-30 17:59:10. transport-timeout = 600

Thu Apr 30 16:03:40 UTC 2009

Hello,

I'm trying rc8 on Centos 5.3 with 4 nodes. Node 1 is server, node2-4 are 
clients.

If I use many rsync command on node3 for writing to server, and node2 
for reading from server. Everything is good.

But when adding one more client is node4 using many rsync command for 
writing to server, and node2 still read from server. Then server hang. I 
can't stop or kill glusterfsd process.

I got client's logs:

/2009-04-30 18:09:15 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 17:59:10. transport-timeout 
= 600
2009-04-30 18:09:15 W [fuse-bridge.c:297:need_fresh_lookup] fuse-bridge: 
revalidate of /mirror failed (Transport endpoint is not connected)
2009-04-30 18:09:15 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 17:59:05. transport-timeout 
= 600
2009-04-30 18:09:15 W [fuse-bridge.c:297:need_fresh_lookup] fuse-bridge: 
revalidate of /mirror_tmp failed (Transport endpoint is not connected)
2009-04-30 18:09:15 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 17:59:05. transport-timeout 
= 600
2009-04-30 18:09:15 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 
22093813: LOOKUP() /mirror/mozilla/firefox/releases/3.0.10/mac => -1 
(Transport endpoint is not connected)
2009-04-30 18:09:15 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 17:59:05. transport-timeout 
= 600
2009-04-30 18:09:15 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 
22093806: LOOKUP() /mirror/mozilla/firefox => -1 (Transport endpoint is 
not connected)
2009-04-30 18:09:35 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 17:59:25. transport-timeout 
= 600
2009-04-30 18:09:35 E [fuse-bridge.c:1548:fuse_readv_cbk] 
glusterfs-fuse: 22093817: READ => -1 (Transport endpoint is not connected)
2009-04-30 18:09:35 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 17:59:25. transport-timeout 
= 600
2009-04-30 18:09:35 E [fuse-bridge.c:1548:fuse_readv_cbk] 
glusterfs-fuse: 22093816: READ => -1 (Transport endpoint is not connected)
2009-04-30 18:10:15 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 18:00:06. transport-timeout 
= 600
2009-04-30 18:10:15 W [fuse-bridge.c:297:need_fresh_lookup] fuse-bridge: 
revalidate of /mirror failed (Transport endpoint is not connected)
2009-04-30 18:18:56 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 18:08:55. transport-timeout 
= 600
2009-04-30 18:18:56 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 
22093812: LOOKUP() /mirror_tmp => -1 (Transport endpoint is not connected)
2009-04-30 18:19:16 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 18:09:15. transport-timeout 
= 600
2009-04-30 18:19:16 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 
22093814: LOOKUP() /mirror_tmp => -1 (Transport endpoint is not connected)
2009-04-30 18:19:16 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 18:09:15. transport-timeout 
= 600
2009-04-30 18:19:16 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 
22093815: LOOKUP() /mirror => -1 (Transport endpoint is not connected)
2009-04-30 18:19:36 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 18:09:35. transport-timeout 
= 600
2009-04-30 18:19:36 E [fuse-bridge.c:923:fuse_err_cbk] glusterfs-fuse: 
22093855: FLUSH() ERR => -1 (Transport endpoint is not connected)
2009-04-30 18:19:36 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 18:09:35. transport-timeout 
= 600
2009-04-30 18:19:36 E [fuse-bridge.c:923:fuse_err_cbk] glusterfs-fuse: 
22093853: FLUSH() ERR => -1 (Transport endpoint is not connected)
2009-04-30 18:20:16 E [client-protocol.c:290:call_bail] mr-store: 
activating bail-out :frame sent = 2009-04-30 18:10:15. transport-timeout 
= 600
2009-04-30 18:20:16 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 
22093818: LOOKUP() /mirror => -1 (Transport endpoint is not connected)/

Here's my configurations:

Client:
volume  mr-store
        type    protocol/client
        option  transport-type  tcp/client
        option  remote-host     ip's server
        option  remote-subvolume        mr-iot
end-volume

Server
volume mr-brick
    type storage/posix
    option directory /store
end-volume

volume mr-locks
     type features/posix-locks
     subvolumes mr-brick
end-volume

volume mr-iot
     type performance/io-thread
     option thread-count   8
     subvolumes mr-brick
end-volume

volume server
  type protocol/server
  option transport-type tcp/server
  option auth.addr.mr-iot.allow  *
  subvolumes mr-iot
end-volume

Can anyone help me fix issue ?

Thx,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20090430/de50ece3/attachment.html>