[Gluster-devel] Unify/AFR crashes

[NovA]

Hi!

I'm using glusterfs-1.3.9tla790 unify (without AFR) and also
periodically has troubles with NS. Sometimes, when many files are
copied to unify volume, the operation is stalled. And the client log
is flooded by messages like:
W [client-protocol.c:332:client_protocol_xfer] c-ns: not connected at
the moment to submit frame type(1) op(35)
The glusterfs server not really crash, but eat 100% CPU. Restarting it
restores the normal work.

There are also many other messages concerning namespace, like
------
W [client-protocol.c:1711:client_closedir] c-ns: no proper fd found, returning
E [client-protocol.c:4834:client_protocol_cleanup] c-ns: forced
unwinding frame type(1) op(34) reply=@0xc165b0
E [client-protocol.c:4430:client_lookup_cbk] c-ns: no proper reply
from server, returning ENOTCONN
E [unify.c:182:unify_lookup_cbk] bricks: c-ns returned 107
------
W [client-protocol.c:205:call_bail] c-ns: activating bail-out. pending
frames = 3. last sent = 2008-07-28 16:05:38. last received =
1970-01-01 03:00:00 transport-timeout = 42
C [client-protocol.c:212:call_bail] c-ns: bailing transport
------
      Note the strange last received date...
-----
W [client-protocol.c:280:client_protocol_xfer] c-ns: attempting to
pipeline request type(1) op(34) with handshake
-----
W [client-protocol.c:4784:client_protocol_cleanup] c-ns: cleaning up
state in transport object 0x64
E [client-protocol.c:4834:client_protocol_cleanup] c-ns: forced
unwinding frame type(1) op(34)  reply=@0x66c140
E [client-protocol.c:4430:client_lookup_cbk] c-ns: no proper reply
from server, returning ENOTCONN
----
[client-protocol.c:4784:client_protocol_cleanup] c-ns: cleaning up
state in transport object 0x64d310
[client-protocol.c:4834:client_protocol_cleanup] c-ns: forced
unwinding frame type(1) op(36) reply=@0x2aaab0048720
[client-protocol.c:4215:client_setdents_cbk] c-ns: no proper reply
from server, returning ENOTCONN
[client-protocol.c:4834:client_protocol_cleanup] c-ns: forced
unwinding frame type(1) op(36) reply=@0x2aaab0048720
[client-protocol.c:4215:client_setdents_cbk] c-ns: no proper reply
from server, returning ENOTCONN
[client-protocol.c:4834:client_protocol_cleanup] c-ns: forced
unwinding frame type(1) op(23) reply=@0x2aaab0048720
[client-protocol.c:3310:client_getdents_cbk] c-ns: no proper reply
from server, returning ENOTCONN
[client-protocol.c:1711:client_closedir] c-ns: no proper fd found, returning
[client-protocol.c:4834:client_protocol_cleanup] c-ns: forced
unwinding frame type(1) op(34) reply=@0x2aaab0048720
[client-protocol.c:4430:client_lookup_cbk] c-ns: no proper reply from
server, returning ENOTCONN
[client-protocol.c:4834:client_protocol_cleanup] c-ns: forced
unwinding frame type(1) op(22) reply=@0x2aaab0048720
[client-protocol.c:3767:client_opendir_cbk] c-ns: no proper reply from
server, returning ENOTCONN
----
E [client-protocol.c:1884:client_fstat] c-ns: : returning EBADFD
E [unify.c:118:unify_buf_cbk] bricks: c-ns returned 77
----

All these are from different places of the log (rather huge now), and
probably not connected to each other. It's just to show different
types of warnings and errors I've really saw.

The name-space in my case is just a folder on common ext3 volume. It's
for testing purposes, I'm going to use separate reiserfs volume for it
later. But probably it's somehow connected with NS problems which I'm
seeing now.

With best regards,
  Andrey


2008/8/5 Dmitriy Kotkin <kotdv at intergam.com>:
> I'm using glusterfs-1.3.10 (the same for 1.3.9tla787) and unify over afrs
> setup.
> The problem is that during intense fs ops every first node listed in AFRs
> crashes (activating boiling transport and so on).