[Gluster-devel] error gf_block_unserialize_transport server: EOFfrom peer
Rob Derr
rderr at weatherflow.com
Tue Jan 29 15:59:29 UTC 2008
Gowda,
I'm running gluster in a production environment and getting similar errors
as Rohan's.
We have 8 computers running as servers and 11 clients connecting to the
servers (including the 8 clients that are also servers). The servers are
grouped into 4 mirrors and then unified as one large filesystem.
This is our current client spec file:
volume client1
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.46
option remote-subvolume brick
option transport-timeout 10
end-volume
volume client2
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.47
option remote-subvolume brick
option transport-timeout 10
end-volume
volume client3
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.48
option remote-subvolume brick
option transport-timeout 10
end-volume
volume client4
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.49
option remote-subvolume brick
option transport-timeout 10
end-volume
volume client5
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.50
option remote-subvolume brick
option transport-timeout 10
end-volume
volume client6
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.65
option remote-subvolume brick
option transport-timeout 10
end-volume
volume client7
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.43
option remote-subvolume brick
option transport-timeout 10
end-volume
volume client8
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.44
option remote-subvolume brick
option transport-timeout 10
end-volume
### Client Namespaves
volume client1ns
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.46
option remote-subvolume brickns
option transport-timeout 10
end-volume
volume client2ns
type protocol/client
option transport-type tcp/client
option remote-host 192.168.2.47
option remote-subvolume brickns
option transport-timeout 10
end-volume
volume brick-ns
type cluster/afr
subvolumes client1ns client2ns
end-volume
volume afr1
type cluster/afr
subvolumes client1 client2
end-volume
volume afr2
type cluster/afr
subvolumes client3 client4
end-volume
volume afr3
type cluster/afr
subvolumes client5 client6
end-volume
volume afr4
type cluster/afr
subvolumes client7 client8
end-volume
# Unify
volume bricks
type cluster/unify
subvolumes afr1 afr2 afr3 afr4
option namespace brick-ns
end-volume
This is the server spec file:
volume brick
type storage/posix # POSIX FS translator
option directory /var/gluster/export # Export this directory
end-volume
volume brickns
type storage/posix # POSIX FS translator
option directory /var/gluster/export-ns # Export this directory
end-volume
### Add network serving capability to above brick.
volume server
type protocol/server
option transport-type tcp/server # For TCP/IP transport
subvolumes brick brickns
option auth.ip.brick.allow 192.168.2.* # Allow access to "brick" volume
option auth.ip.brickns.allow 192.168.2.* # Allow access to "brick" volume
end-volume
Here are the server logs from the last time the servers failed:
2008-01-29 05:22:44 E [protocol.c:254:gf_block_unserialize_transport]
server: EOF from peer (192.168.2.49:1023)
2008-01-29 05:22:44 E [posix.c:1209:posix_close] brick: pfd->dir is 0x512ff0
(not NULL) for file fd=0x50ed50
2008-01-29 05:22:44 E [posix.c:1209:posix_close] brick: pfd->dir is
0x2aaaab002610 (not NULL) for file fd=0x2aaaab000d10
2008-01-29 05:22:44 C [tcp.c:81:tcp_disconnect] server: connection
disconnected
2008-01-29 05:25:19 E [protocol.c:254:gf_block_unserialize_transport]
server: EOF from peer (192.168.2.43:1001)
2008-01-29 05:25:19 C [tcp.c:81:tcp_disconnect] server: connection
disconnected
Here are the client logs:
2008-01-29 05:22:16 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:22:16 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x532d20
2008-01-29 05:22:16 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:22:36 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:22:36 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x5398d0
2008-01-29 05:22:36 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:22:56 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:22:56 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x532a20
2008-01-29 05:22:56 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:23:16 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:23:16 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x531c40
2008-01-29 05:23:16 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:24:14 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:24:14 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x53a7d0
2008-01-29 05:24:14 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:25:19 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:25:19 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x532be0
2008-01-29 05:25:19 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:25:37 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:25:37 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x5338a0
2008-01-29 05:25:37 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x5338a0
2008-01-29 05:25:37 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:26:10 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:26:10 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x533a80
2008-01-29 05:26:10 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:27:20 C [client-protocol.c:217:call_bail] client4: bailing
transport
2008-01-29 05:27:20 E [client-protocol.c:4555:client_protocol_cleanup]
client4: forced unwinding frame type(0) op(34) reply=@0x53f0e0
2008-01-29 05:27:20 C [tcp.c:81:tcp_disconnect] client4: connection
disconnected
2008-01-29 05:27:59 E [tcp-client.c:171:tcp_connect] client4: non-blocking
connect() returned: 113 (No route to host)
2008-01-29 10:16:24 E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse:
6336: /weatherflow_wp/cache/wp-cache-8015965cebd529c3433e331c330e193b.meta=>
-1 (2)
2008-01-29 10:16:24 E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse:
6336: /weatherflow_wp/cache/wp-cache-8015965cebd529c3433e331c330e193b.meta=>
-1 (2)
2008-01-29 10:16:41 E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse:
6442: /weatherflow_wp/cache/wp-cache-bb98ceeeb4ca784b406a1aedd2df7bc0.meta=>
-1 (2)
We are running fuse 2.6.5-2.fc5 and glusterfs--mainline--2.5 patch-643
The problem starts around 5:00am when our cleanup scripts run. It was
failing everyday when updatedb ran across the cluster on every server, but
since I setup updatedb to ignore gluster mounts the system is able to stay
up much longer now.
On Jan 29, 2008 8:19 AM, Basavanagowda Kanur <gowda at zresearch.com> wrote:
> Rohan,
> Can you please send us the server logs and also the volume spec files of
> both server and client? That will help us find out the exact reason for
> this
> problem.
>
> --
> Gowda
>
>
More information about the Gluster-devel
mailing list