[Gluster-devel] glusterfs version3 and xen images

Mon Dec 21 09:34:45 UTC 2009

hi,

changed client volfile:

cat /etc/glusterfs/mount-office-data02-server_repl.vol
#volume office-data02-rr
volume office-data02
   type protocol/client
   option transport-type tcp
#  option remote-host cluster-gfs.magora.local # RRDNS
   option remote-host gfs-01-01
   option remote-host gfs-01-02
   option remote-port 7000
   option remote-subvolume office-data02
end-volume

volume readahead
   type performance/read-ahead
   option page-count 16             # cache per file  = (page-count x 
page-size)
   option force-atime-update off # default is off
   subvolumes office-data02
end-volume

volume writebehind
   type performance/write-behind
   option cache-size 512MB # default is equal to aggregate-size
   option flush-behind on # default is 'off'
   subvolumes readahead
end-volume

volume iocache
   type performance/io-cache
   option cache-size 64MB             # default is 32MB
#  option priority *.h:3,*.html:2,*:1 # default is '*:0'
   option cache-timeout 3             # default is 1 second
   subvolumes writebehind
end-volume

volume quickread
  type performance/quick-read
   option cache-timeout 1
   option max-file-size 512
   subvolumes iocache
end-volume

volume stat-prefetch
   type performance/stat-prefetch
   subvolumes quickread
end-volume

But client connects always to gfs-01-02 - never to gfs-01-01 - whats wrong?

server debug gfs-01-01:
[2009-12-21 10:30:52] D [glusterfsd.c:1335:main] glusterfs: running in 
pid 4978
[New Thread 0x7ffff5b45950 (LWP 4981)]
[2009-12-21 10:30:52] D [transport.c:145:transport_load] transport: 
attempt to load file /usr/lib64/glusterfs/3.0.0/transport/socket.so
[2009-12-21 10:30:52] D [xlator.c:284:_volume_option_value_validate] 
server: no range check required for 'option transport.socket.listen-port 
7000'
[2009-12-21 10:30:52] W [quick-read.c:2187:init] quickread: dangling 
volume. check volfile
[2009-12-21 10:30:52] D [io-threads.c:2841:init] office-data02: 
io-threads: Autoscaling: off, min_threads: 32, max_threads: 32
[New Thread 0x7ffff7f85950 (LWP 4982)]
[New Thread 0x7ffff4f36950 (LWP 4983)]
[New Thread 0x7ffff4e35950 (LWP 4984)]
[New Thread 0x7ffff4d34950 (LWP 4985)]
[New Thread 0x7ffff4c33950 (LWP 4986)]
[New Thread 0x7ffff4b32950 (LWP 4987)]
[New Thread 0x7ffff4a31950 (LWP 4988)]
[New Thread 0x7ffff4930950 (LWP 4989)]
[New Thread 0x7ffff482f950 (LWP 4990)]
[New Thread 0x7ffff472e950 (LWP 4991)]
[New Thread 0x7ffff462d950 (LWP 4992)]
[New Thread 0x7ffff452c950 (LWP 4993)]
[New Thread 0x7ffff442b950 (LWP 4994)]
[New Thread 0x7ffff432a950 (LWP 4995)]
[New Thread 0x7ffff4229950 (LWP 4996)]
[New Thread 0x7ffff4128950 (LWP 4997)]
[New Thread 0x7ffff4027950 (LWP 4998)]
[New Thread 0x7ffff3f26950 (LWP 4999)]
[New Thread 0x7ffff3e25950 (LWP 5000)]
[New Thread 0x7ffff3d24950 (LWP 5001)]
[New Thread 0x7ffff3c23950 (LWP 5002)]
[New Thread 0x7ffff3b22950 (LWP 5003)]
[New Thread 0x7ffff3a21950 (LWP 5004)]
[New Thread 0x7ffff3920950 (LWP 5005)]
[New Thread 0x7ffff381f950 (LWP 5006)]
[New Thread 0x7ffff371e950 (LWP 5007)]
[New Thread 0x7ffff361d950 (LWP 5008)]
[New Thread 0x7ffff351c950 (LWP 5009)]
[New Thread 0x7ffff341b950 (LWP 5010)]
[New Thread 0x7ffff331a950 (LWP 5011)]
[New Thread 0x7ffff3219950 (LWP 5012)]
[New Thread 0x7ffff3118950 (LWP 5013)]
[2009-12-21 10:30:52] D [write-behind.c:2480:init] writebehind: 
disabling write-behind for first 1 bytes
[2009-12-21 10:30:52] D [write-behind.c:2531:init] writebehind: enabling 
flush-behind
[2009-12-21 10:30:52] D [client-protocol.c:6581:init] posix-remote: 
defaulting frame-timeout to 30mins
[2009-12-21 10:30:52] D [client-protocol.c:6589:init] posix-remote: 
setting ping-timeout to 5
[2009-12-21 10:30:52] D [transport.c:145:transport_load] transport: 
attempt to load file /usr/lib64/glusterfs/3.0.0/transport/socket.so
[2009-12-21 10:30:52] D [xlator.c:284:_volume_option_value_validate] 
posix-remote: no range check required for 'option remote-port 7000'
[2009-12-21 10:30:52] D [transport.c:145:transport_load] transport: 
attempt to load file /usr/lib64/glusterfs/3.0.0/transport/socket.so
[2009-12-21 10:30:52] D [xlator.c:284:_volume_option_value_validate] 
posix-remote: no range check required for 'option remote-port 7000'
[2009-12-21 10:30:52] D [client-protocol.c:7005:notify] posix-remote: 
got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-12-21 10:30:52] D [client-protocol.c:7005:notify] posix-remote: 
got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-12-21 10:30:52] D [client-protocol.c:7005:notify] posix-remote: 
got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-12-21 10:30:52] D [client-protocol.c:7005:notify] posix-remote: 
got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-12-21 10:30:52] N [glusterfsd.c:1361:main] glusterfs: Successfully 
started
[2009-12-21 10:30:52] D [client-protocol.c:7019:notify] posix-remote: 
got GF_EVENT_CHILD_UP
[2009-12-21 10:30:52] D [client-protocol.c:7019:notify] posix-remote: 
got GF_EVENT_CHILD_UP
[2009-12-21 10:30:52] N [client-protocol.c:6224:client_setvolume_cbk] 
posix-remote: Connected to 192.168.11.82:7000, attached to remote volume 
'locks'.
[2009-12-21 10:30:52] N [afr.c:2625:notify] gfs-replicate: Subvolume 
'posix-remote' came back up; going online.
[2009-12-21 10:30:52] N [client-protocol.c:6224:client_setvolume_cbk] 
posix-remote: Connected to 192.168.11.82:7000, attached to remote volume 
'locks'.
[2009-12-21 10:30:52] N [afr.c:2625:notify] gfs-replicate: Subvolume 
'posix-remote' came back up; going online.
[2009-12-21 10:31:02] D [addr.c:190:gf_auth] locks: allowed = 
"192.168.11.*", received addr = "192.168.11.82"
[2009-12-21 10:31:02] N [server-protocol.c:5809:mop_setvolume] server: 
accepted client from 192.168.11.82:1023
[2009-12-21 10:31:02] D [addr.c:190:gf_auth] locks: allowed = 
"192.168.11.*", received addr = "192.168.11.82"
[2009-12-21 10:31:02] N [server-protocol.c:5809:mop_setvolume] server: 
accepted client from 192.168.11.82:1022

serverdebug gfs-01-02:
[2009-12-21 10:30:51] D [write-behind.c:2480:init] writebehind: 
disabling write-behind for first 1 bytes
[2009-12-21 10:30:51] D [write-behind.c:2531:init] writebehind: enabling 
flush-behind
[2009-12-21 10:30:51] D [client-protocol.c:6581:init] posix-remote: 
defaulting frame-timeout to 30mins
[2009-12-21 10:30:51] D [client-protocol.c:6589:init] posix-remote: 
setting ping-timeout to 5
[2009-12-21 10:30:51] D [transport.c:145:transport_load] transport: 
attempt to load file /usr/lib64/glusterfs/3.0.0/transport/socket.so
[2009-12-21 10:30:51] D [xlator.c:284:_volume_option_value_validate] 
posix-remote: no range check required for 'option remote-port 7000'
[2009-12-21 10:30:51] D [transport.c:145:transport_load] transport: 
attempt to load file /usr/lib64/glusterfs/3.0.0/transport/socket.so
[2009-12-21 10:30:51] D [xlator.c:284:_volume_option_value_validate] 
posix-remote: no range check required for 'option remote-port 7000'
[2009-12-21 10:30:51] D [client-protocol.c:7005:notify] posix-remote: 
got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-12-21 10:30:51] D [client-protocol.c:7005:notify] posix-remote: 
got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-12-21 10:30:51] D [client-protocol.c:7005:notify] posix-remote: 
got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-12-21 10:30:51] D [client-protocol.c:7005:notify] posix-remote: 
got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-12-21 10:30:51] N [glusterfsd.c:1361:main] glusterfs: Successfully 
started
[2009-12-21 10:30:51] E [socket.c:760:socket_connect_finish] 
posix-remote: connection to  failed (Connection refused)
[2009-12-21 10:30:51] E [socket.c:760:socket_connect_finish] 
posix-remote: connection to  failed (Connection refused)
[2009-12-21 10:30:52] D [addr.c:190:gf_auth] locks: allowed = 
"192.168.11.*", received addr = "192.168.11.11"
[2009-12-21 10:30:52] N [server-protocol.c:5809:mop_setvolume] server: 
accepted client from 192.168.11.11:1023
[2009-12-21 10:30:52] D [addr.c:190:gf_auth] locks: allowed = 
"192.168.11.*", received addr = "192.168.11.11"
[2009-12-21 10:30:52] N [server-protocol.c:5809:mop_setvolume] server: 
accepted client from 192.168.11.11:1022
[2009-12-21 10:31:02] D [client-protocol.c:7019:notify] posix-remote: 
got GF_EVENT_CHILD_UP
[2009-12-21 10:31:02] D [client-protocol.c:7019:notify] posix-remote: 
got GF_EVENT_CHILD_UP
[2009-12-21 10:31:02] N [client-protocol.c:6224:client_setvolume_cbk] 
posix-remote: Connected to 192.168.11.11:7000, attached to remote volume 
'locks'.
[2009-12-21 10:31:02] N [afr.c:2625:notify] gfs-replicate: Subvolume 
'posix-remote' came back up; going online.
[2009-12-21 10:31:02] N [client-protocol.c:6224:client_setvolume_cbk] 
posix-remote: Connected to 192.168.11.11:7000, attached to remote volume 
'locks'.
[2009-12-21 10:31:02] N [afr.c:2625:notify] gfs-replicate: Subvolume 
'posix-remote' came back up; going online.
[2009-12-21 10:31:06] D [addr.c:190:gf_auth] office-data02: allowed = 
"192.168.11.*", received addr = "192.168.11.68"
[2009-12-21 10:31:06] N [server-protocol.c:5809:mop_setvolume] server: 
accepted client from 192.168.11.68:1023
[2009-12-21 10:31:06] D [addr.c:190:gf_auth] office-data02: allowed = 
"192.168.11.*", received addr = "192.168.11.68"
[2009-12-21 10:31:06] N [server-protocol.c:5809:mop_setvolume] server: 
accepted client from 192.168.11.68:1022

regards,
roland