[Gluster-devel] svn co / cp fail
Pooya Woodcock
pooya at packetcloud.net
Thu Apr 26 19:35:04 UTC 2007
I logged in after a few hours and noticed my gluster.log file was
full again with the same SETVOLUME error... So I must be doing
something wrong still. I don't notice any problems with the mount,
but these errors pop up every 30 seconds in the log file.
[Apr 26 12:28:50] [DEBUG/tcp-client.c:301/tcp_connect()] tcp/
client:connection on 4 success, attempting to handshake
[Apr 26 12:28:50] [DEBUG/tcp-client.c:58/do_handshake()] transport/
tcp-client:dictionary length = 50
[Apr 26 12:28:50] [ERROR/tcp-client.c:123/do_handshake()] tcp/
client:SETVOLUME on remote server failed (Volume "locks" is not
attachable from host 10.175.175.134)
[Apr 26 12:28:50] [ERROR/tcp-client.c:310/tcp_connect()] tcp/
client:handshake: failed
[Apr 26 12:28:50] [ERROR/client-protocol.c:204/client_protocol_xfer
()] protocol/client:transport_submit failed
[Apr 26 12:28:50] [DEBUG/client-protocol.c:2581/
client_protocol_cleanup()] protocol/client:cleaning up state in
transport object 0x550300
[Apr 26 12:28:50] [DEBUG/tcp-client.c:174/tcp_connect()] transport:
tcp: :try_connect: socket fd = 4
[Apr 26 12:28:50] [DEBUG/tcp-client.c:196/tcp_connect()] transport:
tcp: :try_connect: finalized on port `1022'
[Apr 26 12:28:50] [DEBUG/tcp-client.c:255/tcp_connect()] tcp/
client:connect on 4 in progress (non-blocking)
[Apr 26 12:28:50] [DEBUG/tcp-client.c:293/tcp_connect()] tcp/
client:connection on 4 still in progress - try later
[Apr 26 12:28:50] [ERROR/client-protocol.c:204/client_protocol_xfer
()] protocol/client:transport_submit failed
[Apr 26 12:28:50] [DEBUG/client-protocol.c:2581/
client_protocol_cleanup()] protocol/client:cleaning up state in
transport object 0x550300
Here are the conf files:
# server.vol#
volume brick
type storage/posix
option directory /GlusterFS
end-volume
volume locks
type features/posix-locks
subvolumes brick
end-volume
volume brick-afr
type storage/posix
option directory /GlusterFS-AFR
end-volume
volume locks-afr
type features/posix-locks
subvolumes brick-afr
end-volume
volume server
type protocol/server
option transport-type tcp/server
option bind-address 10.175.175.134
option listen-port 6996
subvolumes locks locks-afr
option auth.ip.locks.allow 10.175.175.*
option auth.ip.locks-afr.allow 10.175.175.*
end-volume
#
### CLIENT client.vol ####
volume brick1
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.134
option remote-port 6996
option remote-subvolume locks
end-volume
volume brick1-afr
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.135
option remote-port 6996
option remote-subvolume locks-afr
end-volume
volume brick2
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.135
option remote-port 6996
option remote-subvolume locks
end-volume
volume brick2-afr
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.136
option remote-port 6996
option remote-subvolume locks-afr
end-volume
volume brick3
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.136
option remote-port 6996
option remote-subvolume locks
end-volume
volume brick3-afr
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.137
option remote-port 6996
option remote-subvolume locks-afr
end-volume
volume brick4
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.137
option remote-port 6996
option remote-subvolume locks
end-volume
volume brick4-afr
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.134
option remote-port 6996
option remote-subvolume locks-afr
end-volume
volume afr1
type cluster/afr
subvolumes brick1 brick1-afr
option replicate *:2
end-volume
volume afr2
type cluster/afr
subvolumes brick2 brick2-afr
option replicate *:2
end-volume
volume afr3
type cluster/afr
subvolumes brick3 brick3-afr
option replicate *:2
end-volume
volume afr4
type cluster/afr
subvolumes brick4 brick4-afr
option replicate *:2
end-volume
volume unify1
type cluster/unify
subvolumes afr1 afr2 afr3 afr4
option readdir-force-success on
option scheduler rr
option rr.limits.min-free-disk 10GB
end-volume
On Apr 26, 2007, at 11:24 AM, Pooya Woodcock wrote:
> Ooops! Ok it works now!
> -Pooya
>
> On Apr 26, 2007, at 10:58 AM, Anand Avati wrote:
>
>> Pooya,
>> please update the appropriate 'option auth.ip.<brick>.allow'
>> lines in the protocol/server translator to allow the new volume names
>> (locks*) from the preivous names (brick*)
>>
>> thanks,
>> avati
>>
>>
>> On Thu, Apr 26, 2007 at 10:27:46AM -0700, Pooya Woodcock wrote:
>>> Avati,
>>> Thanks, I confirmed that the bug is fixed!
>>>
>>> I loaded the posix-locks translater as follows on the server side.
>>> Did I do this right? I couldn't get the volume to mount when I put
>>> "subvolumes locks locks-afr" in the protocol/server translator (and
>>> then also changing the client side to look for remote-subvolume
>>> locks
>>> and remote-subvolume locks-afr). When I had the remote subvolumes
>>> set
>>> to locks and locks-afr, I got this error:
>>> Apr 26 10:05:13] [ERROR/tcp-client.c:123/do_handshake()] tcp/
>>> client:SETVOLUME on remo
>>> te server failed (Volume "locks-afr" is not attachable from host
>>> 10.175.175.134)
>>>
>>> ## current server.vol ###
>>> volume brick
>>> type storage/posix
>>> option directory /GlusterFS
>>> end-volume
>>>
>>> volume locks
>>> type features/posix-locks
>>> subvolumes brick
>>> end-volume
>>>
>>> volume brick-afr
>>> type storage/posix
>>> option directory /GlusterFS-AFR
>>> end-volume
>>>
>>> volume locks-afr
>>> type features/posix-locks
>>> subvolumes brick-afr
>>> end-volume
>>>
>>> volume server
>>> type protocol/server
>>> option transport-type tcp/server
>>> option bind-address 10.175.175.137
>>> option listen-port 6996
>>> subvolumes brick brick-afr
>>> option auth.ip.brick.allow 10.175.175.*
>>> option auth.ip.brick-afr.allow 10.175.175.*
>>> end-volume
>>>
>>>
>>>
>>> On Apr 26, 2007, at 9:43 AM, Anand Avati wrote:
>>>
>>>> Pooya,
>>>> the bug was in posix-locks. posis-locks is supposed to be
>>>> loaded on
>>>> server side just above the storage/posix trnaslator. loading it
>>>> on the
>>>> client side makes it lock only for processes within the same
>>>> machine.
>>>>
>>>> the bug has been fixed in posix locks, but nonetheless you need to
>>>> load posix-locks on server side for it to be put to use at all.
>>>> please
>>>> tla update to get the fix (or just loading posix-locks on
>>>> serverside
>>>> alone will fix it too)
>>>>
>>>> avati
>>>>
>>>> On Thu, Apr 26, 2007 at 08:31:27AM -0700, Pooya Woodcock wrote:
>>>>>
>>>>> On Apr 26, 2007, at 2:51 AM, Anand Avati wrote:
>>>>>> Pooya,
>>>>>> the svn was failing because of the same root cause as brent was
>>>>>> facing for mtime failures. i have committed some changes to
>>>>>> write-behind to use a more 'play safe' flushing policy. I am
>>>>>> able to
>>>>>> successfully svn co now.
>>>>>>
>>>>>> I am still unable to reprodcue the cp issue. please give me your
>>>>>> spec files and the commands you executed before you got the bug.
>>>>>>
>>>>>> regards,
>>>>>> avati
>>>>>
>>>>>
>>>>> Hi Avati,
>>>>> After upgrading to patch-123, all I did was start copying files
>>>>> around and then noticed that I can't overwrite any file. My shell
>>>>> thinks that the file is a directory when it is not. I just did
>>>>> another test:
>>>>>
>>>>> [root at node01 www]# touch hi
>>>>> [root at node01 www]# touch hi2
>>>>> [root at node01 www]# cp hi2 hi
>>>>> cp: overwrite `hi'? yes
>>>>> cp: cannot create regular file `hi': No such file or directory
>>>>>
>>>>> [root at node01 www]# mv hi2 hi
>>>>> mv: overwrite `hi'? yes
>>>>>
>>>>> [root at node01 www]# cp hi hi2
>>>>> [root at node01 www]# ls -l hi*
>>>>> -rw-r--r-- 1 root root 0 Apr 26 08:22 hi
>>>>> -rw-r--r-- 1 root root 0 Apr 26 08:23 hi2
>>>>>
>>>>> [root at node01 www]# rm hi
>>>>> rm: remove regular empty file `hi'? yes
>>>>> [root at node01 www]# rm hi2
>>>>> rm: remove regular empty file `hi2'? yes
>>>>> [root at node01 www]#
>>>>>
>>>>>
>>>>> So you see, it is just the cp/overwrite which fails. This
>>>>> happens on
>>>>> nodes 2-4 also.
>>>>> ***After reverting to patch 119, all the above commands work
>>>>> fine***
>>>>>
>>>>> Here are the conf files for glusterfs/glusterfsd I am using....
>>>>>
>>>>> There are 4 servers and 4 clients.
>>>>> ## SERVER.VOL (same on each except for bind-address) ##
>>>>>
>>>>> volume brick
>>>>> type storage/posix
>>>>> option directory /GlusterFS
>>>>> end-volume
>>>>>
>>>>> volume brick-afr
>>>>> type storage/posix
>>>>> option directory /GlusterFS-AFR
>>>>> end-volume
>>>>>
>>>>> volume server
>>>>> type protocol/server
>>>>> option transport-type tcp/server
>>>>> option bind-address 10.175.175.134
>>>>> option listen-port 6996
>>>>> subvolumes brick brick-afr
>>>>> option auth.ip.brick.allow 10.175.175.*
>>>>> option auth.ip.brick-afr.allow 10.175.175.*
>>>>> end-volume
>>>>>
>>>>>
>>>>> ### CLIENT.VOL ###
>>>>> volume brick1
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.134
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick
>>>>> end-volume
>>>>>
>>>>> volume brick1-afr
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.135
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick-afr
>>>>> end-volume
>>>>>
>>>>> volume brick2
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.135
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick
>>>>> end-volume
>>>>>
>>>>> volume brick2-afr
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.136
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick-afr
>>>>> end-volume
>>>>>
>>>>> volume brick3
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.136
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick
>>>>> end-volume
>>>>>
>>>>> volume brick3-afr
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.137
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick-afr
>>>>> end-volume
>>>>>
>>>>> volume brick4
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.137
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick
>>>>> end-volume
>>>>>
>>>>> volume brick4-afr
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.134
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick-afr
>>>>> end-volume
>>>>>
>>>>> volume afr1
>>>>> type cluster/afr
>>>>> subvolumes brick1 brick1-afr
>>>>> option replicate *:2
>>>>> end-volume
>>>>>
>>>>> volume afr2
>>>>> type cluster/afr
>>>>> subvolumes brick2 brick2-afr
>>>>> option replicate *:2
>>>>> end-volume
>>>>>
>>>>> volume afr3
>>>>> type cluster/afr
>>>>> subvolumes brick3 brick3-afr
>>>>> option replicate *:2
>>>>> end-volume
>>>>>
>>>>> volume afr4
>>>>> type cluster/afr
>>>>> subvolumes brick4 brick4-afr
>>>>> option replicate *:2
>>>>> end-volume
>>>>>
>>>>> volume unify1
>>>>> type cluster/unify
>>>>> subvolumes afr1 afr2 afr3 afr4
>>>>> option readdir-force-success on
>>>>> option scheduler rr
>>>>> option rr.limits.min-free-disk 10GB
>>>>> end-volume
>>>>>
>>>>> volume locks
>>>>> type features/posix-locks
>>>>> subvolumes unify1
>>>>> end-volume
>>>>>
>>>>
>>>> --
>>>> ultimate_answer_t
>>>> deep_thought (void)
>>>> {
>>>> sleep (years2secs (7500000));
>>>> return 42;
>>>> }
>>>
>>>
>>
>> --
>> ultimate_answer_t
>> deep_thought (void)
>> {
>> sleep (years2secs (7500000));
>> return 42;
>> }
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
More information about the Gluster-devel
mailing list