[Gluster-devel] svn co / cp fail

Pooya Woodcock pooya at packetcloud.net
Thu Apr 26 19:35:04 UTC 2007


I logged in after a few hours and noticed my gluster.log file was  
full again with the same SETVOLUME error... So I must be doing  
something wrong still. I don't notice any problems with the mount,  
but these errors pop up every 30 seconds in the log file.

[Apr 26 12:28:50] [DEBUG/tcp-client.c:301/tcp_connect()] tcp/ 
client:connection on 4 success, attempting to handshake
[Apr 26 12:28:50] [DEBUG/tcp-client.c:58/do_handshake()] transport/ 
tcp-client:dictionary length = 50
[Apr 26 12:28:50] [ERROR/tcp-client.c:123/do_handshake()] tcp/ 
client:SETVOLUME on remote server failed (Volume "locks" is not  
attachable from host 10.175.175.134)
[Apr 26 12:28:50] [ERROR/tcp-client.c:310/tcp_connect()] tcp/ 
client:handshake: failed
[Apr 26 12:28:50] [ERROR/client-protocol.c:204/client_protocol_xfer 
()] protocol/client:transport_submit failed
[Apr 26 12:28:50] [DEBUG/client-protocol.c:2581/ 
client_protocol_cleanup()] protocol/client:cleaning up state in  
transport object 0x550300
[Apr 26 12:28:50] [DEBUG/tcp-client.c:174/tcp_connect()] transport:  
tcp: :try_connect: socket fd = 4
[Apr 26 12:28:50] [DEBUG/tcp-client.c:196/tcp_connect()] transport:  
tcp: :try_connect: finalized on port `1022'
[Apr 26 12:28:50] [DEBUG/tcp-client.c:255/tcp_connect()] tcp/ 
client:connect on 4 in progress (non-blocking)
[Apr 26 12:28:50] [DEBUG/tcp-client.c:293/tcp_connect()] tcp/ 
client:connection on 4 still in progress - try later
[Apr 26 12:28:50] [ERROR/client-protocol.c:204/client_protocol_xfer 
()] protocol/client:transport_submit failed
[Apr 26 12:28:50] [DEBUG/client-protocol.c:2581/ 
client_protocol_cleanup()] protocol/client:cleaning up state in  
transport object 0x550300



Here are the conf files:

# server.vol#
volume brick
         type storage/posix
         option directory /GlusterFS
end-volume

volume locks
         type features/posix-locks
         subvolumes brick
end-volume


volume brick-afr
         type storage/posix
         option directory /GlusterFS-AFR
end-volume

volume locks-afr
   type features/posix-locks
   subvolumes brick-afr
end-volume


volume server
         type protocol/server
         option transport-type tcp/server
         option bind-address 10.175.175.134
         option listen-port 6996
         subvolumes locks locks-afr
         option auth.ip.locks.allow 10.175.175.*
         option auth.ip.locks-afr.allow 10.175.175.*
end-volume


#
### CLIENT client.vol ####
volume brick1
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.134
option remote-port 6996
option remote-subvolume locks
end-volume

volume brick1-afr
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.135
option remote-port 6996
option remote-subvolume locks-afr
end-volume

volume brick2
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.135
option remote-port 6996
option remote-subvolume locks
end-volume

volume brick2-afr
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.136
option remote-port 6996
option remote-subvolume locks-afr
end-volume

volume brick3
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.136
option remote-port 6996
option remote-subvolume locks
end-volume

volume brick3-afr
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.137
option remote-port 6996
option remote-subvolume locks-afr
end-volume

volume brick4
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.137
option remote-port 6996
option remote-subvolume locks
end-volume

volume brick4-afr
type protocol/client
option transport-type tcp/client
option remote-host 10.175.175.134
option remote-port 6996
option remote-subvolume locks-afr
end-volume

volume afr1
type cluster/afr
subvolumes brick1 brick1-afr
option replicate *:2
end-volume

volume afr2
type cluster/afr
subvolumes brick2 brick2-afr
option replicate *:2
end-volume

volume afr3
type cluster/afr
subvolumes brick3 brick3-afr
option replicate *:2
end-volume

volume afr4
type cluster/afr
subvolumes brick4 brick4-afr
option replicate *:2
end-volume

volume unify1
type cluster/unify
subvolumes afr1 afr2 afr3 afr4
   option readdir-force-success on
   option scheduler rr
   option rr.limits.min-free-disk 10GB
end-volume



On Apr 26, 2007, at 11:24 AM, Pooya Woodcock wrote:

> Ooops! Ok it works now!
> -Pooya
>
> On Apr 26, 2007, at 10:58 AM, Anand Avati wrote:
>
>> Pooya,
>>       please update the appropriate 'option auth.ip.<brick>.allow'
>> lines in the protocol/server translator to allow the new volume names
>> (locks*) from the preivous names (brick*)
>>
>> thanks,
>> avati
>>
>>
>> On Thu, Apr 26, 2007 at 10:27:46AM -0700, Pooya Woodcock wrote:
>>> Avati,
>>> Thanks, I confirmed that the bug is fixed!
>>>
>>> I loaded the posix-locks translater as follows on the server side.
>>> Did I do this right? I couldn't get the volume to mount when I put
>>> "subvolumes locks locks-afr" in the protocol/server translator (and
>>> then also changing the client side to look for remote-subvolume  
>>> locks
>>> and remote-subvolume locks-afr). When I had the remote subvolumes  
>>> set
>>> to locks and locks-afr, I got this error:
>>> Apr 26 10:05:13] [ERROR/tcp-client.c:123/do_handshake()] tcp/
>>> client:SETVOLUME on remo
>>> te server failed (Volume "locks-afr" is not attachable from host
>>> 10.175.175.134)
>>>
>>> ## current server.vol ###
>>> volume brick
>>>         type storage/posix
>>>         option directory /GlusterFS
>>> end-volume
>>>
>>> volume locks
>>>   type features/posix-locks
>>>   subvolumes brick
>>> end-volume
>>>
>>> volume brick-afr
>>>         type storage/posix
>>>         option directory /GlusterFS-AFR
>>> end-volume
>>>
>>> volume locks-afr
>>>   type features/posix-locks
>>>   subvolumes brick-afr
>>> end-volume
>>>
>>> volume server
>>>         type protocol/server
>>>         option transport-type tcp/server
>>>         option bind-address 10.175.175.137
>>>         option listen-port 6996
>>>         subvolumes brick brick-afr
>>>         option auth.ip.brick.allow 10.175.175.*
>>>         option auth.ip.brick-afr.allow 10.175.175.*
>>> end-volume
>>>
>>>
>>>
>>> On Apr 26, 2007, at 9:43 AM, Anand Avati wrote:
>>>
>>>> Pooya,
>>>>  the bug was in posix-locks. posis-locks is supposed to be  
>>>> loaded on
>>>> server side just above the storage/posix trnaslator. loading it  
>>>> on the
>>>> client side makes it lock only for processes within the same  
>>>> machine.
>>>>
>>>> the bug has been fixed in posix locks, but nonetheless you need to
>>>> load posix-locks on server side for it to be put to use at all.  
>>>> please
>>>> tla update to get the fix (or just loading posix-locks on  
>>>> serverside
>>>> alone will fix it too)
>>>>
>>>> avati
>>>>
>>>> On Thu, Apr 26, 2007 at 08:31:27AM -0700, Pooya Woodcock wrote:
>>>>>
>>>>> On Apr 26, 2007, at 2:51 AM, Anand Avati wrote:
>>>>>> Pooya,
>>>>>> the svn was failing because of the same root cause as brent was
>>>>>> facing for mtime failures. i have committed some changes to
>>>>>> write-behind to use a more 'play safe' flushing policy. I am  
>>>>>> able to
>>>>>> successfully svn co now.
>>>>>>
>>>>>> I am still unable to reprodcue the cp issue. please give me your
>>>>>> spec files and the commands you executed before you got the bug.
>>>>>>
>>>>>> regards,
>>>>>> avati
>>>>>
>>>>>
>>>>> Hi Avati,
>>>>>   After upgrading to patch-123, all I did was start copying files
>>>>> around and then noticed that I can't overwrite any file. My shell
>>>>> thinks that the file is a directory when it is not. I just did
>>>>> another test:
>>>>>
>>>>> [root at node01 www]# touch hi
>>>>> [root at node01 www]# touch hi2
>>>>> [root at node01 www]# cp hi2 hi
>>>>> cp: overwrite `hi'? yes
>>>>> cp: cannot create regular file `hi': No such file or directory
>>>>>
>>>>> [root at node01 www]# mv hi2 hi
>>>>> mv: overwrite `hi'? yes
>>>>>
>>>>> [root at node01 www]# cp hi hi2
>>>>> [root at node01 www]# ls -l hi*
>>>>> -rw-r--r--  1 root root 0 Apr 26 08:22 hi
>>>>> -rw-r--r--  1 root root 0 Apr 26 08:23 hi2
>>>>>
>>>>> [root at node01 www]# rm hi
>>>>> rm: remove regular empty file `hi'? yes
>>>>> [root at node01 www]# rm hi2
>>>>> rm: remove regular empty file `hi2'? yes
>>>>> [root at node01 www]#
>>>>>
>>>>>
>>>>> So you see, it is just the cp/overwrite which fails. This  
>>>>> happens on
>>>>> nodes 2-4 also.
>>>>> ***After reverting to patch 119, all the above commands work  
>>>>> fine***
>>>>>
>>>>> Here are the conf files for glusterfs/glusterfsd I am using....
>>>>>
>>>>> There are 4 servers and 4 clients.
>>>>> ## SERVER.VOL (same on each except for bind-address) ##
>>>>>
>>>>> volume brick
>>>>>        type storage/posix
>>>>>        option directory /GlusterFS
>>>>> end-volume
>>>>>
>>>>> volume brick-afr
>>>>>        type storage/posix
>>>>>        option directory /GlusterFS-AFR
>>>>> end-volume
>>>>>
>>>>> volume server
>>>>>        type protocol/server
>>>>>        option transport-type tcp/server
>>>>>        option bind-address 10.175.175.134
>>>>>        option listen-port 6996
>>>>>        subvolumes brick brick-afr
>>>>>        option auth.ip.brick.allow 10.175.175.*
>>>>>        option auth.ip.brick-afr.allow 10.175.175.*
>>>>> end-volume
>>>>>
>>>>>
>>>>> ### CLIENT.VOL ###
>>>>> volume brick1
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.134
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick
>>>>> end-volume
>>>>>
>>>>> volume brick1-afr
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.135
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick-afr
>>>>> end-volume
>>>>>
>>>>> volume brick2
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.135
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick
>>>>> end-volume
>>>>>
>>>>> volume brick2-afr
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.136
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick-afr
>>>>> end-volume
>>>>>
>>>>> volume brick3
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.136
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick
>>>>> end-volume
>>>>>
>>>>> volume brick3-afr
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.137
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick-afr
>>>>> end-volume
>>>>>
>>>>> volume brick4
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.137
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick
>>>>> end-volume
>>>>>
>>>>> volume brick4-afr
>>>>> type protocol/client
>>>>> option transport-type tcp/client
>>>>> option remote-host 10.175.175.134
>>>>> option remote-port 6996
>>>>> option remote-subvolume brick-afr
>>>>> end-volume
>>>>>
>>>>> volume afr1
>>>>> type cluster/afr
>>>>> subvolumes brick1 brick1-afr
>>>>> option replicate *:2
>>>>> end-volume
>>>>>
>>>>> volume afr2
>>>>> type cluster/afr
>>>>> subvolumes brick2 brick2-afr
>>>>> option replicate *:2
>>>>> end-volume
>>>>>
>>>>> volume afr3
>>>>> type cluster/afr
>>>>> subvolumes brick3 brick3-afr
>>>>> option replicate *:2
>>>>> end-volume
>>>>>
>>>>> volume afr4
>>>>> type cluster/afr
>>>>> subvolumes brick4 brick4-afr
>>>>> option replicate *:2
>>>>> end-volume
>>>>>
>>>>> volume unify1
>>>>> type cluster/unify
>>>>> subvolumes afr1 afr2 afr3 afr4
>>>>>  option readdir-force-success on
>>>>>  option scheduler rr
>>>>>  option rr.limits.min-free-disk 10GB
>>>>> end-volume
>>>>>
>>>>> volume locks
>>>>>        type features/posix-locks
>>>>>        subvolumes unify1
>>>>> end-volume
>>>>>
>>>>
>>>> -- 
>>>> ultimate_answer_t
>>>> deep_thought (void)
>>>> {
>>>>  sleep (years2secs (7500000));
>>>>  return 42;
>>>> }
>>>
>>>
>>
>> -- 
>> ultimate_answer_t
>> deep_thought (void)
>> {
>>   sleep (years2secs (7500000));
>>   return 42;
>> }
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel






More information about the Gluster-devel mailing list