[Gluster-users] add bricks on distributed replicated volume failed

Laurent DOUCHY Laurent.Douchy at unige.ch
Thu Sep 1 16:39:02 UTC 2011 

see below

Cheers,
Laurent DOUCHY.


On 9/1/11 6:01 PM, Mohit Anchlia wrote:
> You can check few things on 5 and 6:
>
> 1) gluster processes are running on node5 and 6
yes:

node05

root      4902     1  0 Aug31 ?        00:00:00 
/opt/glusterfs/3.2.2/sbin/glusterd
root      9626     1  0 19:55 ?        00:00:00 
/opt/glusterfs/3.2.2/sbin/glusterfs -f /etc/glusterd/nfs/nfs-server.vol 
-p /etc/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log
root      9690  9686  0 20:04 ?        00:00:00 bash -c ps -edf | grep 
gluster
root      9704  9690  0 20:04 ?        00:00:00 grep gluster

node06

root      4441     1  0 Aug31 ?        00:00:00 
/opt/glusterfs/3.2.2/sbin/glusterd
root      9178     1  0 19:55 ?        00:00:00 
/opt/glusterfs/3.2.2/sbin/glusterfs -f /etc/glusterd/nfs/nfs-server.vol 
-p /etc/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log
root      9242  9238  0 20:04 ?        00:00:00 bash -c ps -edf | grep 
gluster
root      9256  9242  0 20:04 ?        00:00:00 grep gluster

> 2) both nodes are able to see each other
yes:

[root at node05 ~]# ping node06
PING node06.isdc.unige.ch (129.194.168.70) 56(84) bytes of data.
64 bytes from node06.isdc.unige.ch (129.194.168.70): icmp_seq=1 ttl=64 
time=0.376 ms

[root at node06 ~]# ping node05
PING node05.isdc.unige.ch (129.194.168.69) 56(84) bytes of data.
64 bytes from node05.isdc.unige.ch (129.194.168.69): icmp_seq=1 ttl=64 
time=0.337 ms
> 3) do gluster peer status on both the nodes and see what you see
node 5 trust node 6 and node 6 trust node 5

[root at node05 ~]# gluster peer status
Number of Peers: 5

Hostname: node08
Uuid: dd114546-5b94-4a62-9301-260703bf5707
State: Peer in Cluster (Connected)

Hostname: node06
Uuid: 3142fb9a-0a6b-46ec-9262-ede95e8f798a
State: Peer in Cluster (Connected)

Hostname: node10
Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0
State: Peer in Cluster (Connected)

Hostname: 129.194.168.71
Uuid: a35fb0a1-af35-4a04-b38a-434f68369508
State: Peer in Cluster (Connected)

Hostname: node09
Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04
State: Peer in Cluster (Connected)




[root at node06 ~]# gluster peer status
Number of Peers: 5

Hostname: node08
Uuid: dd114546-5b94-4a62-9301-260703bf5707
State: Peer in Cluster (Connected)

Hostname: node09
Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04
State: Peer in Cluster (Connected)

Hostname: node05
Uuid: 13ffcf87-6e8d-4c6b-814a-cbc14d15d88b
State: Peer in Cluster (Connected)

Hostname: node10
Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0
State: Peer in Cluster (Connected)

Hostname: 129.194.168.71
Uuid: a35fb0a1-af35-4a04-b38a-434f68369508
State: Peer in Cluster (Connected)


> 4) check iptables
same file on each node (the installation is manage by puppet)
>
> On Thu, Sep 1, 2011 at 8:57 AM, Laurent DOUCHY<Laurent.Douchy at unige.ch>  wrote:
>> It works ...
>>
>> [root at node07 ~]# gluster volume add-brick cluster node09:/gluster3
>> node10:/gluster3
>> Add Brick successful
>>
>>
>> On 9/1/11 5:39 PM, Mohit Anchlia wrote:
>>> Can you try with node09:/gluster3 and node10:gluster3 instead?
>>>
>>> On Thu, Sep 1, 2011 at 2:49 AM, Laurent DOUCHY<Laurent.Douchy at unige.ch>
>>>   wrote:
>>>> Hi,
>>>>
>>>> I work on the node7 so it doesn't appear on the list.
>>>>
>>>> I create a folder /gluster3 on node5 and node6 and try to add them to my
>>>> volume but it failed with the same message :(
>>>>
>>>> [root at node07 ~]# gluster volume add-brick cluster node05:/gluster3
>>>> node06:/gluster3
>>>> Operation failed on node05
>>>>
>>>> next step is to reinstall from scratch the node I hope I can avoid this.
>>>>
>>>> On 8/31/11 9:08 PM, Mohit Anchlia wrote:
>>>>> I don't see node07 in above output of gluster peer status.
>>>>>
>>>>> Can you try to add bricks on the hosts that gluster1, gluster2? So add
>>>>> gluster3 and see if that works.
>>>>>
>>>>> On Wed, Aug 31, 2011 at 11:56 AM, Laurent DOUCHY
>>>>> <Laurent.Douchy at unige.ch>      wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I try to add 2 bricks or 4 bricks for the same effect.
>>>>>>
>>>>>> I try to reinstall gluster without success.
>>>>>>
>>>>>> Cheers,
>>>>>> Laurent DOUCHY.
>>>>>>
>>>>>>
>>>>>> On 8/31/11 8:07 PM, Burnash, James wrote:
>>>>>>> Hi Laurent.
>>>>>>>
>>>>>>> Since your configuration specifies replication, you must add bricks in
>>>>>>> the
>>>>>>> same number as your number of replicas.
>>>>>>>
>>>>>>> For instance - if you have 2 replicas (most normal case), you would
>>>>>>> need
>>>>>>> to do something like this:
>>>>>>>
>>>>>>> gluster volume add-brick cluster node05:/gluster1 node06:/gluster1
>>>>>>>
>>>>>>> James Burnash
>>>>>>> Unix Engineer
>>>>>>> Knight Capital Group
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: gluster-users-bounces at gluster.org
>>>>>>> [mailto:gluster-users-bounces at gluster.org] On Behalf Of Laurent DOUCHY
>>>>>>> Sent: Wednesday, August 31, 2011 12:49 PM
>>>>>>> To: gluster-users at gluster.org
>>>>>>> Subject: [Gluster-users] add bricks on distributed replicated volume
>>>>>>> failed
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm using gluster 3.2.2 on 10 nodes. Each node have 2x2 TB disk for
>>>>>>> gluster.
>>>>>>>
>>>>>>> I manage to configure a distributed and replicated volume on 4 nodes :
>>>>>>>
>>>>>>> [root at node07 ~]# gluster volume info cluster
>>>>>>>
>>>>>>> Volume Name: cluster
>>>>>>> Type: Distributed-Replicate
>>>>>>> Status: Started
>>>>>>> Number of Bricks: 4 x 2 = 8
>>>>>>> Transport-type: tcp
>>>>>>> Bricks:
>>>>>>> Brick1: node09:/gluster1
>>>>>>> Brick2: node10:/gluster1
>>>>>>> Brick3: node09:/gluster2
>>>>>>> Brick4: node10:/gluster2
>>>>>>> Brick5: node07:/gluster1
>>>>>>> Brick6: node08:/gluster1
>>>>>>> Brick7: node07:/gluster2
>>>>>>> Brick8: node08:/gluster2
>>>>>>>
>>>>>>> But I can't add new nodes to this volume
>>>>>>>
>>>>>>> [root at node07 ~]# gluster peer status
>>>>>>> Number of Peers: 5
>>>>>>>
>>>>>>> Hostname: node10
>>>>>>> Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0
>>>>>>> State: Peer in Cluster (Connected)
>>>>>>>
>>>>>>> Hostname: node08
>>>>>>> Uuid: dd114546-5b94-4a62-9301-260703bf5707
>>>>>>> State: Peer in Cluster (Connected)
>>>>>>>
>>>>>>> Hostname: node09
>>>>>>> Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04
>>>>>>> State: Peer in Cluster (Connected)
>>>>>>>
>>>>>>> Hostname: node06
>>>>>>> Uuid: 3142fb9a-0a6b-46ec-9262-ede95e8f798a
>>>>>>> State: Peer in Cluster (Connected)
>>>>>>>
>>>>>>> Hostname: node05
>>>>>>> Uuid: 13ffcf87-6e8d-4c6b-814a-cbc14d15d88b
>>>>>>> State: Peer in Cluster (Connected)
>>>>>>> [root at node07 ~]# gluster volume add-brick cluster node05:/gluster1
>>>>>>> node06:/gluster1 node05:/gluster2 node06:/gluster2 Operation failed on
>>>>>>> node05
>>>>>>>
>>>>>>> I try to detach nodes 5 and 6, restart glusterd do the probe and the
>>>>>>> add-brick but still nothing ...
>>>>>>>
>>>>>>> Did some one have any idea to fix this ?
>>>>>>>
>>>>>>> Thanks in advance,
>>>>>>> Laurent.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Gluster-users mailing list
>>>>>>> Gluster-users at gluster.org
>>>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>>>>
>>>>>>>
>>>>>>> DISCLAIMER:
>>>>>>> This e-mail, and any attachments thereto, is intended only for use by
>>>>>>> the
>>>>>>> addressee(s) named herein and may contain legally privileged and/or
>>>>>>> confidential information. If you are not the intended recipient of
>>>>>>> this
>>>>>>> e-mail, you are hereby notified that any dissemination, distribution
>>>>>>> or
>>>>>>> copying of this e-mail, and any attachments thereto, is strictly
>>>>>>> prohibited.
>>>>>>> If you have received this in error, please immediately notify me and
>>>>>>> permanently delete the original and any copy of any e-mail and any
>>>>>>> printout
>>>>>>> thereof. E-mail transmission cannot be guaranteed to be secure or
>>>>>>> error-free. The sender therefore does not accept liability for any
>>>>>>> errors or
>>>>>>> omissions in the contents of this message which arise as a result of
>>>>>>> e-mail
>>>>>>> transmission.
>>>>>>> NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may,
>>>>>>> at
>>>>>>> its discretion, monitor and review the content of all e-mail
>>>>>>> communications.
>>>>>>> http://www.knight.com
>>>>>>> _______________________________________________
>>>>>>> Gluster-users mailing list
>>>>>>> Gluster-users at gluster.org
>>>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>>> _______________________________________________
>>>>>> Gluster-users mailing list
>>>>>> Gluster-users at gluster.org
>>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>>>

More information about the Gluster-users mailing list