[Gluster-users] (3.1.6-1) How should I add extra peers to existing file systems ?

Wed Aug 17 06:58:48 UTC 2011

kp,

please find attached tar files.

[root at vhead-010 ~]# ssh foo-1-private
[root at foo-1-private ~]# gluster volume create foo foo-1-private:/mnt/brick
Creation of volume foo has been successful. Please start the volume to access data.
[root at foo-1-private ~]# gluster volume start foo
Starting volume foo has been successful
[root at foo-1-private ~]# gluster peer probe foo-3-private
Probe successful
[root at foo-1-private ~]# gluster peer status
Number of Peers: 1

Hostname: foo-3-private
Uuid: 0c76c6ec-a7ea-405e-9d1d-8e100e155ec3
State: Peer Rejected (Connected)
[root at foo-1-private ~]# cd /etc/glusterd/vols/
[root at foo-1-private vols]# tar cf /tmp/foo-1-private.tar foo
[root at foo-1-private vols]# ssh foo-3-private
[root at foo-3-private ~]# cd /etc/glusterd/vols
[root at foo-3-private vols]# tar cf /tmp/foo-3-private.tar foo
[root at foo-3-private vols]#

Thanks,
Tomo

(2011/08/17 15:46), krish wrote:
> Tomoaki,
>
> Can you send the info file under /etc/glusterd/vols/<volname>/ from machines were the peer probe
> command was issued and the 'peer' who is being rejected?
>
> thanks,
> kp
>
> On 08/17/2011 12:10 PM, Tomoaki Sato wrote:
>> Mohit
>>
>> (2011/08/17 13:53), Mohit Anchlia wrote:
>>> Not sure. It could be because the new node doesn't have the volume
>>> configs. Can you try gluster volume sync to sync the configs and see
>>> if that helps?
>>
>> - at foo-1-private -
>> gluster> volume sync foo-3-private
>> please delete all the volumes before full sync
>> gluster>
>> --
>>
>> - at foo-3-private -
>> gluster> volume sync foo foo-1-private
>> sync from localhost not allowed
>> gluster>
>> --
>>
>> Question is "How should I add extra peers to existing file systems ?".
>> extra peers = new nodes.
>>
>> Could you tell me right instructions to gluster probe new nodes after the volume starting ?
>>
>>>
>>> Also, not sure why you are getting "Unable to find hostname: foo-3-private"
>>>
>>
>> "Unable to find hostname: foo-3-private" was printed out on both OK and NG cases.
>> "Cksums of volume foo differ. local cksum = 1403573944, remote cksum = -1413994823" was printed out on NG case only.
>>
>> 'Peer in Cluster', OK case:
>> [2011-08-17 15:08:47.462646] I [glusterd-handler.c:602:glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req foo-3-private 24007
>> [2011-08-17 15:08:47.466194] I [glusterd-handler.c:391:glusterd_friend_find] 0-glusterd: Unable to find hostname: foo-3-private
>> [2011-08-17 15:08:47.466224] I [glusterd-handler.c:3120:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: foo-3-private (24007)
>> [2011-08-17 15:08:47.469365] W [rpc-transport.c:728:rpc_transport_load] 0-rpc-transport: missing 'option transport-type'. defaulting to "socket"
>> [2011-08-17 15:08:47.473671] I [glusterd-handler.c:3102:glusterd_friend_add] 0-glusterd: connect returned 0
>> [2011-08-17 15:08:47.474167] I [glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using Program glusterd clnt mgmt, Num (1238433), Version (1)
>> [2011-08-17 15:08:47.474214] I [glusterd-utils.c:2127:glusterd_friend_find_by_hostname] 0-glusterd: Friend foo-3-private found.. state: 0
>> [2011-08-17 15:08:47.483485] I [glusterd-rpc-ops.c:364:glusterd3_1_probe_cbk] 0-glusterd: Received probe resp from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f, host: foo-3-private
>> [2011-08-17 15:08:47.483516] I [glusterd-handler.c:379:glusterd_friend_find] 0-glusterd: Unable to find peer by uuid
>> [2011-08-17 15:08:47.483562] I [glusterd-utils.c:2127:glusterd_friend_find_by_hostname] 0-glusterd: Friend foo-3-private found.. state: 0
>> [2011-08-17 15:08:47.483764] I [glusterd-rpc-ops.c:409:glusterd3_1_probe_cbk] 0-glusterd: Received resp to probe req
>> [2011-08-17 15:08:47.484040] I [glusterd-rpc-ops.c:454:glusterd3_1_friend_add_cbk] 0-glusterd: Received ACC from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f, host: foo-3-private, port: 0
>> [2011-08-17 15:08:47.484088] I [glusterd-utils.c:2088:glusterd_friend_find_by_uuid] 0-glusterd: Friend found... state: Probe Sent to Peer
>> [2011-08-17 15:08:47.484153] I [glusterd-handler.c:3293:glusterd_xfer_cli_probe_resp] 0-glusterd: Responded to CLI, ret: 0
>> [2011-08-17 15:08:47.484554] I [glusterd-handler.c:2882:glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f
>> [2011-08-17 15:08:47.484585] I [glusterd-utils.c:2088:glusterd_friend_find_by_uuid] 0-glusterd: Friend found... state: Accepted peer request
>> [2011-08-17 15:08:47.484647] I [glusterd-handler.c:2917:glusterd_handle_probe_query] 0-glusterd: Responded to 192.168.1.129, op_ret: 0, op_errno: 0, ret: 0
>> [2011-08-17 15:08:47.485499] I [glusterd-handler.c:2614:glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f
>> [2011-08-17 15:08:47.485536] I [glusterd-utils.c:2088:glusterd_friend_find_by_uuid] 0-glusterd: Friend found... state: Accepted peer request
>> [2011-08-17 15:08:47.485590] I [glusterd-handler.c:3270:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 192.168.1.129 (0), ret: 0
>> [2011-08-17 15:08:47.485713] I [glusterd-sm.c:492:glusterd_ac_send_friend_update] 0-: Added uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f, host: foo-3-private
>> [2011-08-17 15:08:47.486203] I [glusterd-rpc-ops.c:636:glusterd3_1_friend_update_cbk] 0-glusterd: Received ACC from uuid:
>> [2011-08-17 15:08:47.486259] I [glusterd-utils.c:2088:glusterd_friend_find_by_uuid] 0-glusterd: Friend found... state: Peer in Cluster
>> [2011-08-17 15:08:47.486284] I [glusterd-handler.c:2761:glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f
>> [2011-08-17 15:08:47.486316] I [glusterd-handler.c:2806:glusterd_handle_friend_update] 0-: Received uuid: 4b5b0ecb-7d18-4ec4-90d9-0df2d392b63f, hostname:192.168.1.129
>> [2011-08-17 15:08:47.486335] I [glusterd-handler.c:2809:glusterd_handle_friend_update] 0-: Received my uuid as Friend
>>
>>
>> 'Peer Rejected', NG case:
>> [2011-08-17 15:10:19.21262] I [glusterd-handler.c:602:glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req foo-3-private 24007
>> [2011-08-17 15:10:19.24605] I [glusterd-handler.c:391:glusterd_friend_find] 0-glusterd: Unable to find hostname: foo-3-private
>> [2011-08-17 15:10:19.24648] I [glusterd-handler.c:3120:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: foo-3-private (24007)
>> [2011-08-17 15:10:19.27736] W [rpc-transport.c:728:rpc_transport_load] 0-rpc-transport: missing 'option transport-type'. defaulting to "socket"
>> [2011-08-17 15:10:19.32034] I [glusterd-handler.c:3102:glusterd_friend_add] 0-glusterd: connect returned 0
>> [2011-08-17 15:10:19.32389] I [glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using Program glusterd clnt mgmt, Num (1238433), Version (1)
>> [2011-08-17 15:10:19.32426] I [glusterd-utils.c:2127:glusterd_friend_find_by_hostname] 0-glusterd: Friend foo-3-private found.. state: 0
>> [2011-08-17 15:10:19.40671] I [glusterd-rpc-ops.c:364:glusterd3_1_probe_cbk] 0-glusterd: Received probe resp from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f, host: foo-3-private
>> [2011-08-17 15:10:19.40720] I [glusterd-handler.c:379:glusterd_friend_find] 0-glusterd: Unable to find peer by uuid
>> [2011-08-17 15:10:19.40748] I [glusterd-utils.c:2127:glusterd_friend_find_by_hostname] 0-glusterd: Friend foo-3-private found.. state: 0
>> [2011-08-17 15:10:19.40983] I [glusterd-rpc-ops.c:409:glusterd3_1_probe_cbk] 0-glusterd: Received resp to probe req
>> [2011-08-17 15:10:19.42854] I [rpc-clnt.c:696:rpc_clnt_handle_cbk] 0-rpc-clnt: recieved rpc message (XID: 0x2a, Ver: 2, Program: 52743234, ProgVers: 1, Proc: 1) from rpc-transport (management)
>> [2011-08-17 15:10:19.50762] I [glusterd-rpc-ops.c:454:glusterd3_1_friend_add_cbk] 0-glusterd: Received ACC from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f, host: foo-3-private, port: 0
>> [2011-08-17 15:10:19.50794] I [glusterd-utils.c:2088:glusterd_friend_find_by_uuid] 0-glusterd: Friend found... state: Probe Sent to Peer
>> [2011-08-17 15:10:19.50851] I [glusterd-handler.c:3293:glusterd_xfer_cli_probe_resp] 0-glusterd: Responded to CLI, ret: 0
>> [2011-08-17 15:10:19.51413] I [glusterd-handler.c:2882:glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f
>> [2011-08-17 15:10:19.51444] I [glusterd-utils.c:2088:glusterd_friend_find_by_uuid] 0-glusterd: Friend found... state: Accepted peer request
>> [2011-08-17 15:10:19.51487] I [glusterd-handler.c:2917:glusterd_handle_probe_query] 0-glusterd: Responded to 192.168.1.129, op_ret: 0, op_errno: 0, ret: 0
>> [2011-08-17 15:10:19.51853] I [glusterd-handler.c:2614:glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 464b3ea0-1b2b-4683-8209-72220dcb295f
>> [2011-08-17 15:10:19.51885] I [glusterd-utils.c:2088:glusterd_friend_find_by_uuid] 0-glusterd: Friend found... state: Accepted peer request
>> [2011-08-17 15:10:19.51930] E [glusterd-utils.c:1407:glusterd_compare_friend_volume] 0-: Cksums of volume foo differ. local cksum = 1403573944, remote cksum = -1413994823
>> [2011-08-17 15:10:19.51975] I [glusterd-handler.c:3270:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to 192.168.1.129 (0), ret: 0
>>
>>
>>> On Tue, Aug 16, 2011 at 8:18 PM, Tomoaki Sato<tsato at valinux.co.jp> wrote:
>>>> Mohit,
>>>>
>>>> let me say again.
>>>> 3.1.6-1 fail to 'peer probe' after 'start volume' in my environment.
>>>> case-A) peer probe foo-3-private --> Peer in Cluster
>>>>
>>>> <delete all configuration files and reboot all foo-X-private nodes>
>>>>
>>>> [root at foo-1-private ~]# gluster peer probe foo-3-private
>>>> Probe successful
>>>> [root at foo-1-private ~]# gluster peer status
>>>> Number of Peers: 1
>>>>
>>>> Hostname: foo-3-private
>>>> Uuid: ef7d3c43-219a-4d13-a918-2639455cfbe7
>>>> State: Peer in Cluster (Connected)
>>>>
>>>> case-B) create volume then peer probe foo-3-private --> Peer in Cluster
>>>>
>>>> <delete all configuration files and reboot all foo-X-private nodes>
>>>>
>>>> [root at foo-1-private ~]# gluster volume create foo foo-1-private:/mnt/brick
>>>> Creation of volume foo has been successful. Please start the volume to
>>>> access data.
>>>> [root at foo-1-private ~]# gluster peer probe foo-3-private
>>>> Probe successful
>>>> [root at foo-1-private ~]# gluster peer status
>>>> Number of Peers: 1
>>>>
>>>> Hostname: foo-3-private
>>>> Uuid: fe44c954-4679-4389-a0e6-4c1fd4569a02
>>>> State: Peer in Cluster (Connected)
>>>>
>>>> case-C) start volume then peer probe foo-3-private --> Peer Rejected
>>>>
>>>> <delete all configuration files and reboot all foo-X-private nodes>
>>>>
>>>> [root at foo-1-private ~]# gluster volume create foo foo-1-private:/mnt/brick
>>>> Creation of volume foo has been successful. Please start the volume to
>>>> access data.
>>>> [root at foo-1-private ~]# gluster volume start foo
>>>> Starting volume foo has been successful
>>>> [root at foo-1-private ~]# gluster peer probe foo-3-private
>>>> Probe successful
>>>> [root at foo-1-private ~]# gluster peer status
>>>> Number of Peers: 1
>>>>
>>>> Hostname: foo-3-private
>>>> Uuid: bb6932e4-5bf0-4d34-872f-4a5fc1d0b6f8
>>>> State: Peer Rejected (Connected)
>>>>
>>>>
>>>>> Can you for now put it in /etc/hosts and test?
>>>>
>>>> All foo-X-private hosts have no entries in /etc/hosts.
>>>> All the nodes obtain IP addresses from a DHCP server and register the IP
>>>> address - host name paires to a DNS server.
>>>>
>>>>>
>>>>> also, make sure you have same version of gluster running on all the nodes.
>>>>
>>>> Since all three foo-X-private hosts are generated from a common VM template,
>>>> same version of gluster run on all the nodes.
>>>>
>>>>>
>>>>> What's the result of gluster peer status on node 3?
>>>>
>>>> [root at foo-1-private ~]# ssh foo-3-private gluster peer status
>>>> Number of Peers: 1
>>>>
>>>> Hostname: 192.168.1.129
>>>> Uuid: 828bcc00-14d3-4505-8b35-d0ac6ca0730a
>>>> State: Peer Rejected (Connected)
>>>> [root at foo-1-private ~]#
>>>>
>>>>
>>>> Best,
>>>>
>>>>
>>>> (2011/08/17 0:53), Mohit Anchlia wrote:
>>>>>
>>>>> I see this in the logs:
>>>>>
>>>>> [2011-08-16 11:57:05.642903] I
>>>>> [glusterd-handler.c:391:glusterd_friend_find] 0-glusterd: Unable to
>>>>> find hostname: foo-3-private
>>>>>
>>>>> Can you for now put it in /etc/hosts and test?
>>>>>
>>>>> also, make sure you have same version of gluster running on all the nodes.
>>>>>
>>>>> What's the result of gluster peer status on node 3?
>>>>>
>>>>> On Mon, Aug 15, 2011 at 8:18 PM, Tomoaki Sato<tsato at valinux.co.jp> wrote:
>>>>>>
>>>>>> Mohit
>>>>>>
>>>>>> I've tried same test and reproduce the 'Peer Rejected' status.
>>>>>> please find config files and log files in attached taz.
>>>>>>
>>>>>>
>>>>>> [root at vhead-010 ~]# date
>>>>>> Tue Aug 16 11:55:15 JST 2011
>>>>>> [root at vhead-010 ~]# cat a.sh
>>>>>> #!/bin/bash
>>>>>> for i in foo-{1..3}-private
>>>>>> do
>>>>>> ssh ${i} service glusterd stop
>>>>>> ssh ${i} 'find /etc/glusterd -type f|xargs rm -f'
>>>>>> ssh ${i} rm -rf /etc/glusterd/vols/*
>>>>>> ssh ${i} service iptables stop
>>>>>> ssh ${i} cp /dev/null /var/log/glusterfs/nfs.log
>>>>>> ssh ${i} cp /dev/null /var/log/glusterfs/bricks/mnt-brick.log
>>>>>> ssh ${i} cp /dev/null /var/log/glusterfs/.cmd_log_history
>>>>>> ssh ${i} cp /dev/null
>>>>>> /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
>>>>>> ssh ${i} service glusterd start
>>>>>> ssh ${i} find /etc/glusterd
>>>>>> ssh ${i} service glusterd status
>>>>>> done
>>>>>> [root at vhead-010 ~]# bash a.sh
>>>>>> Stopping glusterd:[ OK ]
>>>>>> Flushing firewall rules: [ OK ]
>>>>>> Setting chains to policy ACCEPT: filter [ OK ]
>>>>>> Unloading iptables modules: [ OK ]
>>>>>> Starting glusterd:[ OK ]
>>>>>> /etc/glusterd
>>>>>> /etc/glusterd/glusterd.info
>>>>>> /etc/glusterd/nfs
>>>>>> /etc/glusterd/nfs/run
>>>>>> /etc/glusterd/peers
>>>>>> /etc/glusterd/vols
>>>>>> glusterd (pid 15617) is running...
>>>>>> Stopping glusterd:[ OK ]
>>>>>> Flushing firewall rules: [ OK ]
>>>>>> Setting chains to policy ACCEPT: filter [ OK ]
>>>>>> Unloading iptables modules: [ OK ]
>>>>>> Starting glusterd:[ OK ]
>>>>>> /etc/glusterd
>>>>>> /etc/glusterd/glusterd.info
>>>>>> /etc/glusterd/nfs
>>>>>> /etc/glusterd/nfs/run
>>>>>> /etc/glusterd/peers
>>>>>> /etc/glusterd/vols
>>>>>> glusterd (pid 15147) is running...
>>>>>> Stopping glusterd:[ OK ]
>>>>>> Flushing firewall rules: [ OK ]
>>>>>> Setting chains to policy ACCEPT: filter [ OK ]
>>>>>> Unloading iptables modules: [ OK ]
>>>>>> Starting glusterd:[ OK ]
>>>>>> /etc/glusterd
>>>>>> /etc/glusterd/glusterd.info
>>>>>> /etc/glusterd/nfs
>>>>>> /etc/glusterd/nfs/run
>>>>>> /etc/glusterd/peers
>>>>>> /etc/glusterd/vols
>>>>>> glusterd (pid 15177) is running...
>>>>>> [root at vhead-010 ~]# ssh foo-1-private
>>>>>> Last login: Tue Aug 16 09:51:49 2011 from dlp.local.valinux.co.jp
>>>>>> [root at localhost ~]# gluster peer probe foo-2-private
>>>>>> Probe successful
>>>>>> [root at localhost ~]# gluster peer status
>>>>>> Number of Peers: 1
>>>>>>
>>>>>> Hostname: foo-2-private
>>>>>> Uuid: 20b73d9a-ede0-454f-9fbb-b0eee9ce26a3
>>>>>> State: Peer in Cluster (Connected)
>>>>>> [root at localhost ~]# gluster volume create foo foo-1-private:/mnt/brick
>>>>>> Creation of volume foo has been successful. Please start the volume to
>>>>>> access data.
>>>>>> [root at localhost ~]# gluster volume start foo
>>>>>> Starting volume foo has been successful
>>>>>> [root at localhost ~]# gluster volume add-brick foo foo-2-private:/mnt/brick
>>>>>> Add Brick successful
>>>>>> [root at localhost ~]# gluster peer probe foo-3-private
>>>>>> Probe successful
>>>>>> [root at localhost ~]# gluster peer status
>>>>>> Number of Peers: 2
>>>>>>
>>>>>> Hostname: foo-2-private
>>>>>> Uuid: 20b73d9a-ede0-454f-9fbb-b0eee9ce26a3
>>>>>> State: Peer in Cluster (Connected)
>>>>>>
>>>>>> Hostname: foo-3-private
>>>>>> Uuid: 7587ae34-9209-484a-9576-3939e061720c
>>>>>> State: Peer Rejected (Connected)
>>>>>> [root at localhost ~]# exit
>>>>>> logout
>>>>>> Connection to foo-1-private closed.
>>>>>> [root at vhead-010 ~]# find foo_log_and_conf
>>>>>> foo_log_and_conf
>>>>>> foo_log_and_conf/foo-2-private
>>>>>> foo_log_and_conf/foo-2-private/glusterd
>>>>>> foo_log_and_conf/foo-2-private/glusterd/glusterd.info
>>>>>> foo_log_and_conf/foo-2-private/glusterd/nfs
>>>>>> foo_log_and_conf/foo-2-private/glusterd/nfs/nfs-server.vol
>>>>>> foo_log_and_conf/foo-2-private/glusterd/nfs/run
>>>>>> foo_log_and_conf/foo-2-private/glusterd/nfs/run/nfs.pid
>>>>>> foo_log_and_conf/foo-2-private/glusterd/peers
>>>>>>
>>>>>> foo_log_and_conf/foo-2-private/glusterd/peers/461f6e21-90c4-4b6c-bda8-7b99bacb2722
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/info
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/bricks
>>>>>>
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/bricks/foo-2-private:-mnt-brick
>>>>>>
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/bricks/foo-1-private:-mnt-brick
>>>>>>
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/foo.foo-2-private.mnt-brick.vol
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/cksum
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/run
>>>>>>
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/run/foo-2-private-mnt-brick.pid
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/foo-fuse.vol
>>>>>>
>>>>>> foo_log_and_conf/foo-2-private/glusterd/vols/foo/foo.foo-1-private.mnt-brick.vol
>>>>>> foo_log_and_conf/foo-2-private/glusterfs
>>>>>> foo_log_and_conf/foo-2-private/glusterfs/nfs.log
>>>>>> foo_log_and_conf/foo-2-private/glusterfs/bricks
>>>>>> foo_log_and_conf/foo-2-private/glusterfs/bricks/mnt-brick.log
>>>>>> foo_log_and_conf/foo-2-private/glusterfs/.cmd_log_history
>>>>>> foo_log_and_conf/foo-2-private/glusterfs/etc-glusterfs-glusterd.vol.log
>>>>>> foo_log_and_conf/foo-1-private
>>>>>> foo_log_and_conf/foo-1-private/glusterd
>>>>>> foo_log_and_conf/foo-1-private/glusterd/glusterd.info
>>>>>> foo_log_and_conf/foo-1-private/glusterd/nfs
>>>>>> foo_log_and_conf/foo-1-private/glusterd/nfs/nfs-server.vol
>>>>>> foo_log_and_conf/foo-1-private/glusterd/nfs/run
>>>>>> foo_log_and_conf/foo-1-private/glusterd/nfs/run/nfs.pid
>>>>>> foo_log_and_conf/foo-1-private/glusterd/peers
>>>>>>
>>>>>> foo_log_and_conf/foo-1-private/glusterd/peers/20b73d9a-ede0-454f-9fbb-b0eee9ce26a3
>>>>>>
>>>>>> foo_log_and_conf/foo-1-private/glusterd/peers/7587ae34-9209-484a-9576-3939e061720c
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/info
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/bricks
>>>>>>
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/bricks/foo-2-private:-mnt-brick
>>>>>>
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/bricks/foo-1-private:-mnt-brick
>>>>>>
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/foo.foo-2-private.mnt-brick.vol
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/cksum
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/run
>>>>>>
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/run/foo-1-private-mnt-brick.pid
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/foo-fuse.vol
>>>>>>
>>>>>> foo_log_and_conf/foo-1-private/glusterd/vols/foo/foo.foo-1-private.mnt-brick.vol
>>>>>> foo_log_and_conf/foo-1-private/glusterfs
>>>>>> foo_log_and_conf/foo-1-private/glusterfs/nfs.log
>>>>>> foo_log_and_conf/foo-1-private/glusterfs/bricks
>>>>>> foo_log_and_conf/foo-1-private/glusterfs/bricks/mnt-brick.log
>>>>>> foo_log_and_conf/foo-1-private/glusterfs/.cmd_log_history
>>>>>> foo_log_and_conf/foo-1-private/glusterfs/etc-glusterfs-glusterd.vol.log
>>>>>> foo_log_and_conf/foo-3-private
>>>>>> foo_log_and_conf/foo-3-private/glusterd
>>>>>> foo_log_and_conf/foo-3-private/glusterd/glusterd.info
>>>>>> foo_log_and_conf/foo-3-private/glusterd/nfs
>>>>>> foo_log_and_conf/foo-3-private/glusterd/nfs/run
>>>>>> foo_log_and_conf/foo-3-private/glusterd/peers
>>>>>>
>>>>>> foo_log_and_conf/foo-3-private/glusterd/peers/461f6e21-90c4-4b6c-bda8-7b99bacb2722
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo/info
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo/bricks
>>>>>>
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo/bricks/foo-2-private:-mnt-brick
>>>>>>
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo/bricks/foo-1-private:-mnt-brick
>>>>>>
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo/foo.foo-2-private.mnt-brick.vol
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo/cksum
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo/foo-fuse.vol
>>>>>>
>>>>>> foo_log_and_conf/foo-3-private/glusterd/vols/foo/foo.foo-1-private.mnt-brick.vol
>>>>>> foo_log_and_conf/foo-3-private/glusterfs
>>>>>> foo_log_and_conf/foo-3-private/glusterfs/nfs.log
>>>>>> foo_log_and_conf/foo-3-private/glusterfs/bricks
>>>>>> foo_log_and_conf/foo-3-private/glusterfs/bricks/mnt-brick.log
>>>>>> foo_log_and_conf/foo-3-private/glusterfs/.cmd_log_history
>>>>>> foo_log_and_conf/foo-3-private/glusterfs/etc-glusterfs-glusterd.vol.log
>>>>>> [root at vhead-010 ~]# exit
>>>>>>
>>>>>> Best,
>>>>>>
>>>>>> (2011/08/16 9:35), Mohit Anchlia wrote:
>>>>>>>
>>>>>>> I should have also asked you to stop and delete volume before getting
>>>>>>> rid of gluster config files. Can you get rid of directories also
>>>>>>> inside vols and try to restart? It's trying to look for volume files
>>>>>>> that we just removed.
>>>>>>>
>>>>>>> Also, just disable iptables for now explicitly.
>>>>>>>
>>>>>>> On Mon, Aug 15, 2011 at 5:22 PM, Tomoaki Sato<tsato at valinux.co.jp>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> 1) run peer detach for all the servers
>>>>>>>>
>>>>>>>> done.
>>>>>>>>
>>>>>>>>> 2) from server 1 ->3 and 3->1 make sure ports are open and iptables
>>>>>>>>> are turned off.
>>>>>>>>
>>>>>>>> done.
>>>>>>>> by the way, the same test on 3.1.5-1 works fine with same environment.
>>>>>>>>
>>>>>>>>> 3) remove config files under /etc/glusterd
>>>>>>>>
>>>>>>>> please review following logs.
>>>>>>>>
>>>>>>>>> 4) run your tests again.
>>>>>>>>
>>>>>>>> I don't know why but glusterd service failed to start on all 3 hosts.
>>>>>>>>
>>>>>>>> [root at vhead-010 ~]# date
>>>>>>>> Tue Aug 16 09:12:53 JST 2011
>>>>>>>> [root at vhead-010 ~]# cat a.sh
>>>>>>>> #!/bin/bash
>>>>>>>> for i in foo-{1..3}-private
>>>>>>>> do
>>>>>>>> ssh ${i} service glusterd stop
>>>>>>>> ssh ${i} 'find /etc/glusterd -type f|xargs rm -f'
>>>>>>>> ssh ${i} service iptables restart
>>>>>>>> ssh ${i} iptables -vL
>>>>>>>> ssh ${i} service glusterd start
>>>>>>>> ssh ${i} find /etc/glusterd
>>>>>>>> ssh ${i} service glusterd status
>>>>>>>> done
>>>>>>>> [root at vhead-010 ~]# bash a.sh
>>>>>>>> Stopping glusterd:[ OK ]
>>>>>>>> Flushing firewall rules: [ OK ]
>>>>>>>> Setting chains to policy ACCEPT: filter [ OK ]
>>>>>>>> Unloading iptables modules: [ OK ]
>>>>>>>> Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>>
>>>>>>>> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>>
>>>>>>>> Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>> Starting glusterd:[ OK ]
>>>>>>>> /etc/glusterd
>>>>>>>> /etc/glusterd/glusterd.info
>>>>>>>> /etc/glusterd/nfs
>>>>>>>> /etc/glusterd/nfs/run
>>>>>>>> /etc/glusterd/peers
>>>>>>>> /etc/glusterd/vols
>>>>>>>> /etc/glusterd/vols/foo
>>>>>>>> /etc/glusterd/vols/foo/bricks
>>>>>>>> /etc/glusterd/vols/foo/run
>>>>>>>> glusterd is stopped
>>>>>>>> Stopping glusterd:[ OK ]
>>>>>>>> Flushing firewall rules: [ OK ]
>>>>>>>> Setting chains to policy ACCEPT: filter [ OK ]
>>>>>>>> Unloading iptables modules: [ OK ]
>>>>>>>> Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>>
>>>>>>>> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>>
>>>>>>>> Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>> Starting glusterd:[ OK ]
>>>>>>>> /etc/glusterd
>>>>>>>> /etc/glusterd/glusterd.info
>>>>>>>> /etc/glusterd/nfs
>>>>>>>> /etc/glusterd/nfs/run
>>>>>>>> /etc/glusterd/peers
>>>>>>>> /etc/glusterd/vols
>>>>>>>> /etc/glusterd/vols/foo
>>>>>>>> /etc/glusterd/vols/foo/bricks
>>>>>>>> /etc/glusterd/vols/foo/run
>>>>>>>> glusterd is stopped
>>>>>>>> Stopping glusterd:[ OK ]
>>>>>>>> Flushing firewall rules: [ OK ]
>>>>>>>> Setting chains to policy ACCEPT: filter [ OK ]
>>>>>>>> Unloading iptables modules: [ OK ]
>>>>>>>> Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>>
>>>>>>>> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>>
>>>>>>>> Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
>>>>>>>> pkts bytes target prot opt in out source
>>>>>>>> destination
>>>>>>>> Starting glusterd:[ OK ]
>>>>>>>> /etc/glusterd
>>>>>>>> /etc/glusterd/glusterd.info
>>>>>>>> /etc/glusterd/nfs
>>>>>>>> /etc/glusterd/nfs/run
>>>>>>>> /etc/glusterd/peers
>>>>>>>> /etc/glusterd/vols
>>>>>>>> /etc/glusterd/vols/foo
>>>>>>>> /etc/glusterd/vols/foo/bricks
>>>>>>>> /etc/glusterd/vols/foo/run
>>>>>>>> glusterd is stopped
>>>>>>>> [root at vhead-010 ~]# date
>>>>>>>> Tue Aug 16 09:13:20 JST 2011
>>>>>>>> [root at vhead-010 ~]# ssh foo-1-private
>>>>>>>> Last login: Tue Aug 16 09:06:57 2011 from dlp.local.valinux.co.jp
>>>>>>>> [root at localhost ~]# tail -20
>>>>>>>> /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
>>>>>>>> ...
>>>>>>>> [2011-08-16 09:13:01.85858] I [glusterd.c:304:init] 0-management: Using
>>>>>>>> /etc/glusterd as working directory
>>>>>>>> [2011-08-16 09:13:01.87294] E [rpc-transport.c:799:rpc_transport_load]
>>>>>>>> 0-rpc-transport:
>>>>>>>> /opt/glusterfs/3.1.6/lib64/glusterfs/3.1.6/rpc-transport/rdma.so:
>>>>>>>> cannot
>>>>>>>> open shared object file: No such file or directory
>>>>>>>> [2011-08-16 09:13:01.87340] E [rpc-transport.c:803:rpc_transport_load]
>>>>>>>> 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not
>>>>>>>> valid or not found on this machine
>>>>>>>> [2011-08-16 09:13:01.87402] E
>>>>>>>> [glusterd-store.c:654:glusterd_store_handle_retrieve] 0-glusterd:
>>>>>>>> Unable
>>>>>>>> to
>>>>>>>> retrieve store handle for /etc/glusterd/glusterd.info, error: No such
>>>>>>>> file
>>>>>>>> or directory
>>>>>>>> [2011-08-16 09:13:01.87422] E
>>>>>>>> [glusterd-store.c:761:glusterd_retrieve_uuid]
>>>>>>>> 0-: Unable to get store handle!
>>>>>>>> [2011-08-16 09:13:01.87514] I [glusterd.c:95:glusterd_uuid_init]
>>>>>>>> 0-glusterd:
>>>>>>>> generated UUID: c0cef9f9-a79e-4189-8955-d83927db9cee
>>>>>>>> [2011-08-16 09:13:01.87681] E
>>>>>>>> [glusterd-store.c:654:glusterd_store_handle_retrieve] 0-glusterd:
>>>>>>>> Unable
>>>>>>>> to
>>>>>>>> retrieve store handle for /etc/glusterd/vols/foo/info, error: No such
>>>>>>>> file
>>>>>>>> or directory
>>>>>>>> [2011-08-16 09:13:01.87704] E
>>>>>>>> [glusterd-store.c:1328:glusterd_store_retrieve_volumes] 0-: Unable to
>>>>>>>> restore volume: foo
>>>>>>>
>>>>>>>> [2011-08-16 09:13:01.87732] E [xlator.c:843:xlator_init] 0-management:
>>>>>>>> Initialization of volume 'management' failed, review your volfile again
>>>>>>>> [2011-08-16 09:13:01.87751] E [graph.c:331:glusterfs_graph_init]
>>>>>>>> 0-management: initializing translator failed
>>>>>>>> [2011-08-16 09:13:01.87818] I [glusterfsd.c:712:cleanup_and_exit]
>>>>>>>> 0-glusterfsd: shutting down
>>>>>>>> [root at localhost ~]# exit
>>>>>>>>
>>>>>>>> Best,
>>>>>>>>
>>>>>>>> (2011/08/16 8:52), Mohit Anchlia wrote:
>>>>>>>>>
>>>>>>>>> Logs are generally in /var/log/gluster
>>>>>>>>>
>>>>>>>>> Since you are playing with it. I would suggest this:
>>>>>>>>>
>>>>>>>>> 1) run peer detach for all the servers
>>>>>>>>> 2) from server 1 ->3 and 3->1 make sure ports are open and iptables
>>>>>>>>> are turned off.
>>>>>>>>> 3) remove config files under /etc/glusterd
>>>>>>>>> 4) run your tests again.
>>>>>>>>>
>>>>>>>>> On Mon, Aug 15, 2011 at 4:28 PM, Tomoaki Sato<tsato at valinux.co.jp>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Thanks, Mohit
>>>>>>>>>>
>>>>>>>>>> (2011/08/16 8:05), Mohit Anchlia wrote:
>>>>>>>>>>>
>>>>>>>>>>> What's in your logs?
>>>>>>>>>>
>>>>>>>>>> I can obtain logs needed. could you tell me the instruction to take
>>>>>>>>>> the
>>>>>>>>>> logs?
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Did you have foo-3-private before in your gluster cluster ever or
>>>>>>>>>>> adding this host for the first time?
>>>>>>>>>>
>>>>>>>>>> It was first time.
>>>>>>>>>> All foo-X-private has no entries in /etc/glusterd/peers/ and
>>>>>>>>>> /etc/glusterd/vols/.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Try gluster peer detach and then remove any left over configuration
>>>>>>>>>>> in
>>>>>>>>>>> /etc/glusterd config directory. After that try again and see if that
>>>>>>>>>>> works.
>>>>>>>>>>
>>>>>>>>>> [root at vhead-010 ~]# date
>>>>>>>>>> Tue Aug 16 08:17:49 JST 2011
>>>>>>>>>> [root at vhead-010 ~]# cat a.sh
>>>>>>>>>> #!/bin/bash
>>>>>>>>>> for i in foo-{1..3}-private
>>>>>>>>>> do
>>>>>>>>>> ssh ${i} service glusterd stop
>>>>>>>>>> ssh ${i} rm -rf /etc/glusterd/peers/*
>>>>>>>>>> ssh ${i} rm -rf /etc/glusterd/vols/*
>>>>>>>>>> ssh ${i} service glusterd start
>>>>>>>>>> ssh ${i} find /etc/glusterd
>>>>>>>>>> done
>>>>>>>>>> [root at vhead-010 ~]# bash a.sh
>>>>>>>>>> Stopping glusterd:[ OK ]
>>>>>>>>>> Starting glusterd:[ OK ]
>>>>>>>>>> /etc/glusterd
>>>>>>>>>> /etc/glusterd/glusterd.info
>>>>>>>>>> /etc/glusterd/nfs
>>>>>>>>>> /etc/glusterd/nfs/nfs-server.vol
>>>>>>>>>> /etc/glusterd/nfs/run
>>>>>>>>>> /etc/glusterd/peers
>>>>>>>>>> /etc/glusterd/vols
>>>>>>>>>> Stopping glusterd:[ OK ]
>>>>>>>>>> Starting glusterd:[ OK ]
>>>>>>>>>> /etc/glusterd
>>>>>>>>>> /etc/glusterd/glusterd.info
>>>>>>>>>> /etc/glusterd/nfs
>>>>>>>>>> /etc/glusterd/nfs/nfs-server.vol
>>>>>>>>>> /etc/glusterd/nfs/run
>>>>>>>>>> /etc/glusterd/peers
>>>>>>>>>> /etc/glusterd/vols
>>>>>>>>>> Stopping glusterd:[ OK ]
>>>>>>>>>> Starting glusterd:[ OK ]
>>>>>>>>>> /etc/glusterd
>>>>>>>>>> /etc/glusterd/glusterd.info
>>>>>>>>>> /etc/glusterd/nfs
>>>>>>>>>> /etc/glusterd/nfs/nfs-server.vol
>>>>>>>>>> /etc/glusterd/nfs/run
>>>>>>>>>> /etc/glusterd/peers
>>>>>>>>>> /etc/glusterd/vols
>>>>>>>>>> [root at vhead-010 ~]# ssh foo-1-private
>>>>>>>>>> [root at localhost ~]# gluster peer probe foo-2-private
>>>>>>>>>> Probe successful
>>>>>>>>>> [root at localhost ~]# gluster peer status
>>>>>>>>>> Number of Peers: 1
>>>>>>>>>>
>>>>>>>>>> Hostname: foo-2-private
>>>>>>>>>> Uuid: c2b314ac-6ed1-455a-84d4-ec22041ee2b2
>>>>>>>>>> State: Peer in Cluster (Connected)
>>>>>>>>>> [root at localhost ~]# gluster volume create foo
>>>>>>>>>> foo-1-private:/mnt/brick
>>>>>>>>>> Creation of volume foo has been successful. Please start the volume
>>>>>>>>>> to
>>>>>>>>>> access da
>>>>>>>>>> ta.
>>>>>>>>>> [root at localhost ~]# gluster volume start foo
>>>>>>>>>> Starting volume foo has been successful
>>>>>>>>>> [root at localhost ~]# gluster volume add-brick foo
>>>>>>>>>> foo-2-private:/mnt/brick
>>>>>>>>>> Add Brick successful
>>>>>>>>>> [root at localhost ~]# gluster peer probe foo-3-private
>>>>>>>>>> Probe successful
>>>>>>>>>> [root at localhost ~]# gluster peer status
>>>>>>>>>> Number of Peers: 2
>>>>>>>>>>
>>>>>>>>>> Hostname: foo-2-private
>>>>>>>>>> Uuid: c2b314ac-6ed1-455a-84d4-ec22041ee2b2
>>>>>>>>>> State: Peer in Cluster (Connected)
>>>>>>>>>>
>>>>>>>>>> Hostname: foo-3-private
>>>>>>>>>> Uuid: 7fb98dac-fef7-4b33-837c-6483a767ec3e
>>>>>>>>>> State: Peer Rejected (Connected)
>>>>>>>>>> [root at localhost ~]# cat /var/log/glusterfs/.cmd_log_history
>>>>>>>>>> ...
>>>>>>>>>> [2011-08-16 08:20:28.862619] peer probe : on host
>>>>>>>>>> foo-2-private:24007
>>>>>>>>>> [2011-08-16 08:20:28.912419] peer probe : on host foo-2-private:24007
>>>>>>>>>> FAILED
>>>>>>>>>> [2011-08-16 08:20:58.382350] Volume create : on volname: foo
>>>>>>>>>> attempted
>>>>>>>>>> [2011-08-16 08:20:58.382461] Volume create : on volname: foo
>>>>>>>>>> type:DEFAULT
>>>>>>>>>> count:
>>>>>>>>>> 1 bricks: foo-1-private:/mnt/brick
>>>>>>>>>> [2011-08-16 08:20:58.384674] Volume create : on volname: foo SUCCESS
>>>>>>>>>> [2011-08-16 08:21:04.831772] volume start : on volname: foo SUCCESS
>>>>>>>>>> [2011-08-16 08:21:22.682292] Volume add-brick : on volname: foo
>>>>>>>>>> attempted
>>>>>>>>>> [2011-08-16 08:21:22.682385] Volume add-brick : volname: foo type
>>>>>>>>>> DEFAULT
>>>>>>>>>> count:
>>>>>>>>>> 1 bricks: foo-2-private:/mnt/brick
>>>>>>>>>> [2011-08-16 08:21:22.682499] Volume add-brick : on volname: foo
>>>>>>>>>> SUCCESS
>>>>>>>>>> [2011-08-16 08:21:39.124574] peer probe : on host
>>>>>>>>>> foo-3-private:24007
>>>>>>>>>> [2011-08-16 08:21:39.135609] peer probe : on host foo-3-private:24007
>>>>>>>>>> FAILED
>>>>>>>>>>
>>>>>>>>>> Tomo
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Aug 15, 2011 at 3:37 PM, Tomoaki Sato<tsato at valinux.co.jp>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> following instructions work fine with 3.1.5-1 but with 3.1.6-1.
>>>>>>>>>>>>
>>>>>>>>>>>> 1. make a new file system without peers. [OK]
>>>>>>>>>>>>
>>>>>>>>>>>> foo-1-private# gluster volume create foo foo-1-private:/mnt/brick
>>>>>>>>>>>> foo-1-private# gluster volume start foo
>>>>>>>>>>>> foo-1-private# gluster peer status
>>>>>>>>>>>> No peers present
>>>>>>>>>>>> foo-1-private#
>>>>>>>>>>>>
>>>>>>>>>>>> 2. add a peer to the file system. [NG]
>>>>>>>>>>>>
>>>>>>>>>>>> foo-1-private# gluster peer probe foo-2-private
>>>>>>>>>>>> Probe successful
>>>>>>>>>>>> foo-1-private# gluster peer status
>>>>>>>>>>>> Number of Peers: 1
>>>>>>>>>>>>
>>>>>>>>>>>> Hostname: foo-2-private
>>>>>>>>>>>> Uuid: c2b314ac-6ed1-455a-84d4-ec22041ee2b2
>>>>>>>>>>>> State: Peer Rejected (Connected)
>>>>>>>>>>>> foo-1-private# gluster volume add-brick foo
>>>>>>>>>>>> foo-2-private:/mnt/brick
>>>>>>>>>>>> Host foo-2-private not connected
>>>>>>>>>>>> foo-1-private#
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> following instructions work fine even with 3.1.6-1.
>>>>>>>>>>>>
>>>>>>>>>>>> 1. make a new file system with single peer. [OK]
>>>>>>>>>>>>
>>>>>>>>>>>> foo-1-private# gluster peer status
>>>>>>>>>>>> No peer presents
>>>>>>>>>>>> foo-1-private# gluster peer probe foo-2-private
>>>>>>>>>>>> Probe successful
>>>>>>>>>>>> foo-1-private# gluster peer status
>>>>>>>>>>>> Number of Peers: 1
>>>>>>>>>>>>
>>>>>>>>>>>> Hostname: foo-2-private
>>>>>>>>>>>> Uuid: c2b314ac-6ed1-455a-84d4-ec22041ee2b2
>>>>>>>>>>>> State: Peer in Cluster (Connected)
>>>>>>>>>>>> foo-1-private# gluster volume create foo foo-1-private:/mnt/brick
>>>>>>>>>>>> Creation of volume foo has been successful. Please start the volume
>>>>>>>>>>>> to
>>>>>>>>>>>> access data.
>>>>>>>>>>>> foo-1-private# gluster volume start foo
>>>>>>>>>>>> Starting volume foo has been successful
>>>>>>>>>>>> foo-1-private# gluster volume add-brick foo
>>>>>>>>>>>> foo-2-private:/mnt/brick
>>>>>>>>>>>> Add Brick successful
>>>>>>>>>>>> foo-1-private#
>>>>>>>>>>>>
>>>>>>>>>>>> But ...
>>>>>>>>>>>>
>>>>>>>>>>>> 2. add a peer to the file system. [NG]
>>>>>>>>>>>>
>>>>>>>>>>>> foo-1-private# gluster peer probe foo-3-private
>>>>>>>>>>>> Probe successful
>>>>>>>>>>>> foo-1-private# gluster peer status
>>>>>>>>>>>> Number of Peers: 2
>>>>>>>>>>>>
>>>>>>>>>>>> Hostname: foo-2-private
>>>>>>>>>>>> Uuid: c2b314ac-6ed1-455a-84d4-ec22041ee2b2
>>>>>>>>>>>> State: Peer in Cluster (Connected)
>>>>>>>>>>>>
>>>>>>>>>>>> Hostname: foo-3-private
>>>>>>>>>>>> Uuid: 7fb98dac-fef704b33-837c-6483a767ec3e
>>>>>>>>>>>> State: Peer Rejected (Connected)
>>>>>>>>>>>> foo-1-private# gluster volume add-brick foo
>>>>>>>>>>>> foo-3-private:/mnt/brick
>>>>>>>>>>>> Host foo-3-private not connected
>>>>>>>>>>>> foo-1-private#
>>>>>>>>>>>>
>>>>>>>>>>>> How should I add extra peers to existing file systems ?
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>>>> Gluster-users at gluster.org
>>>>>>>>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

-------------- next part --------------
A non-text attachment was scrubbed...
Name: foo-1-private.tar
Type: application/octet-stream
Size: 10240 bytes
Desc: not available
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110817/448b88a6/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: foo-3-private.tar
Type: application/octet-stream
Size: 10240 bytes
Desc: not available
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110817/448b88a6/attachment-0001.obj>