[Gluster-users] Cannot get geo-replicate working with Gluster 3.7
Aravinda
avishwan at redhat.com
Wed Aug 12 05:41:43 UTC 2015
This looks like SSH configuration issue. Please cleanup all the lines in
/root/.ssh/authorized_keys which are connected from Master nodes which
do not starts with "command="
Please let us know, which ssh key is used to create passwordless SSH
from Master node to Slave node.
To resolve the issue.
1. In "neptune" node,
cat /var/lib/glusterd/geo-replication/common_secret.pem.pub
2. Open /root/.ssh/authorized_keys file in "volume01-replicate" node,
and see the keys present or not(Output from previous step)
3. There should not be any other lines which has the same key without
"command=" in the beginning
For example
ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAABAQDhR4kp978pjze9y1ozySB6jgz2VjeLKnCWIIsZ7NFCue1S4lCU7TgNg2g8FwfXR7LX4mRuLFtQeOkEN9kLGaiJZiN06oU2Jz3Y2gx6egxR5lGiumMg7QLPH1PQPJIfT8Qaz1znH+NlpM1BuivjfOsbtVWTBQpANq4uA8ooln2rLTKIzRGQrS6adUD6KwbjIpVEahJqkZf8YaiaTDJZdXdGGvT6YtytogPmuKwrJ+XujaRd49dDcjeOrcjkFxsf9/IuqBvbZYwW2hwTcqqtSHZfIwHaf6X9fhDizVX4WxPhToiK9LZaEF57hnPAa7bl2if9KFoOyfwZByTIwQPqjymv
root at neptune
If exists, remove that line.
4. Try to connect from "neptune" node using following command to see the
issue is resolved
ssh -i /var/lib/glusterd/geo-replication/secret.pem root at volume01-replicate
If you see gsyncd messages then everything is normal. If you see ssh
errors then issue is not resolved.
Let us know if any questions.
regards
Aravinda
On 08/11/2015 03:08 AM, Don Ky wrote:
> Hello all,
>
> I've been struggling to get gluster-geo replicate functionality
> working for the last couple of days. I keep getting the following errors:
>
>
> 2015-08-10 17:27:07.855817] E [resource(/gluster/volume1):222:errlog]
> Popen: command "ssh -oPasswordAuthentication=no
> -oStrictHostKeyChecking=no -i
> /var/lib/glusterd/geo-replication/secret.pem -oControlMaster=auto -S
> /tmp/gsyncd-aux-ssh-Cnh7xL/ee1e6b6c8823302e93454e632bd81fbe.sock
> root at gluster02.example.com <mailto:root at gluster02.example.com>
> /nonexistent/gsyncd --session-owner
> 50600483-7aa3-4fab-a66c-63350af607b0 -N --listen --timeout 120
> gluster://localhost:volume1-replicate" returned with 127, saying:
> [2015-08-10 17:27:07.856066] E [resource(/gluster/volume1):226:logerr]
> Popen: ssh> bash: /nonexistent/gsyncd: No such file or directory
> [2015-08-10 17:27:07.856441] I
> [syncdutils(/gluster/volume1):220:finalize] <top>: exiting.
> [2015-08-10 17:27:07.858120] I [repce(agent):92:service_loop]
> RepceServer: terminating on reaching EOF.
> [2015-08-10 17:27:07.858361] I [syncdutils(agent):220:finalize] <top>:
> exiting.
> [2015-08-10 17:27:07.858211] I [monitor(monitor):274:monitor] Monitor:
> worker(/gluster/volume1) died before establishing connection
> [2015-08-10 17:27:18.181344] I [monitor(monitor):221:monitor] Monitor:
> ------------------------------------------------------------
> [2015-08-10 17:27:18.181842] I [monitor(monitor):222:monitor] Monitor:
> starting gsyncd worker
> [2015-08-10 17:27:18.387790] I [gsyncd(/gluster/volume1):649:main_i]
> <top>: syncing: gluster://localhost:volume1 ->
> ssh://root@gluster02.example.com:gluster://localhost:volume1-replicate
> [2015-08-10 17:27:18.389427] D [gsyncd(agent):643:main_i] <top>:
> rpc_fd: '7,11,10,9'
> [2015-08-10 17:27:18.390553] I [changelogagent(agent):75:__init__]
> ChangelogAgent: Agent listining...
> [2015-08-10 17:27:18.418788] D [repce(/gluster/volume1):191:push]
> RepceClient: call 8460:140341431777088:1439242038.42
> __repce_version__() ...
> [2015-08-10 17:27:18.629983] E
> [syncdutils(/gluster/volume1):252:log_raise_exception] <top>:
> connection to peer is broken
> [2015-08-10 17:27:18.630651] W
> [syncdutils(/gluster/volume1):256:log_raise_exception] <top>:
> !!!!!!!!!!!!!
> [2015-08-10 17:27:18.630794] W
> [syncdutils(/gluster/volume1):257:log_raise_exception] <top>: !!!
> getting "No such file or directory" errors is most likely due to
> MISCONFIGURATION, please consult
> https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.1/html/Administration_Guide/chap-User_Guide-Geo_Rep-Preparation-Settingup_Environment.html
> [2015-08-10 17:27:18.630929] W
> [syncdutils(/gluster/volume1):265:log_raise_exception] <top>:
> !!!!!!!!!!!!!
> [2015-08-10 17:27:18.631129] E [resource(/gluster/volume1):222:errlog]
> Popen: command "ssh -oPasswordAuthentication=no
> -oStrictHostKeyChecking=no -i
> /var/lib/glusterd/geo-replication/secret.pem -oControlMaster=auto -S
> /tmp/gsyncd-aux-ssh-RPuEyN/ee1e6b6c8823302e93454e632bd81fbe.sock
> root at gluster02.example.com <mailto:root at gluster02.example.com>
> /nonexistent/gsyncd --session-owner
> 50600483-7aa3-4fab-a66c-63350af607b0 -N --listen --timeout 120
> gluster://localhost:volume1-replicate" returned with 127, saying:
> [2015-08-10 17:27:18.631280] E [resource(/gluster/volume1):226:logerr]
> Popen: ssh> bash: /nonexistent/gsyncd: No such file or directory
> [2015-08-10 17:27:18.631567] I
> [syncdutils(/gluster/volume1):220:finalize] <top>: exiting.
> [2015-08-10 17:27:18.633125] I [repce(agent):92:service_loop]
> RepceServer: terminating on reaching EOF.
> [2015-08-10 17:27:18.633183] I [monitor(monitor):274:monitor] Monitor:
> worker(/gluster/volume1) died before establishing connection
> [2015-08-10 17:27:18.633392] I [syncdutils(agent):220:finalize] <top>:
> exiting.
>
> and the status is continuously faulty:
>
> [root at neptune volume1]# gluster volume geo-replication volume01
> gluster02::volume01-replicate status
>
> MASTER NODE MASTER VOL MASTER BRICK SLAVE USER
> SLAVE SLAVE NODE STATUS CRAWL STATUS
> LAST_SYNCED
> -------------------------------------------------------------------------------------------------------------------------------------------------------
> neptune volume01 /gluster/volume01 root
> gluster02::volume01-replicate N/A Faulty N/A
> N/A
>
> What I'm trying to accomplish is to mirror a volume from gluster01
> (master) to gluster02 (slave).
>
> Here is a break down of the steps I took
>
> yum -y install glusterfs-server glusterfs-geo-replication
> service glusterd start
>
> #gluster01
> gluster volume create volume1 gluster01.example.com:/gluster/volume1
> gluster volume start volume1
>
> #gluster02
> gluster volume create volume1-replicate
> gluster02.example.com:/gluster/volume1-replicate
> gluster volume start volume1-replicate
>
>
> #geo replicate
> gluster system:: execute gsec_create
>
> #gluster01
> gluster volume geo-replication volume1 gluster02::volume1-replicate
> create push-pem
> gluster volume geo-replication volume1 gluster02::volume1-replicate start
> gluster volume geo-replication volume1 gluster02::volume1-replicate status
>
> #mouting and testing
> mkdir /mnt/gluster
> mount -t glusterfs gluster01.example.com:/volume1 /mnt/gluster
> mount -t glusterfs gluster02.example.com:/volume1-replicate /mnt/gluster
>
> #troubleshooting
> gluster volume geo-replication volume1 gluster02::volume1-replicate
> config log-level DEBUG
> service glusterd restart
>
> gluster volume geo-replication volume1 gluster02::volume1-replicate config
>
> There was one step before running
>
> gluster volume geo-replication volume1 gluster02::volume1-replicate
> create push-pem
>
> I copied the secret.pub to gluster02(the slave) and added it to
> .ssh/authorized_keys. I can ssh as root from gluster01 to gluster02 fine.
>
> I'm currently running:
>
> glusterfs-3.7.3-1.el7.x86_64
> glusterfs-cli-3.7.3-1.el7.x86_64
> glusterfs-libs-3.7.3-1.el7.x86_64
> glusterfs-client-xlators-3.7.3-1.el7.x86_64
> glusterfs-fuse-3.7.3-1.el7.x86_64
> glusterfs-server-3.7.3-1.el7.x86_64
> glusterfs-api-3.7.3-1.el7.x86_64
> glusterfs-geo-replication-3.7.3-1.el7.x86_64
>
> on both slave and master servers. Both servers have ntp installed are
> in sync and patched.
>
> I can mount volume1 or volume1-replicate on each host and confirmed
> that iptables have been flushed.
>
> Not sure exactly what else to check at this point. There appeared to
> be another user with similar errors but the mailing list says he
> resolved it on his own.
>
> Any ideas? I'm completely lost on what could be issue. Some of the
> redhat docs mentioned it could be fuse but it looks like fuse is
> installed as part of gluster.
>
> Thanks
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150812/5f28711e/attachment.html>
More information about the Gluster-users
mailing list