[Gluster-users] problems with geo-replication on 3.7.4

Tue Sep 22 07:00:13 UTC 2015

Hi,
Replies inline.

Thanks,
Saravana

On 09/21/2015 03:56 PM, ML mail wrote:
> That's right, the earlier error I've posted with ZFS actually only appeared during the setup of the geo replication and does not appear anymore. In fact ZFS does not have any inodes so I guess you would need to adapt the GlusterFS code to check if the FS is ZFS or not.
>
> Now regarding the "/nonexistent/gsyncd: No such file or directory" I have manually fixed it by editing the gsyncd_template.conf file on all nodes, I guess creating a symlink as you suggest would have also worked. Shouldn't this work out of the box btw??
>
Yes, It should work without any issues. I think these issues crop due to 
difference in environment.

Anyway , We will look into this.
> Another informational error message I have seen in the log file on my slave (/var/log/glusterfs/geo-replication-slaves) is the following:
>
> [2015-09-21 10:21:12.646161] I [dict.c:473:dict_get] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.4/xlator/system/posix-acl.so(posix_acl_setxattr_cbk+0x26) [0x7fa8a24e7166] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.4/xlator/system/posix-acl.so(handling_other_acl_related_xattr+0xb0) [0x7fa8a24e70f0] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get+0x93) [0x7fa8a96c9093] ) 0-dict: !this || key=system.posix_acl_default [Invalid argument]
>
> It's appearing every minute and wanted to check if this is a bug maybe? or how bad is it?
Is the synchronization happening properly? ( copy some files into master 
and verified the contents at Slave side?)
Please verify it.

Please share the complete log. This may be a possible issue with ZFS( 
again I am  only guessing)
>
> Finally I have set up everything like mentioned in the github documentation using the mountbroker and a separate user for replication but still when I run a "gluster volume geo-replication status" I see "root" under the "SLAVE USER". Is this normal???
>
> So to resume, I've got geo-replication setup but it's quite patchy and messy and does not run under my special replication user I wanted it to run under.
>
As per my knowledge, it should display with specific user which you have 
setup.
Please share complete command details and logs. (Also, review all your 
commands to check whether everything is setup as mentioned).
>
> On Monday, September 21, 2015 8:07 AM, Saravanakumar Arumugam <sarumuga at redhat.com> wrote:
> Replies inline.
>
> On 09/19/2015 03:37 PM, ML mail wrote:
>> So yes indeed I am using ZFS on Linux v.0.6.5 as filesystem behind Gluster. As operating system I use Debian 8.2 GNU/Linux.
>>
>>
>> I also followed that documentation you mention in order to enable POSIX acltype for example on my ZFS volume.
>>
>> I checked and on my two bricks as well as my slave I have the coreutils package with its stat util. I have read quite a few posts of people using ZFS with Gluster and this should not be a problem. Or is this maybe a new bug in GlusterFS?
>>
>> By checking the log file
>> /var/log/glusterfs/geo-replication/reptest/ssh%3A%2F%2Froot%40192.168.40.3%3Agluster%3A%2F%2F127.0.0.1%3Areptest.log I saw the following error message which might help to debug this issue:
>>
>> [2015-09-18 23:41:09.646944] E [resource(/data/reptest/brick):226:logerr] Popen: ssh> bash: /nonexistent/gsyncd: No such file or directory
>>
>> Does this ring any bells?
>>
> Do you meant to say , the earlier error "could not find (null) to
> getinode size for data"  is gone and now you are getting this error.
>
> check whether these steps helps you :
> http://irclog.perlgeek.de/gluster/2015-01-08#i_9903500
>
> Please share the complete log , if you still face any issues.
>
> Also, report back if it helps, so that we can fix it here.
>
>>
>> On Saturday, September 19, 2015 6:18 AM, Saravanakumar Arumugam <sarumuga at redhat.com> wrote:
>> Hi,
>>
>> The underlying filesystem which you use seems like ZFS.
>>
>> I don't have much idea about zfs. You may want to check this link:
>> http://www.gluster.org/community/documentation/index.php/GlusterOnZFS
>>
>> As far as the error is concerned, it is trying to use stat command to
>> get inode details.
>> (stat command is provided by coreutils, which is quite a basic package).
>>
>> Could you share your System details? Is it a Linux system ?
>>
>> ps:
>> XFS is the recommended and widely tested filesystem for glusterfs.
>>
>> Thanks,
>> Saravana
>>
>>
>>
>> On 09/19/2015 03:03 AM, ML mail wrote:
>>> Hello,
>>>
>>> I am trying in vain to setup geo-replication on now version 3.7.4 of GlusterFS but it still does not seem to work. I have at least managed to run succesfully the georepsetup using the following command:
>>>
>>>
>>> georepsetup reptest gfsgeo at gfs1geo reptest
>>>
>>> But as soon as I run:
>>>
>>>
>>> gluster volume geo-replication reptest gfs1geo::reptest start
>>>
>>> i see the following error messages every 2 minutes in /var/log/glusterfs/etc-glusterfs-glusterd.vol.log:
>>>
>>> [2015-09-18 21:27:26.341524] I [MSGID: 106488] [glusterd-handler.c:1463:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req
>>> [2015-09-18 21:27:26.474240] I [MSGID: 106499] [glusterd-handler.c:4258:__glusterd_handle_status_volume] 0-management: Received status volume req for volume reptest
>>> [2015-09-18 21:27:26.475231] E [MSGID: 106419] [glusterd-utils.c:4972:glusterd_add_inode_size_to_dict] 0-management: could not find (null) to getinode size for data/reptest (zfs): (null) package missing?
>>>
>>>
>>> and nothing really happens.
>>>
>>> Does anyone have an idea what's wrong now?
>>>
>>> Regards
>>> ML
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-users