[Gluster-users] geo-replication 3.6.7 - no transition from hybrid to changelog crawl
Milind Changire
milindchangire at gmail.com
Tue Jan 19 09:56:46 UTC 2016
Hi Dietmar,
After discussion with Aravinda we realized that unfortunately the
suggestion to:
setfattr -n glusterfs.geo-rep.trigger-sync -v "1" <DIR>
setfattr -n glusterfs.geo-rep.trigger-sync -v "1" <file-path>
won't work with 3.6.7, since provision for that workaround was added after
3.6.7.
There's an alternative way to achieve the geo-replication:
1. stop geo-replication
2. delete files and directories with conflicting gfid on SLAVE
3. use the "touch" command to touch files and directories with conflicting
gfid
on MASTER
4. start geo-replication
This *should* get things correctly replicated to SLAVE.
Geo-replication should start with hybrid-crawl and trigger the replication
to SLAVE.
If not, then there's more to look at.
You could then send us output of the strace command for the gsyncd process,
while
geo-replication is running:
# strace -ff -p <gsyncd-pid> -o gsyncd-strace
You could terminate strace after about one minute and send us all the
gsyncd-strace.<pid>
files which will help us debug the issue if its not resolved by the
alternative
mechanism mentioned above.
Also, crawl status Hybrid Crawl is not an entirely bad thing. It just could
mean
that there's a lot of entries that are being processed. However, if things
don't
return back to the normal state after trying out the alternative
suggestion, we
could take a look at the strace output and get some clues.
--
Milind
On Thu, Jan 14, 2016 at 11:07 PM, Dietmar Putz <putz at 3qmedien.net> wrote:
> Hello all,
>
> after some days of inactivity i started another attempt to solve this
> geo-replication issue...step by step.
> it looks like that some of the directories on the slave volume does not
> have the same gfid like the corresponding directory on the master volume.
>
> for example :
>
> on a master-node i can see a lot of 'errcode: 23' lines like :
> [2016-01-14 09:58:36.96585] W [master(/gluster-export):301:regjob]
> _GMaster: Rsync: .gfid/a8d0387d-c5ad-4eeb-9fc6-637fb8299a50 [errcode: 23]
>
> on the corresponding slave the corresponding message :
> [2016-01-14 09:57:06.070452] W [fuse-bridge.c:1967:fuse_create_cbk]
> 0-glusterfs-fuse: 1185648: /.gfid/a8d0387d-c5ad-4eeb-9fc6-637fb8299a50 =>
> -1 (Operation not permitted)
>
> This is the file on the master, the file is still not replicated to the
> slave.
>
> 120533444364 97332 -rw-r--r-- 2 2001 2001 99662854 Jan 8 13:40
> /gluster-export/3912/uploads/BSZ-2015/Z_002895D0-C832-4698-84E6-89F34CDEC2AE_20144555_ST_1.mp4
> 120533444364 97332 -rw-r--r-- 2 2001 2001 99662854 Jan 8 13:40
> /gluster-export/.glusterfs/a8/d0/a8d0387d-c5ad-4eeb-9fc6-637fb8299a50
>
> The directory on the slave already contain some files, all of them are not
> available on the master anymore, obviously deleted in meantime on the
> master by a client.
> i have deleted and recreated this file on the master and observed the logs
> for recurrence of the newly created gfid of this file...same as before.
>
> in http://comments.gmane.org/gmane.comp.file-systems.gluster.user/20703 a
> user reports a geo-replication problem which is possibly caused by
> different gfid's of underlying directories.
> and yes, the directory of this file-example above shows that the gfid of
> the underlying directory differs from the gfid on the master while the most
> other directories have the same gfid.
>
> master :
> ...
> # file: gluster-export/3912/uploads/BSP-2012
> trusted.gfid=0x8f1d480351bb455b9adde190f2c2b350
> --------------
> # file: gluster-export/3912/uploads/BSZ-2003
> trusted.gfid=0xe80adc088e604234b778997d8e8c2018
> --------------
> # file: gluster-export/3912/uploads/BSZ-2004
> trusted.gfid=0xfe417dd16bbe4ae4a6a1936cfee7aced
> --------------
> # file: gluster-export/3912/uploads/BSZ-2010
> trusted.gfid=0x8044e436407d4ed3a67c81df8a7ad47f ###
> --------------
> # file: gluster-export/3912/uploads/BSZ-2015
> trusted.gfid=0x0c30f50480204e02b65d4716a048b029 ###
>
> slave :
> ...
> # file: gluster-export/3912/uploads/BSP-2012
> trusted.gfid=0x8f1d480351bb455b9adde190f2c2b350
> --------------
> # file: gluster-export/3912/uploads/BSZ-2003
> trusted.gfid=0xe80adc088e604234b778997d8e8c2018
> --------------
> # file: gluster-export/3912/uploads/BSZ-2004
> trusted.gfid=0xfe417dd16bbe4ae4a6a1936cfee7aced
> --------------
> # file: gluster-export/3912/uploads/BSZ-2010
> trusted.gfid=0xd83e8fb568c74e33a2091c547512a6ce ###
> --------------
> # file: gluster-export/3912/uploads/BSZ-2015
> trusted.gfid=0xa406e1bec7f3454d8f2ce9c5f9c70eb3 ###
>
>
> now the question...how to fix this..?
> in the thread above Aravinda wrote :
>
> ...
> To fix the issue,
> -----------------
> Find the parent directory of "main.mdb",
> Get the GFID of that directory, using getfattr
> Check the GFID of the same directory in Slave(To confirm GFIDs are
> different)
> To fix the issue, Delete that directory in Slave.
> Set virtual xattr for that directory and all the files inside that
> directory.
> setfattr -n glusterfs.geo-rep.trigger-sync -v "1" <DIR>
> setfattr -n glusterfs.geo-rep.trigger-sync -v "1" <file-path>
>
> Geo-rep will recreate the directory with Proper GFID and starts sync.
>
> deletion of the affected slave directory might be helpful...
> but do i have to execute above shown setfattr commands on the master or do
> they just speed up synchronization ?
> usually sync should start automatically or could there be a problem
> because crawl status is still in 'hybrid crawl'...?
>
> thanks in advance...
> best regards
> dietmar
>
>
>
>
>
> On 04.01.2016 12:08, Dietmar Putz wrote:
>
>> Hello Aravinda,
>>
>> thank you for your reply.
>> i just made a 'find /gluster-export -type f -exec ls -lisa {} \; >
>> ls-lisa-gluster-export-`hostname`.out' on each brick and checked the output
>> for files with less than 2 link counts.
>> i found nothing...all files on each brick have exact 2 links.
>>
>> the entire output for all bricks contain more than 7 million lines
>> including .glusterfs but without non relevant directories and files..
>> tron at dp-server:~/geo_rep_3$ cat ls-lisa-gluster-wien-0* | egrep -v
>> 'indices|landfill|changelogs|health_check' | wc -l
>> 7007316
>>
>> link count is on $4 :
>> tron at dp-server:~/geo_rep_3$ cat ls-lisa-gluster-wien-0* | egrep -v
>> 'indices|landfill|changelogs|health_check' | awk '{if($4=="2")print}' |
>> tail -1
>> 62648153697 4 -rw-rw-rw- 2 root root 1713 Jan 4 01:44
>> /gluster-export/3500/files/16/01/387233/3500-6dqMmBcVby97PQtR.ism
>>
>> tron at dp-server:~/geo_rep_3$ cat ls-lisa-gluster-wien-0* | egrep -v
>> 'indices|landfill|changelogs|health_check' | awk '{if($4=="1")print}'
>> tron at dp-server:~/geo_rep_3$
>> tron at dp-server:~/geo_rep_3$ cat ls-lisa-gluster-wien-0* | egrep -v
>> 'indices|landfill|changelogs|health_check' | awk '{if($4!="2")print}'
>> tron at dp-server:~/geo_rep_3$
>> tron at dp-server:~/geo_rep_3$ cat ls-lisa-gluster-wien-0* | egrep -v
>> 'indices|landfill|changelogs|health_check' | awk '{print $4}' | sort | uniq
>> -c
>> 7007316 2
>> tron at dp-server:~/geo_rep_3$
>>
>> If i understood you right this can not be the reason for the problem.
>> is there any other hint which i can check on the master or slave to
>> analyse the problem....?
>>
>> Any help would be very appreciated
>> best regards
>> dietmar
>>
>>
>>
>> Am 04.01.2016 um 07:14 schrieb Aravinda:
>>
>>> Hi,
>>>
>>> Looks like issue with Geo-rep due to race between Create and Rename.
>>> Geo-replication uses gfid-access (Mount Volume with aux-gfid-mount) to
>>> create and rename files. If Create and Rename replayed more than once then
>>> Geo-rep creates a two files with same GFID(not hardlink). This causes one
>>> file without backend GFID link.
>>>
>>> Milind is working on the patch to disallow the creation of second file
>>> with same GFID.
>>> @Milind, Please provide more update about your patch.
>>>
>>> As a workaround, identify all the files in Slave volume which do not
>>> have backend links and delete those files(Only in Slaves, keep backup if
>>> required)
>>>
>>> In Brick backend, Crawl and look for files with link count less than 2.
>>> (Exclude .glusterfs and .trashcan directory)
>>>
>>> regards
>>> Aravinda
>>>
>>> On 01/02/2016 09:56 PM, Dietmar Putz wrote:
>>>
>>>> Hello all,
>>>>
>>>> one more time i need some help with a geo-replication problem.
>>>> recently i started a new geo-replication. the master volume contains
>>>> about 45 TB data and the slave volume was new created before
>>>> geo-replication setup was done.
>>>> master and slave is a 6 node distributed replicated volume running
>>>> glusterfs-server 3.6.7-ubuntu1~trusty1.
>>>> geo-rep was starting without problems. since few days the slave volume
>>>> contains about 200 GB more data than the master volume and i expected that
>>>> the crawl status changes from 'hybrid crawl' to 'changelog crawl' but it
>>>> remains in 'hybrid crawl'.
>>>> the 'status detail' output far below shows more than 10 million synced
>>>> files while the entire master volume contains just about 2 million files.
>>>> some tests show that files are not deleted on the slave volume.
>>>> as far as i know the hybrid crawl has the limitation of not replicating
>>>> deletes and renames to the slave thus the geo-rep needs to achieve the
>>>> 'changelog crawl' status after initial sync...
>>>> usually this should happen more or less automatically, is this right ?
>>>>
>>>> the geo-rep frequently fails with below shown "OSError: [Errno 16]
>>>> Device or resource busy", this error appears about every 3-4 hours on each
>>>> active master node.
>>>> i guess the frequent appearance of this error prevent geo-rep from
>>>> changing to 'changelog crawl', does somebody experienced such problem, is
>>>> this the cause of the problem ?
>>>>
>>>> i found some similar reports on gluster.org for gfs 3.5, 3.6 and 3.7
>>>> but none of them point me to a solution...
>>>> does anybody know a solution or is there a workaround to achieve the
>>>> changelog crawl status...?
>>>>
>>>> Any help would be very appreciated
>>>> best regards
>>>> dietmar
>>>>
>>>>
>>>>
>>>>
>>>> Master gluster-ger-ber-07:
>>>> -----------------------------
>>>>
>>>> [2016-01-02 11:39:48.122546] I [master(/gluster-export):1343:crawl]
>>>> _GMaster: processing xsync changelog
>>>> /var/lib/misc/glusterfsd/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01/9d7139ecf10a6fc33a
>>>> 6e41d8d6e56984/xsync/XSYNC-CHANGELOG.1451724692
>>>> [2016-01-02 11:42:55.182342] I [master(/gluster-export):1343:crawl]
>>>> _GMaster: processing xsync changelog
>>>> /var/lib/misc/glusterfsd/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01/9d7139ecf10a6fc33a
>>>> 6e41d8d6e56984/xsync/XSYNC-CHANGELOG.1451724751
>>>> [2016-01-02 11:44:11.168962] I [master(/gluster-export):1340:crawl]
>>>> _GMaster: finished hybrid crawl syncing, stime: (-1, 0)
>>>> [2016-01-02 11:44:11.246845] I [master(/gluster-export):490:crawlwrap]
>>>> _GMaster: primary master with volume id
>>>> 6a071cfa-b150-4f0b-b1ed-96ab5d4bd671 ...
>>>> [2016-01-02 11:44:11.265209] I [master(/gluster-export):501:crawlwrap]
>>>> _GMaster: crawl interval: 3 seconds
>>>> [2016-01-02 11:44:11.896940] I [master(/gluster-export):1192:crawl]
>>>> _GMaster: slave's time: (-1, 0)
>>>> [2016-01-02 11:44:12.171761] E [repce(/gluster-export):207:__call__]
>>>> RepceClient: call 18897:139899553576768:1451735052.09 (entry_ops) failed on
>>>> peer with OSError
>>>> [2016-01-02 11:44:12.172101] E
>>>> [syncdutils(/gluster-export):270:log_raise_exception] <top>: FAIL:
>>>> Traceback (most recent call last):
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py", line
>>>> 164, in main
>>>> main_i()
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py", line
>>>> 643, in main_i
>>>> local.service_loop(*[r for r in [remote] if r])
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/resource.py", line
>>>> 1344, in service_loop
>>>> g2.crawlwrap()
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/master.py", line
>>>> 539, in crawlwrap
>>>> self.crawl(no_stime_update=no_stime_update)
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/master.py", line
>>>> 1204, in crawl
>>>> self.process(changes)
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/master.py", line
>>>> 956, in process
>>>> self.process_change(change, done, retry)
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/master.py", line
>>>> 920, in process_change
>>>> self.slave.server.entry_ops(entries)
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/repce.py", line 226,
>>>> in __call__
>>>> return self.ins(self.meth, *a)
>>>> File
>>>> "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/repce.py", line 208,
>>>> in __call__
>>>> raise res
>>>> OSError: [Errno 16] Device or resource busy
>>>> [2016-01-02 11:44:12.258982] I
>>>> [syncdutils(/gluster-export):214:finalize] <top>: exiting.
>>>> [2016-01-02 11:44:12.321808] I [repce(agent):92:service_loop]
>>>> RepceServer: terminating on reaching EOF.
>>>> [2016-01-02 11:44:12.349766] I [syncdutils(agent):214:finalize] <top>:
>>>> exiting.
>>>> [2016-01-02 11:44:12.435992] I [monitor(monitor):141:set_state]
>>>> Monitor: new state: faulty
>>>> [2016-01-02 11:44:23.164284] I [monitor(monitor):215:monitor] Monitor:
>>>> ------------------------------------------------------------
>>>> [2016-01-02 11:44:23.169981] I [monitor(monitor):216:monitor] Monitor:
>>>> starting gsyncd worker
>>>> [2016-01-02 11:44:23.216662] I [changelogagent(agent):72:__init__]
>>>> ChangelogAgent: Agent listining...
>>>> [2016-01-02 11:44:23.239778] I [gsyncd(/gluster-export):633:main_i]
>>>> <top>: syncing: gluster://localhost:ger-ber-01 ->
>>>> ssh://root@gluster-wien-07-int:gluster://localhost:aut-wien-vol-01
>>>> [2016-01-02 11:44:26.358613] I
>>>> [master(/gluster-export):75:gmaster_builder] <top>: setting up xsync change
>>>> detection mode
>>>> [2016-01-02 11:44:26.358983] I [master(/gluster-export):413:__init__]
>>>> _GMaster: using 'rsync' as the sync engine
>>>> [2016-01-02 11:44:26.359985] I
>>>> [master(/gluster-export):75:gmaster_builder] <top>: setting up changelog
>>>> change detection mode
>>>> [2016-01-02 11:44:26.360243] I [master(/gluster-export):413:__init__]
>>>> _GMaster: using 'rsync' as the sync engine
>>>> [2016-01-02 11:44:26.361159] I
>>>> [master(/gluster-export):75:gmaster_builder] <top>: setting up
>>>> changeloghistory change detection mode
>>>> [2016-01-02 11:44:26.361377] I [master(/gluster-export):413:__init__]
>>>> _GMaster: using 'rsync' as the sync engine
>>>> [2016-01-02 11:44:26.402601] I [master(/gluster-export):1311:register]
>>>> _GMaster: xsync temp directory:
>>>> /var/lib/misc/glusterfsd/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01/9d7139ecf10a6fc33a6e
>>>> 41d8d6e56984/xsync
>>>> [2016-01-02 11:44:26.402848] I
>>>> [resource(/gluster-export):1318:service_loop] GLUSTER: Register time:
>>>> 1451735066
>>>> [2016-01-02 11:44:27.26012] I [master(/gluster-export):490:crawlwrap]
>>>> _GMaster: primary master with volume id
>>>> 6a071cfa-b150-4f0b-b1ed-96ab5d4bd671 ...
>>>> [2016-01-02 11:44:27.31605] I [master(/gluster-export):501:crawlwrap]
>>>> _GMaster: crawl interval: 1 seconds
>>>> [2016-01-02 11:44:27.66868] I [master(/gluster-export):1226:crawl]
>>>> _GMaster: starting history crawl... turns: 1, stime: (-1, 0)
>>>> [2016-01-02 11:44:27.67043] I [master(/gluster-export):1229:crawl]
>>>> _GMaster: stime not available, abandoning history crawl
>>>> [2016-01-02 11:44:27.112426] I [master(/gluster-export):490:crawlwrap]
>>>> _GMaster: primary master with volume id
>>>> 6a071cfa-b150-4f0b-b1ed-96ab5d4bd671 ...
>>>> [2016-01-02 11:44:27.117506] I [master(/gluster-export):501:crawlwrap]
>>>> _GMaster: crawl interval: 60 seconds
>>>> [2016-01-02 11:44:27.140610] I [master(/gluster-export):1333:crawl]
>>>> _GMaster: starting hybrid crawl..., stime: (-1, 0)
>>>> [2016-01-02 11:45:23.417233] I [monitor(monitor):141:set_state]
>>>> Monitor: new state: Stable
>>>> [2016-01-02 11:45:48.225915] I [master(/gluster-export):1343:crawl]
>>>> _GMaster: processing xsync changelog
>>>> /var/lib/misc/glusterfsd/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01/9d7139ecf10a6fc33a
>>>> 6e41d8d6e56984/xsync/XSYNC-CHANGELOG.1451735067
>>>> [2016-01-02 11:47:08.65231] I [master(/gluster-export):1343:crawl]
>>>> _GMaster: processing xsync changelog
>>>> /var/lib/misc/glusterfsd/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01/9d7139ecf10a6fc33a6
>>>> e41d8d6e56984/xsync/XSYNC-CHANGELOG.1451735148
>>>> ...
>>>>
>>>>
>>>> slave gluster-wien-07 :
>>>> ------------------------
>>>>
>>>> [2016-01-02 11:44:12.007744] W [fuse-bridge.c:1261:fuse_err_cbk]
>>>> 0-glusterfs-fuse: 1959820: SETXATTR()
>>>> /.gfid/5e436e5b-086b-4720-9e70-0e49c8e09698 => -1 (File exists)
>>>> [2016-01-02 11:44:12.010970] W
>>>> [client-rpc-fops.c:240:client3_3_mknod_cbk] 0-aut-wien-vol-01-client-5:
>>>> remote operation failed: File exists. Path:
>>>> <gfid:666bceac-7c14-4efd-81fe-8185458fcf1f>/11-kxyrM3NgdtBWPFv4.webm
>>>> [2016-01-02 11:44:12.011327] W
>>>> [client-rpc-fops.c:240:client3_3_mknod_cbk] 0-aut-wien-vol-01-client-4:
>>>> remote operation failed: File exists. Path:
>>>> <gfid:666bceac-7c14-4efd-81fe-8185458fcf1f>/11-kxyrM3NgdtBWPFv4.webm
>>>> [2016-01-02 11:44:12.012054] W [fuse-bridge.c:1261:fuse_err_cbk]
>>>> 0-glusterfs-fuse: 1959822: SETXATTR()
>>>> /.gfid/666bceac-7c14-4efd-81fe-8185458fcf1f => -1 (File exists)
>>>> [2016-01-02 11:44:12.024743] W
>>>> [client-rpc-fops.c:240:client3_3_mknod_cbk] 0-aut-wien-vol-01-client-5:
>>>> remote operation failed: File exists. Path:
>>>> <gfid:5bfd6f99-07e8-4b2f-844b-aa0b6535c055>/Gf4FYbpDTC7yK2mv.png
>>>> [2016-01-02 11:44:12.024970] W
>>>> [client-rpc-fops.c:240:client3_3_mknod_cbk] 0-aut-wien-vol-01-client-4:
>>>> remote operation failed: File exists. Path:
>>>> <gfid:5bfd6f99-07e8-4b2f-844b-aa0b6535c055>/Gf4FYbpDTC7yK2mv.png
>>>> [2016-01-02 11:44:12.025601] W [fuse-bridge.c:1261:fuse_err_cbk]
>>>> 0-glusterfs-fuse: 1959823: SETXATTR()
>>>> /.gfid/5bfd6f99-07e8-4b2f-844b-aa0b6535c055 => -1 (File exists)
>>>> [2016-01-02 11:44:12.100688] I
>>>> [dht-selfheal.c:1065:dht_selfheal_layout_new_directory]
>>>> 0-aut-wien-vol-01-dht: chunk size = 0xffffffff / 57217563 = 0x4b
>>>> [2016-01-02 11:44:12.100765] I
>>>> [dht-selfheal.c:1103:dht_selfheal_layout_new_directory]
>>>> 0-aut-wien-vol-01-dht: assigning range size 0x5542c4a3 to
>>>> aut-wien-vol-01-replicate-0
>>>> [2016-01-02 11:44:12.100785] I
>>>> [dht-selfheal.c:1103:dht_selfheal_layout_new_directory]
>>>> 0-aut-wien-vol-01-dht: assigning range size 0x5542c4a3 to
>>>> aut-wien-vol-01-replicate-1
>>>> [2016-01-02 11:44:12.100800] I
>>>> [dht-selfheal.c:1103:dht_selfheal_layout_new_directory]
>>>> 0-aut-wien-vol-01-dht: assigning range size 0x5542c4a3 to
>>>> aut-wien-vol-01-replicate-2
>>>> [2016-01-02 11:44:12.100839] I [MSGID: 109036]
>>>> [dht-common.c:6296:dht_log_new_layout_for_dir_selfheal]
>>>> 0-aut-wien-vol-01-dht: Setting layout of
>>>> <gfid:d4815ee4-3348-4105-9136-d0219d956ed8>/.dstXXX0HUpRD with
>>>> [Subvol_name: aut-wien-vol-01-re
>>>> plicate-0, Err: -1 , Start: 0 , Stop: 1430439074 ], [Subvol_name:
>>>> aut-wien-vol-01-replicate-1, Err: -1 , Start: 1430439075 , Stop: 2860878149
>>>> ], [Subvol_name: aut-wien-vol-01-replicate-2, Err: -1 , Start: 2860878150 ,
>>>> Stop: 4294967295 ],
>>>> [2016-01-02 11:44:12.114192] W
>>>> [client-rpc-fops.c:306:client3_3_mkdir_cbk] 0-aut-wien-vol-01-client-2:
>>>> remote operation failed: File exists. Path:
>>>> <gfid:cd3fd9ba-34b8-4c6b-ba72-4796b80b0ff2>/.dstXXb70G3x
>>>> [2016-01-02 11:44:12.114275] W
>>>> [client-rpc-fops.c:306:client3_3_mkdir_cbk] 0-aut-wien-vol-01-client-3:
>>>> remote operation failed: File exists. Path:
>>>> <gfid:cd3fd9ba-34b8-4c6b-ba72-4796b80b0ff2>/.dstXXb70G3x
>>>> [2016-01-02 11:44:12.114879] W [fuse-bridge.c:1261:fuse_err_cbk]
>>>> 0-glusterfs-fuse: 1959831: SETXATTR()
>>>> /.gfid/cd3fd9ba-34b8-4c6b-ba72-4796b80b0ff2 => -1 (File exists)
>>>> [2016-01-02 11:44:12.118473] I [dht-layout.c:663:dht_layout_normalize]
>>>> 0-aut-wien-vol-01-dht: Found anomalies in
>>>> /.gfid/cd3fd9ba-34b8-4c6b-ba72-4796b80b0ff2/.dstXXb70G3x (gfid =
>>>> 00000000-0000-0000-0000-000000000000). Holes=1 overlaps=0
>>>> [2016-01-02 11:44:12.118537] I
>>>> [dht-selfheal.c:1065:dht_selfheal_layout_new_directory]
>>>> 0-aut-wien-vol-01-dht: chunk size = 0xffffffff / 57217563 = 0x4b
>>>> [2016-01-02 11:44:12.118562] I
>>>> [dht-selfheal.c:1103:dht_selfheal_layout_new_directory]
>>>> 0-aut-wien-vol-01-dht: assigning range size 0x5542c4a3 to
>>>> aut-wien-vol-01-replicate-2
>>>> [2016-01-02 11:44:12.118579] I
>>>> [dht-selfheal.c:1103:dht_selfheal_layout_new_directory]
>>>> 0-aut-wien-vol-01-dht: assigning range size 0x5542c4a3 to
>>>> aut-wien-vol-01-replicate-0
>>>> [2016-01-02 11:44:12.118613] I
>>>> [dht-selfheal.c:1103:dht_selfheal_layout_new_directory]
>>>> 0-aut-wien-vol-01-dht: assigning range size 0x5542c4a3 to
>>>> aut-wien-vol-01-replicate-1
>>>> [2016-01-02 11:44:12.120352] I [MSGID: 109036]
>>>> [dht-common.c:6296:dht_log_new_layout_for_dir_selfheal]
>>>> 0-aut-wien-vol-01-dht: Setting layout of
>>>> /.gfid/cd3fd9ba-34b8-4c6b-ba72-4796b80b0ff2/.dstXXb70G3x with [Subvol_name:
>>>> aut-wien-vol-01-rep
>>>> licate-0, Err: -1 , Start: 1430439075 , Stop: 2860878149 ],
>>>> [Subvol_name: aut-wien-vol-01-replicate-1, Err: -1 , Start: 2860878150 ,
>>>> Stop: 4294967295 ], [Subvol_name: aut-wien-vol-01-replicate-2, Err: -1 ,
>>>> Start: 0 , Stop: 1430439074 ],
>>>> [2016-01-02 11:44:12.630949] I [fuse-bridge.c:4927:fuse_thread_proc]
>>>> 0-fuse: unmounting /tmp/gsyncd-aux-mount-tOUOsz
>>>> [2016-01-02 11:44:12.633952] W [glusterfsd.c:1211:cleanup_and_exit]
>>>> (--> 0-: received signum (15), shutting down
>>>> [2016-01-02 11:44:12.633964] I [fuse-bridge.c:5607:fini] 0-fuse:
>>>> Unmounting '/tmp/gsyncd-aux-mount-tOUOsz'.
>>>> [2016-01-02 11:44:23.946702] I [MSGID: 100030] [glusterfsd.c:2035:main]
>>>> 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.6.7
>>>> (args: /usr/sbin/glusterfs --aux-gfid-mount
>>>> --log-file=/var/log/glusterfs/geo-replication-slav
>>>> es/6a071cfa-b150-4f0b-b1ed-96ab5d4bd671:gluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01.gluster.log
>>>> --volfile-server=localhost --volfile-id=aut-wien-vol-01 --client-pid=-1
>>>> /tmp/gsyncd-aux-mount-otU3wS)
>>>> [2016-01-02 11:44:24.042128] I [dht-shared.c:337:dht_init_regex]
>>>> 0-aut-wien-vol-01-dht: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$
>>>> [2016-01-02 11:44:24.046315] I [client.c:2268:notify]
>>>> 0-aut-wien-vol-01-client-0: parent translators are ready, attempting
>>>> connect on transport
>>>> [2016-01-02 11:44:24.046532] I [client.c:2268:notify]
>>>> 0-aut-wien-vol-01-client-1: parent translators are ready, attempting
>>>> connect on transport
>>>> [2016-01-02 11:44:24.046664] I [client.c:2268:notify]
>>>> 0-aut-wien-vol-01-client-2: parent translators are ready, attempting
>>>> connect on transport
>>>> [2016-01-02 11:44:24.046806] I [client.c:2268:notify]
>>>> 0-aut-wien-vol-01-client-3: parent translators are ready, attempting
>>>> connect on transport
>>>> [2016-01-02 11:44:24.046940] I [client.c:2268:notify]
>>>> 0-aut-wien-vol-01-client-4: parent translators are ready, attempting
>>>> connect on transport
>>>> [2016-01-02 11:44:24.047070] I [client.c:2268:notify]
>>>> 0-aut-wien-vol-01-client-5: parent translators are ready, attempting
>>>> connect on transport
>>>> Final graph:
>>>> +------------------------------------------------------------------------------+
>>>>
>>>> 1: volume aut-wien-vol-01-client-0
>>>> 2: type protocol/client
>>>> 3: option ping-timeout 10
>>>> 4: option remote-host gluster-wien-02-int
>>>> 5: option remote-subvolume /gluster-export
>>>> 6: option transport-type socket
>>>> 7: option username 6b3d1fae-fa3e-4305-a4a0-fd27c7ac9929
>>>> 8: option password 8777e154-476c-449a-89b2-3199872e4a1f
>>>> 9: option send-gids true
>>>> 10: end-volume
>>>> 11:
>>>> 12: volume aut-wien-vol-01-client-1
>>>> 13: type protocol/client
>>>> 14: option ping-timeout 10
>>>> 15: option remote-host gluster-wien-03-int
>>>> 16: option remote-subvolume /gluster-export
>>>> 17: option transport-type socket
>>>> 18: option username 6b3d1fae-fa3e-4305-a4a0-fd27c7ac9929
>>>> 19: option password 8777e154-476c-449a-89b2-3199872e4a1f
>>>> 20: option send-gids true
>>>> 21: end-volume
>>>> 22:
>>>> 23: volume aut-wien-vol-01-replicate-0
>>>> 24: type cluster/replicate
>>>> 25: subvolumes aut-wien-vol-01-client-0 aut-wien-vol-01-client-1
>>>> 26: end-volume
>>>> 27:
>>>> 28: volume aut-wien-vol-01-client-2
>>>> 29: type protocol/client
>>>> 30: option ping-timeout 10
>>>> 31: option remote-host gluster-wien-04-int
>>>> 32: option remote-subvolume /gluster-export
>>>> 33: option transport-type socket
>>>> 34: option username 6b3d1fae-fa3e-4305-a4a0-fd27c7ac9929
>>>> 35: option password 8777e154-476c-449a-89b2-3199872e4a1f
>>>> 36: option send-gids true
>>>> 37: end-volume
>>>> 38:
>>>> 39: volume aut-wien-vol-01-client-3
>>>> 40: type protocol/client
>>>> 41: option ping-timeout 10
>>>> 42: option remote-host gluster-wien-05-int
>>>> 43: option remote-subvolume /gluster-export
>>>> 44: option transport-type socket
>>>> 45: option username 6b3d1fae-fa3e-4305-a4a0-fd27c7ac9929
>>>> 46: option password 8777e154-476c-449a-89b2-3199872e4a1f
>>>> 47: option send-gids true
>>>> 48: end-volume
>>>> 49:
>>>> 50: volume aut-wien-vol-01-replicate-1
>>>> 51: type cluster/replicate
>>>> 52: subvolumes aut-wien-vol-01-client-2 aut-wien-vol-01-client-3
>>>> 53: end-volume
>>>> 54:
>>>> 55: volume aut-wien-vol-01-client-4
>>>> 56: type protocol/client
>>>> 57: option ping-timeout 10
>>>> 58: option remote-host gluster-wien-06-int
>>>> 59: option remote-subvolume /gluster-export
>>>> 60: option transport-type socket
>>>> 61: option username 6b3d1fae-fa3e-4305-a4a0-fd27c7ac9929
>>>> 62: option password 8777e154-476c-449a-89b2-3199872e4a1f
>>>> 63: option send-gids true
>>>> 64: end-volume
>>>> 65:
>>>> 66: volume aut-wien-vol-01-client-5
>>>> 67: type protocol/client
>>>> 68: option ping-timeout 10
>>>> 69: option remote-host gluster-wien-07-int
>>>> 70: option remote-subvolume /gluster-export
>>>> 71: option transport-type socket
>>>> 72: option username 6b3d1fae-fa3e-4305-a4a0-fd27c7ac9929
>>>> 73: option password 8777e154-476c-449a-89b2-3199872e4a1f
>>>> 74: option send-gids true
>>>> 75: end-volume
>>>> 76:
>>>> 77: volume aut-wien-vol-01-replicate-2
>>>> 78: type cluster/replicate
>>>> 79: subvolumes aut-wien-vol-01-client-4 aut-wien-vol-01-client-5
>>>> 80: end-volume
>>>> 81:
>>>> 82: volume aut-wien-vol-01-dht
>>>> 83: type cluster/distribute
>>>> 84: subvolumes aut-wien-vol-01-replicate-0
>>>> aut-wien-vol-01-replicate-1 aut-wien-vol-01-replicate-2
>>>> 85: end-volume
>>>> 86:
>>>> 87: volume aut-wien-vol-01-write-behind
>>>> 88: type performance/write-behind
>>>> 89: subvolumes aut-wien-vol-01-dht
>>>> 90: end-volume
>>>> 91:
>>>> 92: volume aut-wien-vol-01-read-ahead
>>>> 93: type performance/read-ahead
>>>> 94: subvolumes aut-wien-vol-01-write-behind
>>>> 95: end-volume
>>>> 96:
>>>> 97: volume aut-wien-vol-01-io-cache
>>>> 98: type performance/io-cache
>>>> 99: option min-file-size 0
>>>> 100: option cache-timeout 2
>>>> 101: option cache-size 1024MB
>>>> 102: subvolumes aut-wien-vol-01-read-ahead
>>>> 103: end-volume
>>>> 104:
>>>> 105: volume aut-wien-vol-01-quick-read
>>>> 106: type performance/quick-read
>>>> 107: option cache-size 1024MB
>>>> 108: subvolumes aut-wien-vol-01-io-cache
>>>> 109: end-volume
>>>> 110:
>>>> 111: volume aut-wien-vol-01-open-behind
>>>> 112: type performance/open-behind
>>>> 113: subvolumes aut-wien-vol-01-quick-read
>>>> 114: end-volume
>>>> 115:
>>>> 116: volume aut-wien-vol-01-md-cache
>>>> 117: type performance/md-cache
>>>> 118: subvolumes aut-wien-vol-01-open-behind
>>>> 119: end-volume
>>>> 120:
>>>> 121: volume aut-wien-vol-01
>>>> 122: type debug/io-stats
>>>> 123: option latency-measurement off
>>>> 124: option count-fop-hits off
>>>> 125: subvolumes aut-wien-vol-01-md-cache
>>>> 126: end-volume
>>>> 127:
>>>> 128: volume gfid-access-autoload
>>>> 129: type features/gfid-access
>>>> 130: subvolumes aut-wien-vol-01
>>>> 131: end-volume
>>>> 132:
>>>> 133: volume meta-autoload
>>>> 134: type meta
>>>> 135: subvolumes gfid-access-autoload
>>>> 136: end-volume
>>>> 137:
>>>> +------------------------------------------------------------------------------+
>>>>
>>>> [2016-01-02 11:44:24.047642] I [rpc-clnt.c:1761:rpc_clnt_reconfig]
>>>> 0-aut-wien-vol-01-client-5: changing port to 49153 (from 0)
>>>> [2016-01-02 11:44:24.047927] I
>>>> [client-handshake.c:1413:select_server_supported_programs]
>>>> 0-aut-wien-vol-01-client-5: Using Program GlusterFS 3.3, Num (1298437),
>>>> Version (330)
>>>> [2016-01-02 11:44:24.048044] I
>>>> [client-handshake.c:1200:client_setvolume_cbk] 0-aut-wien-vol-01-client-5:
>>>> Connected to aut-wien-vol-01-client-5, attached to remote volume
>>>> '/gluster-export'.
>>>> [2016-01-02 11:44:24.048050] I
>>>> [client-handshake.c:1210:client_setvolume_cbk] 0-aut-wien-vol-01-client-5:
>>>> Server and Client lk-version numbers are not same, reopening the fds
>>>> [2016-01-02 11:44:24.048088] I [MSGID: 108005]
>>>> [afr-common.c:3684:afr_notify] 0-aut-wien-vol-01-replicate-2: Subvolume
>>>> 'aut-wien-vol-01-client-5' came back up; going online.
>>>> [2016-01-02 11:44:24.048114] I
>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>> 0-aut-wien-vol-01-client-5: Server lk version = 1
>>>> [2016-01-02 11:44:24.048124] I [rpc-clnt.c:1761:rpc_clnt_reconfig]
>>>> 0-aut-wien-vol-01-client-0: changing port to 49153 (from 0)
>>>> [2016-01-02 11:44:24.048132] I [rpc-clnt.c:1761:rpc_clnt_reconfig]
>>>> 0-aut-wien-vol-01-client-1: changing port to 49153 (from 0)
>>>> [2016-01-02 11:44:24.048138] I [rpc-clnt.c:1761:rpc_clnt_reconfig]
>>>> 0-aut-wien-vol-01-client-2: changing port to 49153 (from 0)
>>>> [2016-01-02 11:44:24.048146] I [rpc-clnt.c:1761:rpc_clnt_reconfig]
>>>> 0-aut-wien-vol-01-client-3: changing port to 49153 (from 0)
>>>> [2016-01-02 11:44:24.048153] I [rpc-clnt.c:1761:rpc_clnt_reconfig]
>>>> 0-aut-wien-vol-01-client-4: changing port to 49153 (from 0)
>>>> [2016-01-02 11:44:24.049070] I
>>>> [client-handshake.c:1413:select_server_supported_programs]
>>>> 0-aut-wien-vol-01-client-0: Using Program GlusterFS 3.3, Num (1298437),
>>>> Version (330)
>>>> [2016-01-02 11:44:24.049094] I
>>>> [client-handshake.c:1413:select_server_supported_programs]
>>>> 0-aut-wien-vol-01-client-3: Using Program GlusterFS 3.3, Num (1298437),
>>>> Version (330)
>>>> [2016-01-02 11:44:24.049113] I
>>>> [client-handshake.c:1413:select_server_supported_programs]
>>>> 0-aut-wien-vol-01-client-2: Using Program GlusterFS 3.3, Num (1298437),
>>>> Version (330)
>>>> [2016-01-02 11:44:24.049131] I
>>>> [client-handshake.c:1413:select_server_supported_programs]
>>>> 0-aut-wien-vol-01-client-1: Using Program GlusterFS 3.3, Num (1298437),
>>>> Version (330)
>>>> [2016-01-02 11:44:24.049224] I
>>>> [client-handshake.c:1413:select_server_supported_programs]
>>>> 0-aut-wien-vol-01-client-4: Using Program GlusterFS 3.3, Num (1298437),
>>>> Version (330)
>>>> [2016-01-02 11:44:24.049307] I
>>>> [client-handshake.c:1200:client_setvolume_cbk] 0-aut-wien-vol-01-client-0:
>>>> Connected to aut-wien-vol-01-client-0, attached to remote volume
>>>> '/gluster-export'.
>>>> [2016-01-02 11:44:24.049312] I
>>>> [client-handshake.c:1210:client_setvolume_cbk] 0-aut-wien-vol-01-client-0:
>>>> Server and Client lk-version numbers are not same, reopening the fds
>>>> [2016-01-02 11:44:24.049324] I [MSGID: 108005]
>>>> [afr-common.c:3684:afr_notify] 0-aut-wien-vol-01-replicate-0: Subvolume
>>>> 'aut-wien-vol-01-client-0' came back up; going online.
>>>> [2016-01-02 11:44:24.049384] I
>>>> [client-handshake.c:1200:client_setvolume_cbk] 0-aut-wien-vol-01-client-3:
>>>> Connected to aut-wien-vol-01-client-3, attached to remote volume
>>>> '/gluster-export'.
>>>> [2016-01-02 11:44:24.049389] I
>>>> [client-handshake.c:1210:client_setvolume_cbk] 0-aut-wien-vol-01-client-3:
>>>> Server and Client lk-version numbers are not same, reopening the fds
>>>> [2016-01-02 11:44:24.049400] I [MSGID: 108005]
>>>> [afr-common.c:3684:afr_notify] 0-aut-wien-vol-01-replicate-1: Subvolume
>>>> 'aut-wien-vol-01-client-3' came back up; going online.
>>>> [2016-01-02 11:44:24.049418] I
>>>> [client-handshake.c:1200:client_setvolume_cbk] 0-aut-wien-vol-01-client-2:
>>>> Connected to aut-wien-vol-01-client-2, attached to remote volume
>>>> '/gluster-export'.
>>>> [2016-01-02 11:44:24.049422] I
>>>> [client-handshake.c:1210:client_setvolume_cbk] 0-aut-wien-vol-01-client-2:
>>>> Server and Client lk-version numbers are not same, reopening the fds
>>>> [2016-01-02 11:44:24.049460] I
>>>> [client-handshake.c:1200:client_setvolume_cbk] 0-aut-wien-vol-01-client-1:
>>>> Connected to aut-wien-vol-01-client-1, attached to remote volume
>>>> '/gluster-export'.
>>>> [2016-01-02 11:44:24.049465] I
>>>> [client-handshake.c:1210:client_setvolume_cbk] 0-aut-wien-vol-01-client-1:
>>>> Server and Client lk-version numbers are not same, reopening the fds
>>>> [2016-01-02 11:44:24.049493] I
>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>> 0-aut-wien-vol-01-client-0: Server lk version = 1
>>>> [2016-01-02 11:44:24.049567] I
>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>> 0-aut-wien-vol-01-client-3: Server lk version = 1
>>>> [2016-01-02 11:44:24.049632] I
>>>> [client-handshake.c:1200:client_setvolume_cbk] 0-aut-wien-vol-01-client-4:
>>>> Connected to aut-wien-vol-01-client-4, attached to remote volume
>>>> '/gluster-export'.
>>>> [2016-01-02 11:44:24.049638] I
>>>> [client-handshake.c:1210:client_setvolume_cbk] 0-aut-wien-vol-01-client-4:
>>>> Server and Client lk-version numbers are not same, reopening the fds
>>>> [2016-01-02 11:44:24.052103] I [fuse-bridge.c:5086:fuse_graph_setup]
>>>> 0-fuse: switched to graph 0
>>>> [2016-01-02 11:44:24.052150] I
>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>> 0-aut-wien-vol-01-client-2: Server lk version = 1
>>>> [2016-01-02 11:44:24.052163] I
>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>> 0-aut-wien-vol-01-client-4: Server lk version = 1
>>>> [2016-01-02 11:44:24.052192] I
>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>> 0-aut-wien-vol-01-client-1: Server lk version = 1
>>>> [2016-01-02 11:44:24.052204] I [fuse-bridge.c:4015:fuse_init]
>>>> 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel
>>>> 7.20
>>>> [2016-01-02 11:44:24.053991] I
>>>> [afr-common.c:1491:afr_local_discovery_cbk] 0-aut-wien-vol-01-replicate-2:
>>>> selecting local read_child aut-wien-vol-01-client-5
>>>> [2016-01-02 11:45:48.613563] W
>>>> [client-rpc-fops.c:306:client3_3_mkdir_cbk] 0-aut-wien-vol-01-client-5:
>>>> remote operation failed: File exists. Path: /keys
>>>> [2016-01-02 11:45:48.614131] W
>>>> [client-rpc-fops.c:306:client3_3_mkdir_cbk] 0-aut-wien-vol-01-client-4:
>>>> remote operation failed: File exists. Path: /keys
>>>> [2016-01-02 11:45:48.614436] W [fuse-bridge.c:1261:fuse_err_cbk]
>>>> 0-glusterfs-fuse: 12: SETXATTR()
>>>> /.gfid/00000000-0000-0000-0000-000000000001 => -1 (File exists)
>>>> ...
>>>>
>>>>
>>>> [ 13:41:40 ] - root at gluster-ger-ber-07
>>>> /var/log/glusterfs/geo-replication/ger-ber-01 $gluster volume
>>>> geo-replication ger-ber-01 gluster-wien-07::aut-wien-vol-01 status detail
>>>>
>>>> MASTER NODE MASTER VOL MASTER BRICK SLAVE
>>>> STATUS CHECKPOINT STATUS CRAWL STATUS FILES
>>>> SYNCD FILES PENDING BYTES PENDING DELETES PENDING FILES SKIPPED
>>>> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>
>>>> gluster-ger-ber-07 ger-ber-01 /gluster-export
>>>> gluster-wien-07-int::aut-wien-vol-01 Active N/A Hybrid
>>>> Crawl 10743644 8192 0 0 0
>>>> gluster-ger-ber-11 ger-ber-01 /gluster-export
>>>> gluster-wien-03-int::aut-wien-vol-01 Active N/A Hybrid
>>>> Crawl 16037091 8192 0 0 0
>>>> gluster-ger-ber-10 ger-ber-01 /gluster-export
>>>> gluster-wien-02-int::aut-wien-vol-01 Passive N/A N/A
>>>> 0 0 0 0 0
>>>> gluster-ger-ber-12 ger-ber-01 /gluster-export
>>>> gluster-wien-06-int::aut-wien-vol-01 Passive N/A N/A
>>>> 0 0 0 0 0
>>>> gluster-ger-ber-09 ger-ber-01 /gluster-export
>>>> gluster-wien-05-int::aut-wien-vol-01 Active N/A Hybrid
>>>> Crawl 16180514 8192 0 0 0
>>>> gluster-ger-ber-08 ger-ber-01 /gluster-export
>>>> gluster-wien-04-int::aut-wien-vol-01 Passive N/A N/A
>>>> 0 0 0 0 0
>>>>
>>>>
>>>> [ 13:41:55 ] - root at gluster-ger-ber-07
>>>> /var/log/glusterfs/geo-replication/ger-ber-01 $gluster volume
>>>> geo-replication ger-ber-01 gluster-wien-07::aut-wien-vol-01 config
>>>> special_sync_mode: partial
>>>> state_socket_unencoded:
>>>> /var/lib/glusterd/geo-replication/ger-ber-01_gluster-wien-07_aut-wien-vol-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01.socket
>>>> gluster_log_file:
>>>> /var/log/glusterfs/geo-replication/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01.gluster.log
>>>> ssh_command: ssh -p 2503 -oPasswordAuthentication=no
>>>> -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem
>>>> ignore_deletes: true
>>>> change_detector: changelog
>>>> ssh_command_tar: ssh -p 2503 -oPasswordAuthentication=no
>>>> -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/tar_ssh.pem
>>>> state_file:
>>>> /var/lib/glusterd/geo-replication/ger-ber-01_gluster-wien-07_aut-wien-vol-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01.status
>>>> remote_gsyncd: /nonexistent/gsyncd
>>>> log_file:
>>>> /var/log/glusterfs/geo-replication/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01.log
>>>> changelog_log_file:
>>>> /var/log/glusterfs/geo-replication/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01-changes.log
>>>> socketdir: /var/run
>>>> working_dir:
>>>> /var/lib/misc/glusterfsd/ger-ber-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01
>>>> state_detail_file:
>>>> /var/lib/glusterd/geo-replication/ger-ber-01_gluster-wien-07_aut-wien-vol-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01-detail.status
>>>> session_owner: 6a071cfa-b150-4f0b-b1ed-96ab5d4bd671
>>>> gluster_command_dir: /usr/sbin/
>>>> pid_file:
>>>> /var/lib/glusterd/geo-replication/ger-ber-01_gluster-wien-07_aut-wien-vol-01/ssh%3A%2F%2Froot%4082.199.131.132%3Agluster%3A%2F%2F127.0.0.1%3Aaut-wien-vol-01.pid
>>>> georep_session_working_dir:
>>>> /var/lib/glusterd/geo-replication/ger-ber-01_gluster-wien-07_aut-wien-vol-01/
>>>> gluster_params: aux-gfid-mount
>>>> volume_id: 6a071cfa-b150-4f0b-b1ed-96ab5d4bd671
>>>> [ 13:42:11 ] - root at gluster-ger-ber-07
>>>> /var/log/glusterfs/geo-replication/ger-ber-01 $
>>>>
>>>>
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>>
>>>
>>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-users
>>
>
> --
> Dietmar Putz
> 3Q Medien GmbH
> Wetzlarer Str. 86
> D-14482 Potsdam
> Telefax: +49 (0)331 / 2797 866 - 1
> Telefon: +49 (0)331 / 2792 866 - 8
> Mobile: +49 171 / 90 160 39
> Mail: putz at 3qmedien.net
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160119/128217c7/attachment.html>
More information about the Gluster-users
mailing list