[Gluster-users] Unify over AFR with pre-existing data
baggio liu
baggioss at gmail.com
Thu Jul 24 05:27:01 UTC 2008
Hi chawkins:
I have made some tests, it seems work well except
*WARNING: initlog is deprecated and will be removed in a future release *
*logger *can work well too.
BTW, to use data existed,
1. set trused.glusterfs.version xatrr
2. remount the glusterfs client ins necessary.And namespace can be
update. (I've check source code, update namespace has been done in
fuse_getattr() )
Thanks & Regards
Baggio
2008/7/23 <chawkins at bplinux.com>:
> Hi - I wrote a script to set attributes on pre-existing data, but I have
> done very little testing with it so far. If you want to try it out it might
> save you some manual labor. After some more testing I was going to post it
> to the list and the wiki but it looks like you might be able to use it
> now... It works and it seems safe enough as it doesn't do anything but add
> gluster attributes to files which don't have any. Let me know if it works
> for you, and if anyone has any suggestions for improvement.
>
> Thanks,
> Chris
>
> #!/bin/bash
>
> DIR=$1
>
> if [ -z $DIR ]
> then
> echo
> echo "Error - no path was given."
> echo
> echo "USAGE: set-attrs /path/to/check"
> echo
> echo "Run set-attrs to check all files and directories in the"
> echo "given path for glusterfs attributes. Any files without"
> echo "a glusterfs version will be given a version attr of 2"
> echo "and a createtime attr equal to the files last modified time."
> echo
> echo "Output information is logged to syslog (usually
> /var/log/messages)"
> echo
> exit 0
> fi
>
> set_attrs() {
>
> get_ver="getfattr --absolute-names -n trusted.glusterfs.version"
> set_ver="setfattr -n trusted.glusterfs.version"
> set_ct="setfattr -n trusted.glusterfs.createtime"
>
> if test "`$get_ver $i 2>/dev/null | fgrep -c trusted.glusterfs.version=`" =
> "0"
> then
> mods=$((mods+1))
> # comment out next line to turn off logging
> initlog -n set-attrs -s "Setting attrs on $i"
> CT=`stat -c %Y $i`
> $set_ct -v $CT $i
> $set_ver -v 2 $i
> fi
> }
>
> mods=0
> checks=0
>
> initlog -n watch-dir -s "Starting a set-attrs run on path: $DIR"
>
> for i in `find $DIR -print`
> do
> set_attrs $i
> checks=$((checks+1))
> done
>
> initlog -n watch-dir -s "Completed. $checks checks and $mods mods in this
> run"
>
> exit 0
>
> ----- Original Message -----
> From: "baggio liu" <baggioss at gmail.com>
> To: "Keith Freedman" <freedman at freeformit.com>, gluster-users at gluster.org
> Sent: Wednesday, July 23, 2008 2:47:01 AM GMT -05:00 US/Canada Eastern
> Subject: Re: [Gluster-users] Unify over AFR with pre-existing data
>
> Hi ,
> thanks for reply. And pls see my comment inline. And question is still
> pending. has any more grace way to fix this issue??
>
> Regards
> Baggio
>
>
>
>
> 2008/7/23 Keith Freedman <freedman at freeformit.com>:
>
> I had 2 situations with pre-existing data and I solved the both 2 different
>> ways.
>>
>> 1), I followed the instructions in the link provided, however, I was
>> running xfs on the underlying filesystem and so the command to add
>> attributes is different.
>> in this case, it's attr instead of setfattr to set the extended attribute.
>> Also, I *think* the actual attribute name is different.
>
>
> The extension attr names are hard-code , so it should not change
> according to underlying fs.
>
>
>> so, what I recommend is to create one file through gluster, view it's
>> extended attributes through the base filesystem then apply those to the
>> remaining files.
>>
>> 2) This is the less friendly but more surefire solution:
>> make the gluster source point a sub directory in the source filesystem.
>> then "move" the files into the gluster filesystem.
>> This is less handy for "live" data, but it does insure the existing data
>> is properly "glustered"
>
>
> This way can make data right, but too many extra cost has been
> involved. In our plan, hundreds of TB should be stored in this fs , This
> loop may cost several days . So it can not be tolerated.
>
>
>>
>>
>> here's what I did:
>> CURRENT SOURCE: /home
>> GLUSTER SOURCE /home/home
>> GLUSTER MOUNT POINT /gluster/home
>>
>> (this may not be syntatically correct):
>> cd /home
>> for i in *
>> do
>> if [ $i != 'glusterhome' ]
>> mv $i glusterhome
>> ln -s gluster/home/$i $i
>> fi
>>
>> then unmount /gluster/home, unmount /home remount /home on /gluster
>> now you have /gluster/home with your data which has the gluster attributes
>> attached.
>> mount glusterfs from /gluster/home to /home
>>
>> I actually used this second procedure to get the data onto my new AFR
>> mirror. It slowed the process down since it was replicating 50GB of data
>> while moving it form disk to disk, but I'm not sure it slowed it down much.
>>
>
>
>
>
>> it seems to really mostly just reduce disk thrashing and my copy was going
>> at near network speed.
>
>
> BTW, I'm not very clear of it, Could you explain in detail? or
> list some example, test result, and so on. thx
>
>>
>>
>> Keith
>> At 07:11 PM 7/22/2008, baggio liu wrote:
>>
>>> Hi guys,
>>> I have configured a cluster with unify over afr. one of server in afr
>>> has some pre-existing data. And I want to make these data can be seen by
>>> client.
>>> Some config have been done according to
>>> "Setting up AFR on two servers with pre-existing data"
>>>
>>> <
>>> http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data
>>> >
>>> http://gluster.org/docs/index.php/Setting_up_AFR_on_two_servers_with_pre-existing_data
>>>
>>> But namespace can not aware with those data.
>>> After afr completing, ls at client , pre-existing data can not be
>>> seen. cat them, Input/output error can be found.
>>>
>>>
>>> What should I fix them? Thank you for any help.
>>>
>>> Regards
>>> Baggio
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>>>
>>
>>
>
> _______________________________________________ Gluster-users mailing list
> Gluster-users at gluster.org
> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20080724/6d12e084/attachment.html>
More information about the Gluster-users
mailing list