[Gluster-users] migration operations: Stopping a migration
Eric
epretorious at yahoo.com
Thu Sep 6 05:40:58 UTC 2012
FYI: While working on another project, I discovered that the file system attributes had been restored to their original state (or at least what I remember their original state to be):
> [eric at sn1 srv]$ for x in sda7 sdb7 sda8 sdb8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done
> # file: srv/sda7
> trusted.afr.Repositories-client-0
> trusted.afr.Repositories-client-1
> trusted.gfid
> trusted.glusterfs.dht
> trusted.glusterfs.volume-id
>
> # file: srv/sdb7
> trusted.afr.Repositories-client-2
> trusted.afr.Repositories-client-3
> trusted.gfid
> trusted.glusterfs.dht
> trusted.glusterfs.volume-id
>
> ...
Eric Pretorious
Truckee, CA
>________________________________
> From: Eric <epretorious at yahoo.com>
>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>Sent: Wednesday, September 5, 2012 9:34 PM
>Subject: Re: [Gluster-users] migration operations: Stopping a migration
>
>
>I chose to clear the attributes, leave the files intact, and replace the new brick with the old brick:
>
>
>> [eric at sn1 srv]$ for x in `sudo getfattr -m - /srv/sda7 2> /dev/null | tail -n +2` ; do sudo setfattr -x $x /srv/sda7 ; done
>
>
>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start
>> replace-brick started successfully
>>
>> gluster> volume
replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 status
>> Number of files migrated = 24631 Migration complete
>>
>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 commit
>> replace-brick commit successful
>>
>> gluster> volume info Repositories
>>
>> Volume Name: Repositories
>> Type: Distributed-Replicate
>> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c
>> Status: Started
>> Number of Bricks: 2 x 2 = 4
>> Transport-type: tcp
>> Bricks:
>> Brick1: 192.168.1.1:/srv/sda7
>> Brick2: 192.168.1.2:/srv/sda7
>> Brick3: 192.168.1.1:/srv/sdb7
>> Brick4: 192.168.1.2:/srv/sdb7
>
>
>...but the file system attributes of /srv/sda7 (i.e., the old brick) are not the same as they were when I began this journey:
>
>
>> [eric at sn1 srv]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done
>> # file: srv/sda7
>> trusted.gfid
>> trusted.glusterfs.volume-id
>>
>> # file: srv/sdb7
>> trusted.afr.Repositories-client-2
>> trusted.afr.Repositories-client-3
>> trusted.gfid
>> trusted.glusterfs.dht
>> trusted.glusterfs.volume-id
>
>
>Why is this? What will the long-term effects of this change? Are there other features that depend on those attributes?
>
>
>
>(The *original* attributes of /srv/sda7 are at the very bottom of this message.)
>
>
>Eric Pretorious
>Truckee, CA
>
>
>>________________________________
>> From: Eric <epretorious at yahoo.com>
>>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>>Sent: Wednesday, September 5, 2012 9:08 PM
>>Subject: Re: [Gluster-users] migration operations: Stopping a migration
>>
>>
>>On yet another related note: I noticed that - consistent with the keeping the file system attributes - the files them selves are left intact on the old brick. Once I remove the file system attributes...
>>
>>
>>> # for x in `getfattr -m - /srv/sda7 2> /dev/null | tail -n +2` ; do setfattr -x $x /srv/sda7 ; done
>>
>>
>>....should I remove the old files? Or should I leave the files intact and migrate back to the original brick:
>>
>>
>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 start
>>
>>
>>
>>...and then heal the volume?
>>
>>
>>Eric Pretorious
>>Truckee, CA
>>
>>
>>
>>
>>>________________________________
>>> From: Eric <epretorious at yahoo.com>
>>>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>>>Sent: Wednesday, September 5, 2012 5:42 PM
>>>Subject: Re: [Gluster-users] migration operations: Stopping a migration
>>>
>>>
>>>On a related note: Now that the PoC has been completed, I'm not able to migrate back to the original brick because of the undocumented, save-you-from-yourself file system attribute feature:
>>>
>>>
>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start
>>>> /srv/sda7 or a prefix of it is already part of a volume
>>>
>>>
>>>
>>>Is there a simpler, more-direct method of migrating back to the original brick or should I wipe the file system attributes manually? I only ask because:
>>>
>>>
>>>1. the long-term effects of this strategy aren't addressed in the Administration Guide AFAICT, and;
>>>
>>>2. though the intent of the feature has merit, it lacks elegance. e.g., the addition of a "force" attribute
>>>
>>> (like that of the commit feature) could be justified in this instance, IMHO.
>>>
>>>
>>> > gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda8 192.168.1.1:/srv/sda7 start force
>>> > Usage: volume replace-brick <VOLNAME> <BRICK> <NEW-BRICK> {start|pause|abort|status|commit [force]}
>>>
>>>
>>>
>>>Eric Pretorious
>>>Truckee, CA
>>>
>>>
>>>
>>>
>>>>________________________________
>>>> From: Eric <epretorious at yahoo.com>
>>>>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>>>>Sent: Wednesday, September 5, 2012 5:27 PM
>>>>Subject: Re: [Gluster-users] migration operations: Stopping a migration
>>>>
>>>>
>>>>On a hunch, I attempted the "volume replace-brick <VOLNAME> <BRICK> <NEW-BRICK> commit" command and, without much fanfare, the volume information was updated:
>>>>
>>>>
>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 commit
>>>>> replace-brick commit successful
>>>>>
>>>>> gluster> volume info
>>>>>
>>>>> Volume Name: Repositories
>>>>> Type:
Distributed-Replicate
>>>>> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c
>>>>> Status: Started
>>>>> Number of Bricks: 2 x 2 = 4
>>>>> Transport-type: tcp
>>>>> Bricks:
>>>>> Brick1: 192.168.1.1:/srv/sda8
>>>>> Brick2: 192.168.1.2:/srv/sda7
>>>>> Brick3: 192.168.1.1:/srv/sdb7
>>>>> Brick4: 192.168.1.2:/srv/sdb7
>>>>>
>>>>> gluster> volume status
>>>>> Status of volume: Repositories
>>>>> Gluster process Port Online Pid
>>>>> ------------------------------------------------------------------------------
>>>>> Brick 192.168.1.1:/srv/sda8 24012 Y 13796
>>>>> Brick 192.168.1.2:/srv/sda7
24009 Y 4946
>>>>> Brick 192.168.1.1:/srv/sdb7 24010 Y 5438
>>>>> Brick 192.168.1.2:/srv/sdb7 24010 Y 4951
>>>>> NFS Server on localhost 38467 Y 13803
>>>>> Self-heal Daemon on localhost N/A Y 13808
>>>>> NFS Server on 192.168.1.2 38467 Y 7969
>>>>> Self-heal Daemon on 192.168.1.2
N/A Y 7974
>>>>
>>>>
>>>>The XFS attributes are still intact on the old brick, however:
>>>>
>>>>
>>>>> [eric at sn1 ~]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done
>>>>> # file: srv/sda7
>>>>> trusted.afr.Repositories-client-0
>>>>> trusted.afr.Repositories-client-1
>>>>> trusted.afr.Repositories-io-threads
>>>>> trusted.afr.Repositories-replace-brick
>>>>> trusted.gfid
>>>>> trusted.glusterfs.dht
>>>>> trusted.glusterfs.volume-id
>>>>>
>>>>> # file: srv/sdb7
>>>>> trusted.afr.Repositories-client-2
>>>>> trusted.afr.Repositories-client-3
>>>>> trusted.gfid
>>>>> trusted.glusterfs.dht
>>>>> trusted.glusterfs.volume-id
>>>>>
>>>>> # file:
srv/sda8
>>>>> trusted.afr.Repositories-io-threads
>>>>> trusted.afr.Repositories-replace-brick
>>>>> trusted.gfid
>>>>> trusted.glusterfs.volume-id
>>>>
>>>>
>>>>
>>>>Is this intentional (i.e., leaving the the attributes intact)? Or functionality that has yet to be implemented?
>>>>
>>>>
>>>>Eric Pretorious
>>>>Truckee, CA
>>>>
>>>>
>>>>
>>>>>________________________________
>>>>> From: Eric <epretorious at yahoo.com>
>>>>>To: "gluster-users at gluster.org" <gluster-users at gluster.org>
>>>>>Sent: Wednesday, September 5, 2012 5:05 PM
>>>>>Subject: [Gluster-users] migration operations: Stopping a migration
>>>>>
>>>>>
>>>>>I've created a distributed replicated volume:
>>>>>
>>>>>
>>>>>> gluster> volume info
>>>>>>
>>>>>> Volume Name: Repositories
>>>>>> Type: Distributed-Replicate
>>>>>> Volume ID: 926262ae-2aa6-4bf7-b19e-cf674431b06c
>>>>>> Status: Started
>>>>>> Number of Bricks: 2 x 2 = 4
>>>>>> Transport-type: tcp
>>>>>> Bricks:
>>>>>> Brick1: 192.168.1.1:/srv/sda7
>>>>>> Brick2: 192.168.1.2:/srv/sda7
>>>>>> Brick3: 192.168.1.1:/srv/sdb7
>>>>>> Brick4: 192.168.1.2:/srv/sdb7
>>>>>
>>>>>
>>>>>
>>>>>...and begun migrating data from one brick to another as a PoC:
>>>>>
>>>>>
>>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 start
>>>>>> replace-brick started successfully
>>>>>>
>>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7 192.168.1.1:/srv/sda8 status
>>>>>> Number of files migrated = 5147 Current file= /centos/5.8/os/x86_64/CentOS/gnome-pilot-conduits-2.0.13-7.el5.x86_64.rpm
>>>>>> gluster> volume replace-brick Repositories 192.168.1.1:/srv/sda7
192.168.1.1:/srv/sda8 status
>>>>>> Number of files migrated = 24631 Migration complete
>>>>>
>>>>>
>>>>>After the migration is finished, though, the list of bricks is wrong:
>>>>>
>>>>>
>>>>>
>>>>>> gluster> volume heal Repositories info
>>>>>> Heal operation on volume Repositories has been successful
>>>>>>
>>>>>> Brick 192.168.1.1:/srv/sda7
>>>>>> Number of entries: 0
>>>>>>
>>>>>> Brick 192.168.1.2:/srv/sda7
>>>>>> Number of entries: 0
>>>>>>
>>>>>> Brick 192.168.1.1:/srv/sdb7
>>>>>> Number of entries: 0
>>>>>>
>>>>>> Brick 192.168.1.2:/srv/sdb7
>>>>>> Number of entries: 0
>>>>>
>>>>>
>>>>>...and the XFS attributes are still intact on the old brick:
>>>>>
>>>>>
>>>>>> [eric at sn1 ~]$ for x in sda7 sdb7 sda8 ; do sudo getfattr -m - /srv/$x 2> /dev/null ; done
>>>>>> # file: srv/sda7
>>>>>> trusted.afr.Repositories-client-0
>>>>>> trusted.afr.Repositories-client-1
>>>>>> trusted.afr.Repositories-io-threads
>>>>>> trusted.afr.Repositories-replace-brick
>>>>>> trusted.gfid
>>>>>> trusted.glusterfs.dht
>>>>>> trusted.glusterfs.pump-path
>>>>>> trusted.glusterfs.volume-id
>>>>>>
>>>>>> # file:
srv/sdb7
>>>>>> trusted.afr.Repositories-client-2
>>>>>> trusted.afr.Repositories-client-3
>>>>>> trusted.gfid
>>>>>> trusted.glusterfs.dht
>>>>>> trusted.glusterfs.volume-id
>>>>>>
>>>>>> # file: srv/sda8
>>>>>> trusted.afr.Repositories-io-threads
>>>>>> trusted.afr.Repositories-replace-brick
>>>>>> trusted.gfid
>>>>>> trusted.glusterfs.volume-id
>>>>>
>>>>>
>>>>>
>>>>>Have I missed a step? Or: Is this (i.e., clean-up) a bug or functionality that hasn't been implemented yet?
>>>>>
>>>>>
>>>>>Eric Pretorious
>>>>>Truckee, CA
>>>>>
>>>>>_______________________________________________
>>>>>Gluster-users mailing list
>>>>>Gluster-users at gluster.org
>>>>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>>
>>>>>
>>>>>
>>>>_______________________________________________
>>>>Gluster-users mailing list
>>>>Gluster-users at gluster.org
>>>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>
>>>>
>>>>
>>>_______________________________________________
>>>Gluster-users mailing list
>>>Gluster-users at gluster.org
>>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>
>>>
>>>
>>_______________________________________________
>>Gluster-users mailing list
>>Gluster-users at gluster.org
>>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>
>>
>>
>_______________________________________________
>Gluster-users mailing list
>Gluster-users at gluster.org
>http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120905/46f9ea22/attachment.html>
More information about the Gluster-users
mailing list