[Gluster-devel] Backup support for GlusterFS

[Alok Srivastava]

On Wed, Aug 10, 2016 at 1:35 AM, Vijay Bellur <vbellur at redhat.com> wrote:

> On 08/08/2016 04:57 AM, Alok Srivastava wrote:
>
>> Reviving the old thread..
>>
>> I was thinking about a possible NDMP based migration tool built into
>> GlusterFS.
>> Migration from any other file system storage (source) to gluster storage
>> (target) can be challenging because of lack of a tool built into
>> GlusterFS. Host based migration is a common approach but it can be
>> painful for large datasets.
>>
>> The proposed migration tool will  transfer data over the Network Data
>> Management Protocol (NDMP). This tool will make use of the snapshots
>> feature on the source system to ensure that a point-in- time copy of the
>> file system is transferred to the target. The proposed tool can also be
>> utilized to migrate from gluster on old hardware to gluster on new
>> hardware.
>>
>> Following is the proposed workflow:
>>
>> *1) **First copy*
>>
>> Migrate recursively from source directory to the gluster cluster. This
>> is expected to be a time consuming step.
>>
>> * 2)      Incremental copy*
>>
>> Migrating any changes that occur after the first copy.
>>
>>   3)      Cutover
>>
>> Restrict clients from accessing data on the source , migrate remaining
>> changes , and redirect clients to access their data on the gluster
>> cluster.
>>
>>
>> Note that there may be issues with migrating existing file permissions,
>> quota enforcements, etc on the source.
>>
>> Missing pieces in gluster
>>
>>
>> -NDMP support
>>
>> -May have to revive “ndmp-server” project?
>>
>>
>>
>
> Why is NDMP better than rsync from NFS (src) to FUSE (destination)?. Is it
> owing to the incremental copy that NDMP can handle better?
>

Right, Determining the files for incremental  migration will take longer
with rsync. Also, the proposed approach is based on NDMP + Snapshots, Hence
it's not a one size fits all approach. However, Making use of the snapshots
will ensure that  a point in time copy is migrated and the in-flight
directories are also accessible to the clients connected to the source.

NDMP support with gluster will help in integration with other backup
utilities as well.

>
> I have not seen the ndmp-server project in action or the code to comment
> on how functional it was. We might have to implement from scratch if we
> decide to go down this route.
>
> Aravinda - do you remember the state of this project?
>
> Thanks,
> Vijay
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160819/089002af/attachment.html>