[Gluster-devel] Backup support for GlusterFS

[Alok Srivastava]

Reviving the old thread..

I was thinking about a possible NDMP based migration tool built into
GlusterFS.
Migration from any other file system storage (source) to gluster storage
(target) can be challenging because of lack of a tool built into GlusterFS.
Host based migration is a common approach but it can be painful for large
datasets.

The proposed migration tool will  transfer data over the Network Data
Management Protocol (NDMP). This tool will make use of the snapshots
feature on the source system to ensure that a point-in- time copy of the
file system is transferred to the target. The proposed tool can also be
utilized to migrate from gluster on old hardware to gluster on new hardware.

Following is the proposed workflow:

*1) **First copy*

Migrate recursively from source directory to the gluster cluster. This is
expected to be a time consuming step.

*  2)      Incremental copy*

Migrating any changes that occur after the first copy.

  3)      Cutover

Restrict clients from accessing data on the source , migrate remaining
changes , and redirect clients to access their data on the gluster cluster.

Note that there may be issues with migrating existing file permissions,
quota enforcements, etc on the source.

Missing pieces in gluster


-NDMP support

-May have to revive “ndmp-server” project?


Please share your thoughts.


Regards

On Mon, Dec 1, 2014 at 6:43 PM, Niels de Vos <ndevos at redhat.com> wrote:

> On Mon, Dec 01, 2014 at 05:40:53PM +0530, Aravinda wrote:
> > Hi,
> >
> > We are trying to implement backup support for GlusterFS. Many Network
> backup
> > utilities like Bacula(open source), Amanda(open source), Symantec
> NetBackup
> > support NDMP(http://www.ndmp.org/). Comparison is available here
> > http://wiki.bacula.org/doku.php?id=comparisons
> >
> > Plan is to create glusterfs-ndmp-server, which utilizes glusterfs
> changelogs
> > to detect changes for incremental backup. Design is not yet finalized,
> > Comments & Suggestions Welcome.
> >
> > Looks like a project(https://forge.gluster.org/ndmp-server) in
> forge.gluster
> > is discontinued.
> >
> > PS: NDMP support is not available in Open Source editions of Bacula and
> > Amanda, but available in Enterprise Editions.
>
> Bareos (www.bareos.org) a full Open Source fork of Bacula seems to
> support NDMP:
> - https://github.com/bareos/bareos/blob/master/README.NDMP
> - http://doc.bareos.org/master/html/bareos-manual-main-
> reference.html#x1-12800010.3
>
> I think Bareos has already support to backup to a Gluster Volume.
>
> From my understanding you are proposing a feature that integrates a
> (async/changelog) NDMP-client in the Gluster xlator stack that writes to
> a NDMP-server?
>
> HTH,
> Niels
>
> >
> > References:
> > -----------
> > 1. NDMP support in NetBackup
> > http://www.symantec.com/business/support/index?page=content&id=DOC6456
> > 2. NDMP Presentation
> > http://www.ndmp.org/download/sdk_v4/ndmp-overview-r2.ppt
> > 3. NDMP website http://www.ndmp.org/
> > 4. Bacula and Amanda website http://bacula.org/ and
> http://www.amanda.org/
> >
> > --
> > regards
> > Aravinda
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160808/becb1eab/attachment-0001.html>