[Gluster-users] Bareos backup from Gluster mount
David Robinson
david.robinson at corvidtec.com
Thu Jul 30 20:19:47 UTC 2015
Thanks for the update Michael.
>That's more a limitation in the gluster protocol as it currently sits,
>not so much with whether you're using FUSE or libgfapi. There are
>probably other things you can do, though.
I thought that less "trips" were required when using libgfapi, and hence
the likely better performance.
>One might be to use dispersion instead of replication. Another might be
>to disable atime on the server. Per this thread (
>http://lists.gnu.org/archive/html/gluster-devel/2008-03/msg00149.html
>), if the server fs does not have atime enabled, gluster won't update
>it. That may well save you a few network round trips.
Already do this. We use noatime on all of the brick mounts.
>Another thing you might try is taking steps to ensure your ARP caches
>on your clients and bricks remain hot. (Random guess, here).
No idea how to do this...
>
>I've seen feature request tickets about bundling IOPs on the wire to
>opportunistically eliminate round trips, but I don't know what the
>status is of those.
Copy that... Thanks again... David
>
>
>On Thu, Jul 30, 2015 at 12:07 PM David Robinson
><david.robinson at corvidtec.com> wrote:
>>Copy that. I am very aware of the small file performance issues and I
>>have been tracking 3.7. I am waiting for 3.7 to stabilize before I
>>roll
>>it out to my production system.
>>
>>My thought was that it looks like Bareos now has built in use of
>>libgfapi, which from my understanding should improve performance for
>>gluster. My issues with rsync aren't with the actual transfer speed
>>of
>>the files. My issue is with the time it takes to search through the
>>filesystem to figure out which files to transfer. This takes an
>>extremely long time on 400TB of data, especially when it is going
>>through the directories with large numbers of small files.
>>
>>I was curious if Bareos with built-in support for libgfapi would be
>>faster than rsync backups between gluster machines.
>>Anyone on the email list care to comment? Thanks in advance for any
>>info that can be provided.
>>
>>David
>>
>>
>>------ Original Message ------
>>From: "André Bauer" <abauer at magix.net>
>>To: "David F. Robinson" <david.robinson at corvidtec.com>
>>Sent: 7/30/2015 11:35:21 AM
>>Subject: Re: [Gluster-users] Bareos backup from Gluster mount
>>
>> >Hi David,
>> >
>> >i never used Bareos until now. We like to switch from Bacula in the
>> >future but i think this will not happen before next Ubuntu LTS
>>release
>> >(16.04).
>> >
>> >I also never directly compared with rsync but i think rsync is faster
>> >in
>> >transfering because it does not have to do any compression and so
>>on...
>> >
>> >What i can say about Bacula on Glusterfs volumes is, that copying big
>> >files works at reasonable speed while small files (especialy if there
>> >are a lot) are a bit slow, whats in Glusterfs nature until versions
>> >prior 3.6(?).
>> >
>> >With Glusterfs 3.6 / 3.7 this should be a bit faster in the meantime
>> >but
>> >i have no experience with the performance gains because i'm still on
>> >Glusterfs 3.5.5.
>> >
>> >In conclusion i still prefer Bacula over Rsync even if its slower.
>> >
>> >Some more info about Glusterfs small file performance can be found
>> >here:
>> >
>> >https://gluster.readthedocs.org/en/latest/Feature%20Planning/GlusterFS%203.7/Small%20File%20Performance/
>> >
>> >Regards
>> >André
>> >
>> >
>> >
>> >Am 30.07.2015 um 15:23 schrieb David F. Robinson:
>> >> Andre,
>> >>
>> >> I am looking at a backup alternative to rsnc for gluster. My
>>storage
>> >>system is growing and rsync takes too long on my system (300TB). Do
>> >>you have any idea of the relative performance of bareos as compared
>>to
>> >>that of rsync? Can it be run in a multi-threaded mode? Rsync takes
>>an
>> >>extremely long time just searching the directory tree to figure out
>> >>what to copy. Before digging into bareos, I was wondering if you had
>> >>any thoughts on performance for gluster.
>> >>
>> >> David (Sent from mobile)
>> >>
>> >> ===============================
>> >> David F. Robinson, Ph.D.
>> >> President - Corvid Technologies
>> >> 704.799.6944 x101 [office]
>> >> 704.252.1310 [cell]
>> >> 704.799.7974 [fax]
>> >> David.Robinson at corvidtec.com
>> >> http://www.corvidtechnologies.com
>> >>
>> >>> On Jul 29, 2015, at 1:36 PM, André Bauer <abauer at magix.net>
>>wrote:
>> >>>
>> >>> We're using Bacula (Bareos is a fork of it) for backups.
>> >>> Never had any problems doing backups of Gluster volumes.
>> >>>
>> >>>> Am 27.07.2015 um 23:02 schrieb Ryan Clough:
>> >>>> Hello,
>> >>>>
>> >>>> I have cross-posted this question in the bareos-users mailing
>>list.
>> >>>>
>> >>>> Wondering if anyone has tried this because I am unable to backup
>> >>>>data
>> >>>> that is mounted via Gluster Fuse or Gluster NFS. Basically, I
>>have
>> >>>>the
>> >>>> Gluster volume mounted on the Bareos Director which also has the
>> >>>>tape
>> >>>> changer attached.
>> >>>>
>> >>>> Here is some information about versions:
>> >>>> Bareos version 14.2.2
>> >>>> Gluster version 3.7.2
>> >>>> Scientific Linux version 6.6
>> >>>>
>> >>>> Our Gluster volume consists of two nodes in distribute only.
>>Here
>> >>>>is the
>> >>>> configuration of our volume:
>> >>>> [root at hgluster02 ~]# gluster volume info
>> >>>>
>> >>>> Volume Name: export_volume
>> >>>> Type: Distribute
>> >>>> Volume ID: c74cc970-31e2-4924-a244-4c70d958dadb
>> >>>> Status: Started
>> >>>> Number of Bricks: 2
>> >>>> Transport-type: tcp
>> >>>> Bricks:
>> >>>> Brick1: hgluster01:/gluster_data
>> >>>> Brick2: hgluster02:/gluster_data
>> >>>> Options Reconfigured:
>> >>>> performance.io-thread-count: 24
>> >>>> server.event-threads: 20
>> >>>> client.event-threads: 4
>> >>>> performance.readdir-ahead: on
>> >>>> features.inode-quota: on
>> >>>> features.quota: on
>> >>>> nfs.disable: off
>> >>>> auth.allow: 192.168.10.*,10.0.10.*,10.8.0.*,10.2.0.*,10.0.60.*
>> >>>> server.allow-insecure: on
>> >>>> server.root-squash: on
>> >>>> performance.read-ahead: on
>> >>>> features.quota-deem-statfs: on
>> >>>> diagnostics.brick-log-level: WARNING
>> >>>>
>> >>>> When I try to backup a directory from Gluster Fuse or Gluster
>>NFS
>> >>>>mount
>> >>>> and I monitor the network communication I only see data being
>> >>>>pulled
>> >>>> from the hgluster01 brick. When the job finishes Bareos thinks
>>that
>> >>>>it
>> >>>> completed without error but included in the messages for the job
>> >>>>are
>> >>>> lots and lots of permission denied errors like this:
>> >>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>> >>>> "/export/rclough/psdv-2014-archives-2/scan_111.tar.bak":
>> >>>>ERR=Permission
>> >>>> denied.
>> >>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>> >>>> "/export/rclough/psdv-2014-archives-2/run_219.tar.bak":
>> >>>>ERR=Permission
>> >>>> denied.
>> >>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>> >>>> "/export/rclough/psdv-2014-archives-2/scan_112.tar.bak":
>> >>>>ERR=Permission
>> >>>> denied.
>> >>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>> >>>> "/export/rclough/psdv-2014-archives-2/run_220.tar.bak":
>> >>>>ERR=Permission
>> >>>> denied.
>> >>>> 15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>> >>>> "/export/rclough/psdv-2014-archives-2/scan_114.tar.bak":
>> >>>>ERR=Permission
>> >>>> denied.
>> >>>>
>> >>>> At first I thought this might be a root-squash problem but, if I
>> >>>>try to
>> >>>> read/copy a file using the root user from the Bareos server that
>>is
>> >>>> trying to do the backup, I can read files just fine.
>> >>>>
>> >>>> When the job finishes is reports that it finished "OK -- with
>> >>>>warnings"
>> >>>> but, again the log for the job is filled with "ERR=Permission
>> >>>>denied"
>> >>>> messages. In my opinion, this job did not finish OK and should
>>be
>> >>>> Failed. Some of the files from the HGluster02 brick are backed
>>up
>> >>>>but
>> >>>> all of the ones with permission errors do not. When I restore
>>the
>> >>>>job,
>> >>>> all of the files with permission errors are empty.
>> >>>>
>> >>>> Has anyone successfully used Bareos to backup data from Gluster
>> >>>>mounts?
>> >>>> This is an important use case for us because this is the largest
>> >>>>single
>> >>>> volume that we have to prepare large amounts of data to be
>> >>>>archived.
>> >>>>
>> >>>> Thank you for your time,
>> >>>> ___________________________________________
>> >>>> ¯\_(ツ)_/¯
>> >>>> Ryan Clough
>> >>>> Information Systems
>> >>>> Decision Sciences International Corporation
>> >>>>
>> >>>><http://www.decisionsciencescorp.com/><http://www.decisionsciencescorp.com/>
>> >>>>
>> >>>> This email and its contents are confidential. If you are not the
>> >>>> intended recipient, please do not disclose or use the
>>information
>> >>>>within
>> >>>> this email or its attachments. If you have received this email
>>in
>> >>>>error,
>> >>>> please report the error to the sender by return email and delete
>> >>>>this
>> >>>> communication from your records.
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Gluster-users mailing list
>> >>>> Gluster-users at gluster.org
>> >>>> http://www.gluster.org/mailman/listinfo/gluster-users
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> Mit freundlichen Grüßen
>> >>> André Bauer
>> >>>
>> >>> MAGIX Software GmbH
>> >>> André Bauer
>> >>> Administrator
>> >>> August-Bebel-Straße 48
>> >>> 01219 Dresden
>> >>> GERMANY
>> >>>
>> >>> tel.: 0351 41884875
>> >>> e-mail: abauer at magix.net
>> >>> abauer at magix.net <mailto:Email>
>> >>> www.magix.com <http://www.magix.com/>
>> >>>
>> >>>
>> >>> Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael
>> >>>Keith
>> >>> Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB
>>127205
>> >>>
>> >>> Find us on:
>> >>>
>> >>> <http://www.facebook.com/MAGIX> <http://www.twitter.com/magix_de>
>> >>> <http://www.youtube.com/wwwmagixcom> <http://www.magixmagazin.de>
>> >>>
>> >>>----------------------------------------------------------------------
>> >>> The information in this email is intended only for the addressee
>> >>>named
>> >>> above. Access to this email by anyone else is unauthorized. If
>>you
>> >>>are
>> >>> not the intended recipient of this message any disclosure,
>>copying,
>> >>> distribution or any action taken in reliance on it is prohibited
>>and
>> >>> may be unlawful. MAGIX does not warrant that any attachments are
>> >>>free
>> >>> from viruses or other defects and accepts no liability for any
>> >>>losses
>> >>> resulting from infected email transmissions. Please note that any
>> >>> views expressed in this email may be those of the originator and
>>do
>> >>> not necessarily represent the agenda of the company.
>> >>>
>> >>>----------------------------------------------------------------------
>> >>> _______________________________________________
>> >>> Gluster-users mailing list
>> >>> Gluster-users at gluster.org
>> >>> http://www.gluster.org/mailman/listinfo/gluster-users
>> >>
>> >
>> >
>> >--
>> >Mit freundlichen Grüßen
>> >André Bauer
>> >
>> >MAGIX Software GmbH
>> >André Bauer
>> >Administrator
>> >August-Bebel-Straße 48
>> >01219 Dresden
>> >GERMANY
>> >
>> >tel.: 0351 41884875
>> >e-mail: abauer at magix.net
>> >abauer at magix.net <mailto:Email>
>> >www.magix.com <http://www.magix.com/>
>> >
>> >
>> >Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael
>>Keith
>> >Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205
>> >
>> >Find us on:
>> >
>> ><http://www.facebook.com/MAGIX> <http://www.twitter.com/magix_de>
>> ><http://www.youtube.com/wwwmagixcom> <http://www.magixmagazin.de>
>> >----------------------------------------------------------------------
>> >The information in this email is intended only for the addressee
>>named
>> >above. Access to this email by anyone else is unauthorized. If you
>>are
>> >not the intended recipient of this message any disclosure, copying,
>> >distribution or any action taken in reliance on it is prohibited and
>> >may be unlawful. MAGIX does not warrant that any attachments are free
>> >from viruses or other defects and accepts no liability for any losses
>> >resulting from infected email transmissions. Please note that any
>> >views expressed in this email may be those of the originator and do
>> >not necessarily represent the agenda of the company.
>> >----------------------------------------------------------------------
>>
>>_______________________________________________
>>Gluster-users mailing list
>>Gluster-users at gluster.org
>>http://www.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150730/4b1552b7/attachment.html>
More information about the Gluster-users
mailing list