[Gluster-users] Bareos backup from Gluster mount

David Robinson david.robinson at corvidtec.com
Thu Jul 30 20:19:47 UTC 2015


Thanks for the update Michael.


>That's more a limitation in the gluster protocol as it currently sits, 
>not so much with whether you're using FUSE or libgfapi. There are 
>probably other things you can do, though.
I thought that less "trips" were required when using libgfapi, and hence 
the likely better performance.


>One might be to use dispersion instead of replication. Another might be 
>to disable atime on the server. Per this thread ( 
>http://lists.gnu.org/archive/html/gluster-devel/2008-03/msg00149.html 
>), if the server fs does not have atime enabled, gluster won't update 
>it. That may well save you a few network round trips.
Already do this.  We use noatime on all of the brick mounts.

>Another thing you might try is taking steps to ensure your ARP caches 
>on your clients and bricks remain hot. (Random guess, here).
No idea how to do this...

>
>I've seen feature request tickets about bundling IOPs on the wire to 
>opportunistically eliminate round trips, but I don't know what the 
>status is of those.

Copy that... Thanks again... David


>
>
>On Thu, Jul 30, 2015 at 12:07 PM David Robinson 
><david.robinson at corvidtec.com> wrote:
>>Copy that.  I am very aware of the small file performance issues and I
>>have been tracking 3.7.  I am waiting for 3.7 to stabilize before I 
>>roll
>>it out to my production system.
>>
>>My thought was that it looks like Bareos now has built in use of
>>libgfapi, which from my understanding should improve performance for
>>gluster.  My issues with rsync aren't with the actual transfer speed 
>>of
>>the files.  My issue is with the time it takes to search through the
>>filesystem to figure out which files to transfer.  This takes an
>>extremely long time on 400TB of data, especially when it is going
>>through the directories with large numbers of small files.
>>
>>I was curious if Bareos with built-in support for libgfapi would be
>>faster than rsync backups between gluster machines.
>>Anyone on the email list care to comment?  Thanks in advance for any
>>info that can be provided.
>>
>>David
>>
>>
>>------ Original Message ------
>>From: "André Bauer" <abauer at magix.net>
>>To: "David F. Robinson" <david.robinson at corvidtec.com>
>>Sent: 7/30/2015 11:35:21 AM
>>Subject: Re: [Gluster-users] Bareos backup from Gluster mount
>>
>> >Hi David,
>> >
>> >i never used Bareos until now. We like to switch from Bacula in the
>> >future but i think this will not happen before next Ubuntu LTS 
>>release
>> >(16.04).
>> >
>> >I also never directly compared with rsync but i think rsync is faster
>> >in
>> >transfering because it does not have to do any compression and so 
>>on...
>> >
>> >What i can say about Bacula on Glusterfs volumes is, that copying big
>> >files works at reasonable speed while small files (especialy if there
>> >are a lot) are a bit slow, whats in Glusterfs nature until versions
>> >prior 3.6(?).
>> >
>> >With Glusterfs 3.6 / 3.7 this should be a bit faster in the meantime
>> >but
>> >i have no experience with the performance gains because i'm still on
>> >Glusterfs 3.5.5.
>> >
>> >In conclusion i still prefer Bacula over Rsync even if its slower.
>> >
>> >Some more info about Glusterfs small file performance can be found
>> >here:
>> >
>> >https://gluster.readthedocs.org/en/latest/Feature%20Planning/GlusterFS%203.7/Small%20File%20Performance/
>> >
>> >Regards
>> >André
>> >
>> >
>> >
>> >Am 30.07.2015 um 15:23 schrieb David F. Robinson:
>> >>  Andre,
>> >>
>> >>  I am looking at a backup alternative to rsnc for gluster. My 
>>storage
>> >>system is growing and rsync takes too long on my system (300TB). Do
>> >>you have any idea of the relative performance of bareos as compared 
>>to
>> >>that of rsync? Can it be run in a multi-threaded mode? Rsync takes 
>>an
>> >>extremely long time just searching the directory tree to figure out
>> >>what to copy. Before digging into bareos, I was wondering if you had
>> >>any thoughts on performance for gluster.
>> >>
>> >>  David  (Sent from mobile)
>> >>
>> >>  ===============================
>> >>  David F. Robinson, Ph.D.
>> >>  President - Corvid Technologies
>> >>  704.799.6944 x101 [office]
>> >>  704.252.1310      [cell]
>> >>  704.799.7974      [fax]
>> >>  David.Robinson at corvidtec.com
>> >>  http://www.corvidtechnologies.com
>> >>
>> >>>  On Jul 29, 2015, at 1:36 PM, André Bauer <abauer at magix.net> 
>>wrote:
>> >>>
>> >>>  We're using Bacula (Bareos is a fork of it) for backups.
>> >>>  Never had any problems doing backups of Gluster volumes.
>> >>>
>> >>>>  Am 27.07.2015 um 23:02 schrieb Ryan Clough:
>> >>>>  Hello,
>> >>>>
>> >>>>  I have cross-posted this question in the bareos-users mailing 
>>list.
>> >>>>
>> >>>>  Wondering if anyone has tried this because I am unable to backup
>> >>>>data
>> >>>>  that is mounted via Gluster Fuse or Gluster NFS. Basically, I 
>>have
>> >>>>the
>> >>>>  Gluster volume mounted on the Bareos Director which also has the
>> >>>>tape
>> >>>>  changer attached.
>> >>>>
>> >>>>  Here is some information about versions:
>> >>>>  Bareos version 14.2.2
>> >>>>  Gluster version 3.7.2
>> >>>>  Scientific Linux version 6.6
>> >>>>
>> >>>>  Our Gluster volume consists of two nodes in distribute only. 
>>Here
>> >>>>is the
>> >>>>  configuration of our volume:
>> >>>>  [root at hgluster02 ~]# gluster volume info
>> >>>>
>> >>>>  Volume Name: export_volume
>> >>>>  Type: Distribute
>> >>>>  Volume ID: c74cc970-31e2-4924-a244-4c70d958dadb
>> >>>>  Status: Started
>> >>>>  Number of Bricks: 2
>> >>>>  Transport-type: tcp
>> >>>>  Bricks:
>> >>>>  Brick1: hgluster01:/gluster_data
>> >>>>  Brick2: hgluster02:/gluster_data
>> >>>>  Options Reconfigured:
>> >>>>  performance.io-thread-count: 24
>> >>>>  server.event-threads: 20
>> >>>>  client.event-threads: 4
>> >>>>  performance.readdir-ahead: on
>> >>>>  features.inode-quota: on
>> >>>>  features.quota: on
>> >>>>  nfs.disable: off
>> >>>>  auth.allow: 192.168.10.*,10.0.10.*,10.8.0.*,10.2.0.*,10.0.60.*
>> >>>>  server.allow-insecure: on
>> >>>>  server.root-squash: on
>> >>>>  performance.read-ahead: on
>> >>>>  features.quota-deem-statfs: on
>> >>>>  diagnostics.brick-log-level: WARNING
>> >>>>
>> >>>>  When I try to backup a directory from Gluster Fuse or Gluster 
>>NFS
>> >>>>mount
>> >>>>  and I monitor the network communication I only see data being
>> >>>>pulled
>> >>>>  from the hgluster01 brick. When the job finishes Bareos thinks 
>>that
>> >>>>it
>> >>>>  completed without error but included in the messages for the job
>> >>>>are
>> >>>>  lots and lots of permission denied errors like this:
>> >>>>  15-Jul 02:03 ripper.red.dsic.com-fd JobId 613:      Cannot open
>> >>>>  "/export/rclough/psdv-2014-archives-2/scan_111.tar.bak":
>> >>>>ERR=Permission
>> >>>>  denied.
>> >>>>  15-Jul 02:03 ripper.red.dsic.com-fd JobId 613:      Cannot open
>> >>>>  "/export/rclough/psdv-2014-archives-2/run_219.tar.bak":
>> >>>>ERR=Permission
>> >>>>  denied.
>> >>>>  15-Jul 02:03 ripper.red.dsic.com-fd JobId 613:      Cannot open
>> >>>>  "/export/rclough/psdv-2014-archives-2/scan_112.tar.bak":
>> >>>>ERR=Permission
>> >>>>  denied.
>> >>>>  15-Jul 02:03 ripper.red.dsic.com-fd JobId 613:      Cannot open
>> >>>>  "/export/rclough/psdv-2014-archives-2/run_220.tar.bak":
>> >>>>ERR=Permission
>> >>>>  denied.
>> >>>>  15-Jul 02:03 ripper.red.dsic.com-fd JobId 613:      Cannot open
>> >>>>  "/export/rclough/psdv-2014-archives-2/scan_114.tar.bak":
>> >>>>ERR=Permission
>> >>>>  denied.
>> >>>>
>> >>>>  At first I thought this might be a root-squash problem but, if I
>> >>>>try to
>> >>>>  read/copy a file using the root user from the Bareos server that 
>>is
>> >>>>  trying to do the backup, I can read files just fine.
>> >>>>
>> >>>>  When the job finishes is reports that it finished "OK -- with
>> >>>>warnings"
>> >>>>  but, again the log for the job is filled with "ERR=Permission
>> >>>>denied"
>> >>>>  messages. In my opinion, this job did not finish OK and should 
>>be
>> >>>>  Failed. Some of the files from the HGluster02 brick are backed 
>>up
>> >>>>but
>> >>>>  all of the ones with permission errors do not. When I restore 
>>the
>> >>>>job,
>> >>>>  all of the files with permission errors are empty.
>> >>>>
>> >>>>  Has anyone successfully used Bareos to backup data from Gluster
>> >>>>mounts?
>> >>>>  This is an important use case for us because this is the largest
>> >>>>single
>> >>>>  volume that we have to prepare large amounts of data to be
>> >>>>archived.
>> >>>>
>> >>>>  Thank you for your time,
>> >>>>  ___________________________________________
>> >>>>  ¯\_(ツ)_/¯
>> >>>>  Ryan Clough
>> >>>>  Information Systems
>> >>>>  Decision Sciences International Corporation
>> >>>>
>> >>>><http://www.decisionsciencescorp.com/><http://www.decisionsciencescorp.com/>
>> >>>>
>> >>>>  This email and its contents are confidential. If you are not the
>> >>>>  intended recipient, please do not disclose or use the 
>>information
>> >>>>within
>> >>>>  this email or its attachments. If you have received this email 
>>in
>> >>>>error,
>> >>>>  please report the error to the sender by return email and delete
>> >>>>this
>> >>>>  communication from your records.
>> >>>>
>> >>>>
>> >>>>  _______________________________________________
>> >>>>  Gluster-users mailing list
>> >>>>  Gluster-users at gluster.org
>> >>>>  http://www.gluster.org/mailman/listinfo/gluster-users
>> >>>>
>> >>>
>> >>>
>> >>>  --
>> >>>  Mit freundlichen Grüßen
>> >>>  André Bauer
>> >>>
>> >>>  MAGIX Software GmbH
>> >>>  André Bauer
>> >>>  Administrator
>> >>>  August-Bebel-Straße 48
>> >>>  01219 Dresden
>> >>>  GERMANY
>> >>>
>> >>>  tel.: 0351 41884875
>> >>>  e-mail: abauer at magix.net
>> >>>  abauer at magix.net <mailto:Email>
>> >>>  www.magix.com <http://www.magix.com/>
>> >>>
>> >>>
>> >>>  Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael
>> >>>Keith
>> >>>  Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 
>>127205
>> >>>
>> >>>  Find us on:
>> >>>
>> >>>  <http://www.facebook.com/MAGIX> <http://www.twitter.com/magix_de>
>> >>>  <http://www.youtube.com/wwwmagixcom> <http://www.magixmagazin.de>
>> >>>
>> >>>----------------------------------------------------------------------
>> >>>  The information in this email is intended only for the addressee
>> >>>named
>> >>>  above. Access to this email by anyone else is unauthorized. If 
>>you
>> >>>are
>> >>>  not the intended recipient of this message any disclosure, 
>>copying,
>> >>>  distribution or any action taken in reliance on it is prohibited 
>>and
>> >>>  may be unlawful. MAGIX does not warrant that any attachments are
>> >>>free
>> >>>  from viruses or other defects and accepts no liability for any
>> >>>losses
>> >>>  resulting from infected email transmissions. Please note that any
>> >>>  views expressed in this email may be those of the originator and 
>>do
>> >>>  not necessarily represent the agenda of the company.
>> >>>
>> >>>----------------------------------------------------------------------
>> >>>  _______________________________________________
>> >>>  Gluster-users mailing list
>> >>>  Gluster-users at gluster.org
>> >>>  http://www.gluster.org/mailman/listinfo/gluster-users
>> >>
>> >
>> >
>> >--
>> >Mit freundlichen Grüßen
>> >André Bauer
>> >
>> >MAGIX Software GmbH
>> >André Bauer
>> >Administrator
>> >August-Bebel-Straße 48
>> >01219 Dresden
>> >GERMANY
>> >
>> >tel.: 0351 41884875
>> >e-mail: abauer at magix.net
>> >abauer at magix.net <mailto:Email>
>> >www.magix.com <http://www.magix.com/>
>> >
>> >
>> >Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael 
>>Keith
>> >Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205
>> >
>> >Find us on:
>> >
>> ><http://www.facebook.com/MAGIX> <http://www.twitter.com/magix_de>
>> ><http://www.youtube.com/wwwmagixcom> <http://www.magixmagazin.de>
>> >----------------------------------------------------------------------
>> >The information in this email is intended only for the addressee 
>>named
>> >above. Access to this email by anyone else is unauthorized. If you 
>>are
>> >not the intended recipient of this message any disclosure, copying,
>> >distribution or any action taken in reliance on it is prohibited and
>> >may be unlawful. MAGIX does not warrant that any attachments are free
>> >from viruses or other defects and accepts no liability for any losses
>> >resulting from infected email transmissions. Please note that any
>> >views expressed in this email may be those of the originator and do
>> >not necessarily represent the agenda of the company.
>> >----------------------------------------------------------------------
>>
>>_______________________________________________
>>Gluster-users mailing list
>>Gluster-users at gluster.org
>>http://www.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150730/4b1552b7/attachment.html>


More information about the Gluster-users mailing list