[Gluster-users] Bareos backup from Gluster mount

Pranith Kumar Karampuri pkarampu at redhat.com
Thu Jul 30 10:06:33 UTC 2015



On 07/30/2015 03:34 PM, Soumya Koduri wrote:
>
>
> On 07/30/2015 01:07 PM, Pranith Kumar Karampuri wrote:
>> Added folks who work on nfs on gluster. Let's see.
>>
>> Pranith
>>
>> On 07/30/2015 09:25 AM, Ryan Clough wrote:
>>> Okay, I think this has to do with Gluster NFS even though I was not
>>> accessing the Gluster volume via NFS.
>>>
>>> Directly on the bricks the files look like this:
>>>
>>> Hgluster01
>>> -r--r--r--. 2       602 602 832M Apr  3 06:17 scan_89.tar.bak
>>> ---------T  2 nfsnobody nfsnobody    0 Jul 13 11:42 scan_90.tar.bak
>>> ---------T  2 nfsnobody nfsnobody    0 Jul 13 11:42 scan_91.tar.bak
>>> ---------T  2 nfsnobody nfsnobody    0 Jul 13 11:42 scan_92.tar.bak
>>> ---------T  2 nfsnobody nfsnobody    0 Jul 13 11:42 scan_94.tar.bak
>>> ---------T  2 nfsnobody nfsnobody    0 Jul 13 11:42 scan_95.tar.bak
>>> -r--r--r--. 2       602       602 839M Apr  3 11:39 scan_96.tar.bak
>>> ---------T  2 nfsnobody nfsnobody    0 Jul 13 11:42 scan_98.tar.bak
>>> ---------T  2 nfsnobody nfsnobody    0 Jul 13 11:42 scan_99.tar.bak
>>>
>>> Hgluster02
>>> ---------T  2 nfsnobody nfsnobody           0 Jul 13 11:42 
>>> scan_89.tar.bak
>>> -r--r--r--. 2       602       602   869939200 Apr  3 07:36 
>>> scan_90.tar.bak
>>> -r--r--r--. 2       602       602   868331520 Apr  3 09:36 
>>> scan_91.tar.bak
>>> -r--r--r--. 2       602       602   870092800 Apr  3 09:37 
>>> scan_92.tar.bak
>>> -r--r--r--. 2       602       602   875448320 Apr  3 09:39 
>>> scan_93.tar.bak
>>> -r--r--r--. 2       602       602   870656000 Apr  3 09:40 
>>> scan_94.tar.bak
>>> -r--r--r--. 2       602       602   869396480 Apr  3 11:38 
>>> scan_95.tar.bak
>>> -r--r--r--. 2       602       602   881858560 Apr  3 11:40 
>>> scan_97.tar.bak
>>> -r--r--r--. 2       602       602   868188160 Apr  3 11:41 
>>> scan_98.tar.bak
>>> -r--r--r--. 2       602       602   865382400 Apr  3 13:32 
>>> scan_99.tar.bak
>>>
>>> So I turned off NFS and from a client tried to move the files to see
>>> if that would get rid of these weird nfsnobody files. Didn't work.
>>> After moving the files to a new directory, the new directory still had
>>> all of the nfsnobody files on the bricks.
>>>
> This is not an issue with Gluster NFS, but the expected behaviour when 
> the option 'server.root-squash' is on. To verify that, disable Gluster 
> NFS, enable this option and try to create a file via Gluster Native 
> mount. The uid/gid of the file created shall be the value of the 
> options anonuid/anongid which are by default 65534 (nfsnobody user - 
> can be checked in 'etc/passwd' file).
>
> Could you please confirm if the option 'server.root-squash' is on for 
> your volume? If yes, are there any values set for 'server.anonuid' and 
> 'server.anongid'.
These are the options on his volume:
     Options Reconfigured:
     performance.io-thread-count: 24
     server.event-threads: 20
     client.event-threads: 4
     performance.readdir-ahead: on
     features.inode-quota: on
     features.quota: on
     nfs.disable: off
     auth.allow: 192.168.10.*,10.0.10.*,10.8.0.*,10.2.0.*,10.0.60.*
     server.allow-insecure: on
     server.root-squash: on
     performance.read-ahead: on
     features.quota-deem-statfs: on
     diagnostics.brick-log-level: WARNING

Pranith
>
> Thanks,
> Soumya
>
>>> Next, I used rsync from the same client to copy all of the files to a
>>> new directory and, lo and behold, the nfsnobody files were gone. I
>>> tested a Bareos backup job and the data was read without issue from
>>> both nodes. There was zero empty files.
>>>
>>> I guess I will blame Gluster NFS for this one? If you need any more
>>> information from me, I would be happy to oblige.
>>>
>
>
>>> ___________________________________________
>>> ¯\_(ツ)_/¯
>>> Ryan Clough
>>> Information Systems
>>> Decision Sciences International Corporation
>>> <http://www.decisionsciencescorp.com/><http://www.decisionsciencescorp.com/> 
>>>
>>>
>>> On Wed, Jul 29, 2015 at 5:53 AM, Pranith Kumar Karampuri
>>> <pkarampu at redhat.com <mailto:pkarampu at redhat.com>> wrote:
>>>
>>>     hi Ryan,
>>>            What do you see in the logs of glusterfs mount and bricks?
>>>     Do you think it is possible for you to attach those logs to this
>>>     thread so that we can see what could be going on?
>>>
>>>     Pranith
>>>
>>>
>>>     On 07/28/2015 02:32 AM, Ryan Clough wrote:
>>>>     Hello,
>>>>
>>>>     I have cross-posted this question in the bareos-users mailing 
>>>> list.
>>>>
>>>>     Wondering if anyone has tried this because I am unable to backup
>>>>     data that is mounted via Gluster Fuse or Gluster NFS. Basically,
>>>>     I have the Gluster volume mounted on the Bareos Director which
>>>>     also has the tape changer attached.
>>>>
>>>>     Here is some information about versions:
>>>>     Bareos version 14.2.2
>>>>     Gluster version 3.7.2
>>>>     Scientific Linux version 6.6
>>>>
>>>>     Our Gluster volume consists of two nodes in distribute only. Here
>>>>     is the configuration of our volume:
>>>>     [root at hgluster02 ~]# gluster volume info
>>>>
>>>>     Volume Name: export_volume
>>>>     Type: Distribute
>>>>     Volume ID: c74cc970-31e2-4924-a244-4c70d958dadb
>>>>     Status: Started
>>>>     Number of Bricks: 2
>>>>     Transport-type: tcp
>>>>     Bricks:
>>>>     Brick1: hgluster01:/gluster_data
>>>>     Brick2: hgluster02:/gluster_data
>>>>     Options Reconfigured:
>>>>     performance.io-thread-count: 24
>>>>     server.event-threads: 20
>>>>     client.event-threads: 4
>>>>     performance.readdir-ahead: on
>>>>     features.inode-quota: on
>>>>     features.quota: on
>>>>     nfs.disable: off
>>>>     auth.allow: 192.168.10.*,10.0.10.*,10.8.0.*,10.2.0.*,10.0.60.*
>>>>     server.allow-insecure: on
>>>>     server.root-squash: on
>>>>     performance.read-ahead: on
>>>>     features.quota-deem-statfs: on
>>>>     diagnostics.brick-log-level: WARNING
>>>>
>>>>     When I try to backup a directory from Gluster Fuse or Gluster NFS
>>>>     mount and I monitor the network communication I only see data
>>>>     being pulled from the hgluster01 brick. When the job finishes
>>>>     Bareos thinks that it completed without error but included in the
>>>>     messages for the job are lots and lots of permission denied
>>>>     errors like this:
>>>>     15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>>>>     "/export/rclough/psdv-2014-archives-2/scan_111.tar.bak":
>>>>     ERR=Permission denied.
>>>>     15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>>>>     "/export/rclough/psdv-2014-archives-2/run_219.tar.bak":
>>>>     ERR=Permission denied.
>>>>     15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>>>>     "/export/rclough/psdv-2014-archives-2/scan_112.tar.bak":
>>>>     ERR=Permission denied.
>>>>     15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>>>>     "/export/rclough/psdv-2014-archives-2/run_220.tar.bak":
>>>>     ERR=Permission denied.
>>>>     15-Jul 02:03 ripper.red.dsic.com-fd JobId 613: Cannot open
>>>>     "/export/rclough/psdv-2014-archives-2/scan_114.tar.bak":
>>>>     ERR=Permission denied.
>>>>
>>>>     At first I thought this might be a root-squash problem but, if I
>>>>     try to read/copy a file using the root user from the Bareos
>>>>     server that is trying to do the backup, I can read files just 
>>>> fine.
>>>>
>>>>     When the job finishes is reports that it finished "OK -- with
>>>>     warnings" but, again the log for the job is filled with
>>>>     "ERR=Permission denied" messages. In my opinion, this job did not
>>>>     finish OK and should be Failed. Some of the files from the
>>>>     HGluster02 brick are backed up but all of the ones with
>>>>     permission errors do not. When I restore the job, all of the
>>>>     files with permission errors are empty.
>>>>
>>>>     Has anyone successfully used Bareos to backup data from Gluster
>>>>     mounts? This is an important use case for us because this is the
>>>>     largest single volume that we have to prepare large amounts of
>>>>     data to be archived.
>>>>
>>>>     Thank you for your time,
>>>>     ___________________________________________
>>>>     ¯\_(ツ)_/¯
>>>>     Ryan Clough
>>>>     Information Systems
>>>>     Decision Sciences International Corporation
>>>> <http://www.decisionsciencescorp.com/><http://www.decisionsciencescorp.com/>
>>>>
>>>>     This email and its contents are confidential. If you are not the
>>>>     intended recipient, please do not disclose or use the information
>>>>     within this email or its attachments. If you have received this
>>>>     email in error, please report the error to the sender by return
>>>>     email and delete this communication from your records.
>>>>
>>>>
>>>>     _______________________________________________
>>>>     Gluster-users mailing list
>>>>     Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>>>>     http://www.gluster.org/mailman/listinfo/gluster-users
>>>
>>>
>>>
>>> This email and its contents are confidential. If you are not the
>>> intended recipient, please do not disclose or use the information
>>> within this email or its attachments. If you have received this email
>>> in error, please report the error to the sender by return email and
>>> delete this communication from your records.
>>



More information about the Gluster-users mailing list