[Gluster-users] Double counting of quota

Mon Jun 8 08:02:19 UTC 2015

Hi Vijiay,
the use case is very simple.
I'm using gluster 3.7.1 with a replicated volume (replica 2), enabling quota. The 2 bricks are on the same machine but using two disks of the same size, but unfortunately one of them is slower, but I think it is irrelevant. Although I do not think it is important for this use case, but I want to note that the volumes are xfs formatted with default options and are thin logical volumes.
The gluster server is running on CentIS 7.1.
The problem occurs when using rsync to copy from an external source into the gluster volume: if I copy without specifying the temp dir rsync uses the current dir for temporaries and this is taken into account, at least in one of the two bricks. I can confirm it's only happening in one of the bricks by looking at the xattrs of the same dir on the two bricks, as the quota values are different.
At the moment I have recreated the bricks and started the copy over and it seems much better now that I'm explicitly asking rsync to use /tmp for temporaries.
Anyways, I'm still seeing errors in the logs that I will report later.
Many thanks for the help,

   Alessandro

> Il giorno 08/giu/2015, alle ore 08:38, Vijaikumar M <vmallika at redhat.com> ha scritto:
> 
> Hi Alessandro,
> 
> Please provide the test-case, so that we can try to re-create this problem in-house?
> 
> Thanks,
> Vijay
> 
>> On Saturday 06 June 2015 05:59 AM, Alessandro De Salvo wrote:
>> Hi,
>> just to answer to myself, it really seems the temp files from rsync are the culprit, it seems that their size are summed up to the real contents of the directories I’m synchronizing, or in other terms their size is not removed from the used size after they are removed. I suppose this is someway connected to the error on removexattr I’m seeing. The temporary solution I’ve found is to use rsync with the option to write the temp files to /tmp, but it would be very interesting to understand why this is happening.
>> Cheers,
>> 
>> 	Alessandro
>> 
>>> Il giorno 06/giu/2015, alle ore 01:19, Alessandro De Salvo <Alessandro.DeSalvo at roma1.infn.it> ha scritto:
>>> 
>>> Hi,
>>> I currently have two brick with replica 2 on the same machine, pointing to different disks of a connected SAN.
>>> The volume itself is fine:
>>> 
>>> # gluster volume info atlas-home-01
>>> 
>>> Volume Name: atlas-home-01
>>> Type: Replicate
>>> Volume ID: 660db960-31b8-4341-b917-e8b43070148b
>>> Status: Started
>>> Number of Bricks: 1 x 2 = 2
>>> Transport-type: tcp
>>> Bricks:
>>> Brick1: host1:/bricks/atlas/home02/data
>>> Brick2: host2:/bricks/atlas/home01/data
>>> Options Reconfigured:
>>> performance.write-behind-window-size: 4MB
>>> performance.io-thread-count: 32
>>> performance.readdir-ahead: on
>>> server.allow-insecure: on
>>> nfs.disable: true
>>> features.quota: on
>>> features.inode-quota: on
>>> 
>>> 
>>> However, when I set a quota on a dir of the volume the size show is twice the physical size of the actual dir:
>>> 
>>> # gluster volume quota atlas-home-01 list /user1
>>>                  Path                   Hard-limit Soft-limit   Used  Available  Soft-limit exceeded? Hard-limit exceeded?
>>> ---------------------------------------------------------------------------------------------------------------------------
>>> /user1                                    4.0GB       80%       3.2GB 853.4MB              No                   No
>>> 
>>> # du -sh /storage/atlas/home/user1
>>> 1.6G    /storage/atlas/home/user1
>>> 
>>> If I remove one of the bricks the quota shows the correct value.
>>> Is there any double counting in case the bricks are on the same machine?
>>> Also, I see a lot of errors in the logs like the following:
>>> 
>>> [2015-06-05 21:59:27.450407] E [posix-handle.c:157:posix_make_ancestryfromgfid] 0-atlas-home-01-posix: could not read the link from the gfid handle /bricks/atlas/home01/data/.glusterfs/be/e5/bee5e2b8-c639-4539-a483-96c19cd889eb (No such file or directory)
>>> 
>>> and also
>>> 
>>> [2015-06-05 22:52:01.112070] E [marker-quota.c:2363:mq_mark_dirty] 0-atlas-home-01-marker: failed to get inode ctx for /user1/file1
>>> 
>>> When running rsync I also see the following errors:
>>> 
>>> [2015-06-05 23:06:22.203968] E [marker-quota.c:2601:mq_remove_contri] 0-atlas-home-01-marker: removexattr trusted.glusterfs.quota.fddf31ba-7f1d-4ba8-a5ad-2ebd6e4030f3.contri failed for /user1/..bashrc.O4kekp: No data available
>>> 
>>> Those files are the temp files of rsync, I’m not sure why the throw errors in glusterfs.
>>> Any help?
>>> Thanks,
>>> 
>>> 	Alessandro
>>> 
>>> 
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-users
>> 
>> 
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-users
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150608/b0ef0820/attachment.html>