[Gluster-users] Quota list not reflecting disk usage

[Steve Dainard]

Hi Manikandan

I'm using 'du' not df in this case.

On Thu, Jan 21, 2016 at 9:20 PM, Manikandan Selvaganesh
<mselvaga at redhat.com> wrote:
> Hi Steve,
>
> If you would like disk usage using df utility by taking quota limits into
> consideration, then you are expected to run the following command.
>
>    'gluster volume set VOLNAME quota-deem-statfs on'
>
> with older versions where quota-deem-statfs is OFF by default. However with
> the latest versions, quota-deem-statfs is by default ON. In this case, the total
> disk space of the directory is taken as the quota hard limit set on the directory
> of the volume and disk utility would display accordingly. This answers why there is
> a mismatch in disk utility.
>
> Next, answering to quota mechanism and accuracy: There is something called timeouts
> in quota. For performance reasons, quota caches the directory size on client. You can
> set timeout indicating the maximum valid duration of directory sizes in cache,
> from the time they are populated. By default the hard-timeout is 5s and soft timeout
> is 60s. Setting a timeout of zero will do a force fetching of directory sizes from server
> for every operation that modifies file data and will effectively disables directory size
> caching on client side. If you do not have a timeout of 0(which we do not encourage due to
> performance reasons), then till you reach soft-limit, soft timeout will be taken into
> consideration, and only for every 60s operations will be synced and that could cause the
> usage to exceed more than the hard-limit specified. If you would like quota to
> strictly enforce then please run the following commands,
>
>     'gluster v quota VOLNAME hard-timeout 0s'
>     'gluster v quota VOLNAME soft-timeout 0s'
>
> Appreciate your curiosity in exploring and if you would like to know more about quota
> please refer[1]
>
> [1] http://gluster.readthedocs.org/en/release-3.7.0-1/Administrator%20Guide/Directory%20Quota/
>
> --
> Thanks & Regards,
> Manikandan Selvaganesh.
>
> ----- Original Message -----
> From: "Steve Dainard" <sdainard at spd1.com>
> To: "gluster-users at gluster.org List" <gluster-users at gluster.org>
> Sent: Friday, January 22, 2016 1:40:07 AM
> Subject: Re: [Gluster-users] Quota list not reflecting disk usage
>
> This is gluster 3.6.6.
>
> I've attempted to disable and re-enable quota's on the volume, but
> when I re-apply the quotas on each directory the same 'Used' value is
> present as before.
>
> Where is quotad getting its information from, and how can I clean
> up/regenerate that info?
>
> On Thu, Jan 21, 2016 at 10:07 AM, Steve Dainard <sdainard at spd1.com> wrote:
>> I have a distributed volume with quota's enabled:
>>
>> Volume Name: storage
>> Type: Distribute
>> Volume ID: 26d355cb-c486-481f-ac16-e25390e73775
>> Status: Started
>> Number of Bricks: 4
>> Transport-type: tcp
>> Bricks:
>> Brick1: 10.0.231.50:/mnt/raid6-storage/storage
>> Brick2: 10.0.231.51:/mnt/raid6-storage/storage
>> Brick3: 10.0.231.52:/mnt/raid6-storage/storage
>> Brick4: 10.0.231.53:/mnt/raid6-storage/storage
>> Options Reconfigured:
>> performance.cache-size: 1GB
>> performance.readdir-ahead: on
>> features.quota: on
>> diagnostics.brick-log-level: WARNING
>>
>> Here is a partial list of quotas:
>> # /usr/sbin/gluster volume quota storage list
>>                   Path                   Hard-limit Soft-limit   Used
>> Available  Soft-limit exceeded? Hard-limit exceeded?
>> ---------------------------------------------------------------------------------------------------------------------------
>> ...
>> /projects-CanSISE                         10.0TB       80%      11.9TB
>>  0Bytes             Yes                  Yes
>> ...
>>
>> If I du on that location I do not get 11.9TB of space used (fuse mount point):
>> [root at storage projects-CanSISE]# du -hs
>> 9.5T .
>>
>> Can someone provide an explanation for how the quota mechanism tracks
>> disk usage? How often does the quota mechanism check its accuracy? And
>> how could it get so far off?
>>
>> Can I get gluster to rescan that location and update the quota usage?
>>
>> Thanks,
>> Steve
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users