[Gluster-devel] GlusterFS User and Group Quotas

[Venky Shankar]

On Fri, Dec 11, 2015 at 2:03 AM, Shyam <srangana at redhat.com> wrote:
> On 12/08/2015 04:32 AM, Vijaikumar Mallikarjuna wrote:
>>
>> Hi All,
>>
>> Below is the design for '*GlusterFS User and Group Quotas', *please
>> provide your feedback on the same.
>>
>>
>> *_Developers:_*
>> Vijaikumar.M and Manikandan.S
>>
>>
>> *_Introduction:_*
>> User and Group quotas is to limit the amount of disk space for a
>> specified user/group ID.
>> This documents provides some details about how the accounting (marker
>> xlator) can be done
>> for user and group quotas
>>
>>
>> *_Design:_*
>> We have three different approaches, each has pros and cons
>>
>
> my 2 c's,
>
> The problem below is that of maintaining a journal as I see it. With the
> addition of pre/post size awareness, so that the delta can be reflected in
> the quota appropriately.

+1. That's precisely what journaled quotas are - providing the same
benefits as journaled filesystem after unclean shutdowns.

>
> With the above in mind, I would say, steer clear on any database stores,
> unless that is the journal, and if so should be generic for all such use
> case consumers.
>
> One other thing that can be done in the interim, though not I would prefer
> the journal method, would be to update the quota using the existing dirty
> flag, but for files instead of parents. That way, some form of tracking is
> built in. Why I state this is, if the brick went offline in between, then
> the client gets a -ve RPC result, and hence the client would have to take
> some action, so this would be the last write that did not make it through,
> or a truncate that did not make it through. In such cases the dirty flag on
> the file should help resolving the space accounting. One problem though is
> remembering the old and new sizes even in this approach.
>
>> *_Approach-1)_*
>> T1 - For each file/dir 'file_x', create a contribution extended
>> attribute say 'trusted.glusterfs.quota.<uid>-contri'
>> T2 - In a lookup/write operation read the actual size from the stat-buf,
>> add the delta size to the contribution xattr
>> T3 - Create a file .glusterfs/quota/users/<uid>.
>>       Update size extended attribute say 'trusted.glusterfs.quota.size'
>> by adding the delta size calculated in T2
>>
>> Same for group quotas a size xattr is updated under
>> .glusterfs/quota/groups/<gid>.
>>
>> cons:
>>      If the brick crashes after executing T2 and before T3. Now
>> accounting information is in-correct.
>>      To recover and correct the accounting information, entire
>> file-systems needs to be crawled to fix the trusted.glusterfs.quota.size
>>      value by summing up the contribution of all files with UID. But is
>> a slow process.
>>
>>
>> *_Approach-2)_*
>> T1 - For each file/dir 'file_x', create a contribution extended
>> attribute say 'trusted.glusterfs.quota.<uid>-contri'
>> T2 - create a directory '.glusterfs/quota/users/<uid>'
>>       create a hardlink for file file_x under this directories
>> T3 - In a lookup/write operation, set dirty flag
>> 'trusted.glusterfs.quota.dirty' for directory
>> '.glusterfs/quota/users/<uid>'
>> T4 - Read the actual size of a file from the stat-buf, add the delta
>> size to the contribution xattr
>> T5 - update size extended attribute say for directory
>> '.glusterfs/quota/users/<uid>'
>> T6 - unset the dirty flag
>>
>> Same for group quotas a size xattr is updated under
>> .glusterfs/quota/groups/<gid>.
>>
>> Problem of approach 1 of crawling entire brick is solved by only
>> crawling the directory which is set dirty.
>>
>> cons:
>>      Need to make sure that the hard-link for a file is consistent when
>> having another hardlinks
>>      under .glusterfs/quota/users/<uid> and .glusterfs/quota/groups/<gid>
>>
>>
>> *_Approach-3)_*
>> T1 - For each file/dir 'file_x', update a contribution entry in the
>> SQL-LITE DB (Create a DB file under .glusterfs/quota/)
>> T2 - In a lookup/write operation read the actual size from the statbuf,
>> add the update the size in the USER-QUOTA schema in the DB
>> T3 - In a lookup/write operation, set dirty flag
>> 'trusted.glusterfs.quota.dirty' for directory
>> '.glusterfs/quota/users/<uid>'
>>
>> Atomicity problem found in approach 1 and 2 is solved by using DB
>> transactions.
>>
>> Note: need to test the consistency of the SQL-LITE DB.
>>
>> We feel approach-3 is more simpler and efficient way of implementing
>> user/group quotas.
>>
>>
>> Thanks,
>> Vijay
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel



-- 
    Venky