[Gluster-devel] GlusterFS User and Group Quotas

[Shyam]

On 12/08/2015 04:32 AM, Vijaikumar Mallikarjuna wrote:
> Hi All,
>
> Below is the design for '*GlusterFS User and Group Quotas', *please
> provide your feedback on the same.
>
>
> *_Developers:_*
> Vijaikumar.M and Manikandan.S
>
>
> *_Introduction:_*
> User and Group quotas is to limit the amount of disk space for a
> specified user/group ID.
> This documents provides some details about how the accounting (marker
> xlator) can be done
> for user and group quotas
>
>
> *_Design:_*
> We have three different approaches, each has pros and cons
>

my 2 c's,

The problem below is that of maintaining a journal as I see it. With the 
addition of pre/post size awareness, so that the delta can be reflected 
in the quota appropriately.

With the above in mind, I would say, steer clear on any database stores, 
unless that is the journal, and if so should be generic for all such use 
case consumers.

One other thing that can be done in the interim, though not I would 
prefer the journal method, would be to update the quota using the 
existing dirty flag, but for files instead of parents. That way, some 
form of tracking is built in. Why I state this is, if the brick went 
offline in between, then the client gets a -ve RPC result, and hence the 
client would have to take some action, so this would be the last write 
that did not make it through, or a truncate that did not make it 
through. In such cases the dirty flag on the file should help resolving 
the space accounting. One problem though is remembering the old and new 
sizes even in this approach.

> *_Approach-1)_*
> T1 - For each file/dir 'file_x', create a contribution extended
> attribute say 'trusted.glusterfs.quota.<uid>-contri'
> T2 - In a lookup/write operation read the actual size from the stat-buf,
> add the delta size to the contribution xattr
> T3 - Create a file .glusterfs/quota/users/<uid>.
>       Update size extended attribute say 'trusted.glusterfs.quota.size'
> by adding the delta size calculated in T2
>
> Same for group quotas a size xattr is updated under
> .glusterfs/quota/groups/<gid>.
>
> cons:
>      If the brick crashes after executing T2 and before T3. Now
> accounting information is in-correct.
>      To recover and correct the accounting information, entire
> file-systems needs to be crawled to fix the trusted.glusterfs.quota.size
>      value by summing up the contribution of all files with UID. But is
> a slow process.
>
>
> *_Approach-2)_*
> T1 - For each file/dir 'file_x', create a contribution extended
> attribute say 'trusted.glusterfs.quota.<uid>-contri'
> T2 - create a directory '.glusterfs/quota/users/<uid>'
>       create a hardlink for file file_x under this directories
> T3 - In a lookup/write operation, set dirty flag
> 'trusted.glusterfs.quota.dirty' for directory '.glusterfs/quota/users/<uid>'
> T4 - Read the actual size of a file from the stat-buf, add the delta
> size to the contribution xattr
> T5 - update size extended attribute say for directory
> '.glusterfs/quota/users/<uid>'
> T6 - unset the dirty flag
>
> Same for group quotas a size xattr is updated under
> .glusterfs/quota/groups/<gid>.
>
> Problem of approach 1 of crawling entire brick is solved by only
> crawling the directory which is set dirty.
>
> cons:
>      Need to make sure that the hard-link for a file is consistent when
> having another hardlinks
>      under .glusterfs/quota/users/<uid> and .glusterfs/quota/groups/<gid>
>
>
> *_Approach-3)_*
> T1 - For each file/dir 'file_x', update a contribution entry in the
> SQL-LITE DB (Create a DB file under .glusterfs/quota/)
> T2 - In a lookup/write operation read the actual size from the statbuf,
> add the update the size in the USER-QUOTA schema in the DB
> T3 - In a lookup/write operation, set dirty flag
> 'trusted.glusterfs.quota.dirty' for directory '.glusterfs/quota/users/<uid>'
>
> Atomicity problem found in approach 1 and 2 is solved by using DB
> transactions.
>
> Note: need to test the consistency of the SQL-LITE DB.
>
> We feel approach-3 is more simpler and efficient way of implementing
> user/group quotas.
>
>
> Thanks,
> Vijay
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>