[Gluster-devel] [NFS-Ganesha-Devel] Re: Problems about cache virtual glusterfs ACLs for ganesha in md-cache

[Soumya Koduri]

On 10/12/18 5:55 PM, Kinglong Mee wrote:
> On 2018/10/12 14:34, Soumya Koduri wrote:> On 10/12/18 7:22 AM, Kinglong Mee wrote:
>>> On 2018/10/11 19:09, Soumya Koduri wrote:
>>>> NFS-Ganesha's md-cache layer already does extensive caching of attributes and ACLs of each file looked upon. Do you see any additional benefit with turning on gluster md-cache as well?  More replies inline..
>>>
>>> Yes, I think.
>>>
>>> The logical is different between md-cache and ganesha's cache,
>>> Ganesha caches xattr data depends on timeout, if timeout, ganesha get it from back-end glusterfs;
>>> Md-cache caches depneds on timeout too, but md-cache can delay the timeout for some cases.
>>
>> Could you please list out which xattrs fall into that category. AFAIK, like iatt, even all xattrs are invalidated post timeout in md-cache xlator.
> 
> The iatt's expire time is ->ia_time + timeout, xatt's expire time is ->xa_time + timeout.
> Most FOPs reply (read/write/truncate...) contains the postattr incidentally,
> and update the ->ia_time.
> But, ->xa_time cannot be updated as ->ia_time now, it only be updated at mdc_lookup_cbk now.
> 
> I will add a case of update ->xa_time if file's mtiem/ctime does not change
> when updating the ->ia_time.
> And, let stat/fstat/setattr/fsetattr/getxattr/fgetxattr's reply contains the xattr incidentally,
> and update the ->xa_time too.

+1

> 
>> By turning on both these caches, the process shall consume more memory and CPUs to store and invalidate same set of attributes at two different layers right. Do you see much better performance when compared to that cost?
> 
> I think md-cache supporting cache virtual ACLs is a function update,
> ganesha can use it by the new added option, or not.
> 
> If setting a smaller timeout for ganesha's cache, it timeout frequently.
> But, md-cache can return the cached xattr.
> 
> I do not have a comparing data between them.
> After testing, we can chose the better combination,
> 1. md-cache on and ganesha's cache on;
> 2. md-cache on but ganesha's cache off;
> 3. md-cache off but ganesha's cache on.
> 

yes..Please share your findings if you get to test these combinations.

Thanks,
Soumya