[Gluster-devel] [Nfs-ganesha-devel] nfsv3 client writing file gets Invalid argument on glusterfs with quota on

[Daniel Gryniewicz]

On 03/06/2018 10:10 PM, Kinglong Mee wrote:
> On 2018/3/7 10:59, Kinglong Mee wrote:
>> When using nfsv3 on glusterfs-3.13.1-1.el7.x86_64 and nfs-ganesha-2.6.0-0.2rc3.el7.centos.x86_64,
>> I gets strange "Invalid argument" when writing file.
>>
>> 1. With quota disabled;
>> nfs client mount nfs-ganesha share, and do 'll' in the testing directory.
>>
>> 2. Enable quota;
>> # getfattr -d -m . -e hex /root/rpmbuild/gvtest/nfs-ganesha/testfile92
>> getfattr: Removing leading '/' from absolute path names
>> # file: root/rpmbuild/gvtest/nfs-ganesha/testfile92
>> trusted.gfid=0xe2edaac0eca8420ebbbcba7e56bbd240
>> trusted.gfid2path.b3250af8fa558e66=0x39663134343566662d653530332d343831352d396635312d3236633565366332633137642f7465737466696c653932
>> trusted.glusterfs.quota.9f1445ff-e503-4815-9f51-26c5e6c2c17d.contri.3=0x00000000000002000000000000000001
>>
>> Notice: testfile92 without trusted.pgfid xattr.
> 
> The trusted.pgfid will be created by the next name lookup; nameless lookup don't create it.
> 
>> 3. restart glusterfs volume by "gluster volume stop/start gvtest"
> 
> Restarting glusterfsd here cleanup all inode cache from memory;
> after starting, inode of testfile92's parent is NULL.
> 
>> 4. echo somedata > testfile92
> 
> Because, nfs-ganesha and nfs client has cache for testfile92,
> before write fops, no name lookup happens that trusted.pgfid is not created for testfile92.
> 
> Quota_writev call quota_build_ancestry building the ancestry in quota_check_limit,
> but testfile92 doesn't contain trusted.pgfid, so that write fops failed with Invalid argument.
> 
> I have no idea of fixing this problem, any comments are welcome.
> 

I think, ideally, Gluster would send an invalidate upcall under the 
circumstances, causing Ganesha do drop it's cached entry.

Daniel