<div dir="ltr"><br><div class="gmail_quote"><div dir="ltr">On Thu, Mar 8, 2018 at 10:17 PM Daniel Gryniewicz <<a href="mailto:dang@redhat.com">dang@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 03/07/2018 10:21 PM, Kinglong Mee wrote:<br>
> On 2018/3/7 21:10, Daniel Gryniewicz wrote:<br>
>> On 03/06/2018 10:10 PM, Kinglong Mee wrote:<br>
>>> On 2018/3/7 10:59, Kinglong Mee wrote:<br>
>>>> When using nfsv3 on glusterfs-3.13.1-1.el7.x86_64 and nfs-ganesha-2.6.0-0.2rc3.el7.centos.x86_64,<br>
>>>> I gets strange "Invalid argument" when writing file.<br>
>>>><br>
>>>> 1. With quota disabled;<br>
>>>> nfs client mount nfs-ganesha share, and do 'll' in the testing directory.<br>
>>>><br>
>>>> 2. Enable quota;<br>
>>>> # getfattr -d -m . -e hex /root/rpmbuild/gvtest/nfs-ganesha/testfile92<br>
>>>> getfattr: Removing leading '/' from absolute path names<br>
>>>> # file: root/rpmbuild/gvtest/nfs-ganesha/testfile92<br>
>>>> trusted.gfid=0xe2edaac0eca8420ebbbcba7e56bbd240<br>
>>>> trusted.gfid2path.b3250af8fa558e66=0x39663134343566662d653530332d343831352d396635312d3236633565366332633137642f7465737466696c653932<br>
>>>> trusted.glusterfs.quota.9f1445ff-e503-4815-9f51-26c5e6c2c17d.contri.3=0x00000000000002000000000000000001<br>
>>>><br>
>>>> Notice: testfile92 without trusted.pgfid xattr.<br>
>>><br>
>>> The trusted.pgfid will be created by the next name lookup; nameless lookup don't create it.<br>
>>><br>
>>>> 3. restart glusterfs volume by "gluster volume stop/start gvtest"<br>
>>><br>
>>> Restarting glusterfsd here cleanup all inode cache from memory;<br>
>>> after starting, inode of testfile92's parent is NULL.<br>
>>><br>
>>>> 4. echo somedata > testfile92<br>
>>><br>
>>> Because, nfs-ganesha and nfs client has cache for testfile92,<br>
>>> before write fops, no name lookup happens that trusted.pgfid is not created for testfile92.<br>
>>><br>
>>> Quota_writev call quota_build_ancestry building the ancestry in quota_check_limit,<br>
>>> but testfile92 doesn't contain trusted.pgfid, so that write fops failed with Invalid argument.<br>
>>><br>
>>> I have no idea of fixing this problem, any comments are welcome.<br>
>>><br>
>><br>
>> I think, ideally, Gluster would send an invalidate upcall under the circumstances, causing Ganesha do drop it's cached entry.<br>
><br>
> It doesn't work.<br>
> I try to restarting nfs-ganesha, and echo data to testfile92, the problem also exists.<br>
><br>
> After nfs-ganesha restart,<br>
> 1. A GETATTR is send from nfs-client for the testfile92, ganesha translates it to nameless lookup.<br>
> 2. A ACCESS gets attributes from nfs-ganesha's cache (cached by #1).<br>
> 3. A SETATTR sets the testfile92's size to 0, ganesha translates it to setattr fop.<br>
> 4. A WRITE also get Invalid argument error.<br>
><br>
> If ganesha drops its cache, nfs client may write file by filehandle;<br>
> ganesha lookup it by nameless lookup from glusterfs,<br>
> so that, trusted.pgfid isn't created too.<br>
><br>
> I think, a name lookup is needed for testfile92 after quota enable.<br>
><br>
> thanks,<br>
> Kinglong Mee<br>
><br>
<br>
If a name lookup is needed, then gluster cannot provide NFS semantics in<br>
these circumstances. NFS *requires* that access by handle continue to<br>
work once the client has a handle. In fact, there's no way to get a<br>
name for this, since the name doesn't exist anywhere in the handle (by<br>
design, since a name is an attribute of a dirent, not of a handle/inode).<br>
<br>
So, this likely is a bug in Gluster, and needs to be fixed there. Would<br>
it be possible to enable quota globally at the start as a workaround?<br></blockquote><div><br></div><div>
<pre>Quota is a function that can be enabled at any time.
If enable quota immediately after creating filesystem (contains only one directory "/"),
the trusted.pgfid extend-attribute is created automatically when creating file,
this problem is not exit.
thanks,
Kinglong Mee</pre>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Daniel<br>
<br>
<br>
</blockquote></div></div>