[Gluster-devel] Regarding the size parameter in readdir(p) fops

[Krutika Dhananjay]

Sorry for the late reply, Niels. 

I tried what you suggested yesterday and yes, like you predicted, the extra entries are getting dropped by fuse-bridge, and the readdir-ahead translator seems to be showing the same behavior. 
Thanks for the idea. :) 

-Krutika 
----- Original Message -----

> From: "Niels de Vos" <ndevos at redhat.com>
> To: "Krutika Dhananjay" <kdhananj at redhat.com>
> Cc: "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Tuesday, May 19, 2015 3:12:20 PM
> Subject: Re: [Gluster-devel] Regarding the size parameter in readdir(p) fops

> On Tue, May 19, 2015 at 05:12:35AM -0400, Krutika Dhananjay wrote:
> > Hi,
> >
> > The following patch fixes an issue with readdir(p) in shard xlator:
> > http://review.gluster.org/#/c/10809/ whose details can be found in the
> > commit message.
> >
> > One side effect of this is that from shard xlator, the size of the dirents
> > list returned to the translators above it could be greater than the
> > requested size in the wind path (thanks to Pranith for pointing this out
> > during the review of this patch), with the worst-case scenario returning
> > (2 * requested_size) worth of entries.
> > For example, if fuse requests readdirp with 128k as the size, in the worst
> > case, 256k worth of entries could be unwound in return.
> > How important is it to strictly adhere to this size limit in each iteration
> > of readdir(p)? And what are the repercussions of such behavior?
> >
> > Note:
> > I tried my hand at simulating this issue on my volume but I have so far
> > been unsuccessful at hitting this test case.
> > Creating large number of files on the root of a sharded volume, triggering
> > readdirp on it until ".shard" becomes the last entry read in a given
> > iteration, winding the next
> > readdirp from shard xlator, and then concatenating the results of two
> > readdirps into one is proving to be an exercise in futility.
> > Therefore, I am asking this question here to know what could happen "in
> > theory" in such situations.

> How about modifying xlators/mount/fuse/src/fuse-bridge.c and increasing
> the size of the readdir(p) argument there? The FUSE kernel side would
> not know about the increased size, but the other xlators would request
> a bigger size in subsequent readdir(p) FOPs.

> I suspect that the additional dentries in readdir(p) get dropped, and
> the next getdents() call from the kernel continues are the offset of the
> readdir(p) of the last returned dentry.

> Niels
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150528/3f7ea7ed/attachment-0001.html>