[Gluster-devel] Dht readdir filtering out names

[Prashanth Pai]

readdirp needs to be explicitly disabled at 3 places if I remember correctly.
At fuse mount option, at dht and at md-cache level.

 -Prashanth Pai

----- Original Message -----
> From: "Pranith Kumar Karampuri" <pkarampu at redhat.com>
> To: "Raghavendra Gowdappa" <rgowdapp at redhat.com>
> Cc: "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Friday, 30 September, 2016 9:53:44 AM
> Subject: Re: [Gluster-devel] Dht readdir filtering out names
> 
> 
> 
> On Fri, Sep 30, 2016 at 9:50 AM, Raghavendra Gowdappa < rgowdapp at redhat.com >
> wrote:
> 
> 
> 
> 
> ----- Original Message -----
> > From: "Pranith Kumar Karampuri" < pkarampu at redhat.com >
> > To: "Raghavendra Gowdappa" < rgowdapp at redhat.com >
> > Cc: "Shyam Ranganathan" < srangana at redhat.com >, "Nithya Balachandran" <
> > nbalacha at redhat.com >, "Gluster Devel"
> > < gluster-devel at gluster.org >
> > Sent: Friday, September 30, 2016 9:15:04 AM
> > Subject: Re: Dht readdir filtering out names
> > 
> > On Fri, Sep 30, 2016 at 9:13 AM, Raghavendra Gowdappa < rgowdapp at redhat.com
> > >
> > wrote:
> > 
> > > dht_readdirp_cbk has different behaviour for directories and files.
> > > 
> > > 1. If file, pick the dentry (passed from subvols as part of readdirp
> > > response) if the it corresponds to data file.
> > > 2. If directory pick the dentry if readdirp response is from
> > > hashed-subvol.
> > > 
> > > In all other cases, the dentry is skipped and not passed to higher
> > > layers/application. To elaborate, the dentries which are ignored are:
> > > 1. dentries corresponding to linkto files.
> > > 2. dentries from non-hashed subvols corresponding to directories.
> > > 
> > > Since the behaviour is different for different filesystem objects, dht
> > > needs ia_type to choose its behaviour.
> > > 
> > > ----- Original Message -----
> > > > From: "Pranith Kumar Karampuri" < pkarampu at redhat.com >
> > > > To: "Shyam Ranganathan" < srangana at redhat.com >, "Raghavendra Gowdappa"
> > > > <
> > > rgowdapp at redhat.com >, "Nithya Balachandran"
> > > > < nbalacha at redhat.com >
> > > > Cc: "Gluster Devel" < gluster-devel at gluster.org >
> > > > Sent: Friday, September 30, 2016 8:39:28 AM
> > > > Subject: Dht readdir filtering out names
> > > > 
> > > > hi,
> > > > In dht_readdirp_cbk() there is a check about skipping files
> > > without
> > > > ia_type. Could you help me understand why this check is added? There
> > > > are
> > > > times when users have to delete gfid of the entries and trigger
> > > > something
> > > > like 'find . | xargs stat' to heal the gfids. This case would fail if
> > > > we
> > > > skip entries without gfid, if the lower xlators don't send stat
> > > information
> > > > for them.
> > > 
> > > Probably we can make readdirp_cbk not rely on ia_type and pass _all_
> > > dentries received by subvols to application without filtering. However we
> > > should make this behaviour optional and use this only for recovery
> > > setups.
> > > If we don't rely on ia_type (during non error scenarios), applications
> > > end
> > > up seeing duplicate dentries in readdir listing.
> > > 
> > 
> > That means dht_readdir() gives duplicate entries? As per the code it seems
> > like it...
> 
> No. It follows the filtering logic of "pick dentry only from hashed subvol".
> This logic doesn't need ia_type. Now, that you brought the topic of
> dht_readdir, I've another solution for your use case (Basically don't use
> readdirp :) ):
> 
> 1. mount glusterfs with "--use-readdirp=no" option.
> 2. disable md-cache/stat-prefetch as it converts all readdir calls into
> readdirp calls
> 
> Probably the ones in dht as well? i.e. use-readdirp option.
> 
> 
> 
> Use this only for recovery setups :).
> 
> > 
> > 
> > > 
> > > > 
> > > > --
> > > > Pranith
> > > > 
> > > 
> > > regards,
> > > Raghavendra
> > > 
> > 
> > 
> > 
> > --
> > Pranith
> > 
> 
> 
> 
> --
> Pranith
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel