[Gluster-devel] glfs_resolve new file force lookup

[Rudra Siva]

If I understand the structure correctly, the lookup itself generally
may not pose a problem - specific flags triggering the over-the-wire
request for me at this time I wish to suppress (the fop on the backend
is fully capable of creating the file if it does not exist) - the
lookup drives through the xlators specifically DHT.

It looks like most of the stuff for the hashed subvolume is inside the
DHT - I will look into it.

On Mon, Nov 24, 2014 at 12:56 AM, Soumya Koduri <skoduri at redhat.com> wrote:
> Hi Siva,
>
> On 11/22/2014 08:44 AM, Rudra Siva wrote:
>>
>> Thanks for the response. In my case, I am trying to avoid doing the
>> network level lookup - since I use the same resolve only pass a null
>> for the attribute structure - essentially in my case, it is an atomic
>> multiple object read/write so I only want to resolve to the specific
>> bricks and then dispatch the requests in a single rpc.
>>
>> Once the files can be resolved, a single read/write is mostly sent to
>
> When do you confirm that the files can be resolved?
> As Raghavendra has mentioned, for the files to be newly created, it is
> necessary to do forceful lookup and verify if the file is not already
> present/can be created before you confirm or say that the files are
> resolved.
>
> The inode_tables/inode entries which you are referring to are maintained on
> the client side and AFAIK doesnt store any brick-related information.
> Each volume will have a unique inode_table and the inode entries in that
> table are populated as and when the client does lookups on the files present
> in that volume (or rather try to resolve them) on the back-end.
>
> So incase if the client does not find any inode entry for a particular file,
> its not *necessary* that that file is not present on the back-end too. To
> confirm that and set gfid (if needed incase of files with missing gfids), it
> needs to do force_lookup in such cases.
>
> Hope this helps.
>
> Thanks,
> Soumya
>
>
>
>> the back-end. Most places seem to be calling with the attribute
>> structure which translates into a force lookup, is it okay to move the
>> force out of the if-block so it can apply to files that we are not
>> interested in? That way it will help avoid doing the lookups for
>> multiple files.
>>
>> I was curious to know how the inode, table maps to the brick -
>> presently I have 1 brick but planning to play with a couple of bricks
>> - the returned loc has different inodes for each file, and inode table
>> is common - I do see one spurious lookup over the network (eg. I used
>> lookup for 100 files, there was only 1 lookup generated over the
>> network) - with the force, it becomes 100 lookups which simply return
>> that the file does not exist.
>>
>> --Siva
>>
>> On Fri, Nov 21, 2014 at 1:21 PM, RAGHAVENDRA TALUR
>> <raghavendra.talur at gmail.com> wrote:
>>>
>>> Hi Rudra,
>>>
>>> The inode and inode table data structures here represent the in-memory
>>> inode
>>> on the client side.(gfapi)
>>>
>>> When we are trying to create a new file, it becomes
>>> *necessary* that we confirm with the backend if it can be created, hence
>>> a force lookup.
>>>
>>> Only case where we avoid a force lookup is when in-memory inode is
>>> present(means we had resolved this recently and have all the stat data)
>>> and it is not the last component in the path. Say "b" in the path
>>> "/a/b/c".
>>>
>>> Please do tell if that does not clarify your question.
>>>
>>> Raghavendra Talur
>>>
>>>
>>> On Fri, Nov 21, 2014 at 5:50 PM, Rudra Siva <rudrasiva11 at gmail.com>
>>> wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> A new file create does not honour the force lookup avoidance - in my
>>>> case I am not interested in the attributes or forcing a lookup, just
>>>> need the inode details - is there a specific reason why !force_lookup
>>>> is not outside the block?
>>>>
>>>> https://github.com/gluster/glusterfs/blob/master/api/src/glfs-resolve.c
>>>>
>>>> 270:     if (!force_lookup) {
>>>>
>>>> suggested fix:
>>>>
>>>> move this to outside of if-else to line 296 - so it applies to
>>>> existing as well as new files.
>>>>
>>>> Can someone explain what do the inode and inode table data structures
>>>> represent?
>>>> --
>>>> -Siva
>>>> _______________________________________________
>>>> Gluster-devel mailing list
>>>> Gluster-devel at gluster.org
>>>> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Raghavendra Talur
>>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>>
>



-- 
-Siva