[Gluster-devel] lookup caching

[Olivier Le Cam]

Hi again,

Thanks for information.

Raghavendra G wrote:

> Metadata caching is done in stat-prefetch. But that is not suitable to 
> your use-case. in stat-prefetch stats of dentries are stored from the 
> period of readdir to close on directory fd.

Although this translators is not easy to understand for a new comer in 
glusterfs devel, I'll carefully read its code.

> Did you try with a very high cache-timeout value (say 5 minutes or 
> more)? Also a bigger cache-size might help.

Yes, I did. In my situation, the problem is indeed the latency caused by 
tons of stats. My guess is that it should be possible to reach better 
improvement with metadata caching than with the io-cache (files are 
small and can usually be fetched with only a single frame).

Well, now I have a crutial question regarding the way to wrote a 
translator. I used the rot-13 code as a starting point.

I have a "lookup" xlator fop and its associated callback function. Here 
is a part my (very simple) speudo code:

int32_t
mdc_lookup_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
                 int32_t op_ret, int32_t op_errno, inode_t *inode,
                 struct stat *stbuf, dict_t *dict,
                 struct stat *postparent)
{
         /* TODO: cache metadata */

         STACK_UNWIND_STRICT (lookup, frame, op_ret, op_errno, inode,
                 stbuf, dict, postparent);
}

int32_t
mdc_lookup (call_frame_t *frame, xlator_t *this, loc_t *loc,
             dict_t *xattr_req)
{
         uint8_t cached = 0;

         if (xattr_req == NULL)
                 goto out;

         /* TODO: check if we have cached metadata for "loc->inode"
            If so, prepare data and set cache to 1 */

         if (cached) {
                 /* How to "re-inject" metadata in the engine? */
                 STACK_UNWIND_STRICT (lookup, frame, -1, 0, NULL, NULL,
                 	NULL, NULL);
                 return 0;
         }

out :
         STACK_WIND (frame, mdc_lookup_cbk, FIRST_CHILD (this),
                 FIRST_CHILD (this)->fops->lookup, loc, xattr_req);
         return 0;
}

In mdc_lookup, I think I can stop going through the lower-level xlators 
with the help of STACK_UNWIND_STRICT. But how can I deliver cached 
metadata back to the engine at this step?

Thanks,
-- 
Olivier