[Gluster-devel] new test failure in tests/basic/mount-nfs-auth.t

[Niels de Vos]

On Wed, May 06, 2015 at 10:12:02AM +0200, Niels de Vos wrote:
> On Wed, May 06, 2015 at 07:03:45AM +0530, Pranith Kumar Karampuri wrote:
> > Niels,
> >         Any ideas?
> > 
> > http://build.gluster.org/job/rackspace-regression-2GB-triggered/8462/consoleFull
> > 
> > mount.nfs: access denied by server while mounting slave46.cloud.gluster.org:/patchy
> > mount.nfs: access denied by server while mounting slave46.cloud.gluster.org:/patchy
> > mount.nfs: access denied by server while mounting slave46.cloud.gluster.org:/patchy
> > dd: closing output file `/mnt/nfs/0/test-big-write': Input/output error
> > [20:48:27] ./tests/basic/mount-nfs-auth.t ..
> > not ok 33
> 
> Hmm, I suspect a race in the new auth-cache/netgroup stuff. The nfs.log
> contains these:
> 
>     [2015-05-05 20:40:54.125431] E [auth-cache.c:117:auth_cache_lookup] (--> /build/install/lib/libglusterfs.so.0(_gf_log_callingfn+0x240)[0x7f93a0823ae4] (--> /build/install/lib/glusterfs/3.8dev/xlator/nfs/server.so(auth_cache_lookup+0xcb)[0x7f9392a13c11] (--> /build/install/lib/glusterfs/3.8dev/xlator/nfs/server.so(is_nfs_fh_cached+0x5b)[0x7f9392a13f65] (--> /build/install/lib/glusterfs/3.8dev/xlator/nfs/server.so(mnt3_check_cached_fh+0x38)[0x7f93929de49b] (--> /build/install/lib/glusterfs/3.8dev/xlator/nfs/server.so(_mnt3_authenticate_req+0x206)[0x7f93929de6c0] ))))) 0-nfs: invalid argument: cache->cache_dict
>     [2015-05-05 20:40:54.125662] I [dict.c:467:dict_get] (--> /build/install/lib/libglusterfs.so.0(_gf_log_callingfn+0x240)[0x7f93a0823ae4] (--> /build/install/lib/libglusterfs.so.0(dict_get+0x67)[0x7f93a081a7e4] (--> /build/install/lib/glusterfs/3.8dev/xlator/nfs/server.so(auth_cache_lookup+0x21d)[0x7f9392a13d63] (--> /build/install/lib/glusterfs/3.8dev/xlator/nfs/server.so(cache_nfs_fh+0x171)[0x7f9392a1417a] (--> /build/install/lib/glusterfs/3.8dev/xlator/nfs/server.so(_mnt3_authenticate_req+0x34c)[0x7f93929de806] ))))) 0-dict: !this || key=()
> 
> It seems that "struct auth_cache" is not completely initialized in some
> code paths... I'll look into that and will send a patch soon.

This should address the issue, I think:

    http://review.gluster.org/10600
    nfs: allocatie the auth_cache->cache_dict on auth_cache_init()

Do you know how easily this is reproducible? My guess is that it is
*very* hard to hit this, but we could run the test in a loop and see if
this patch fixes it (or just makes it even less likely to occur).

Thanks,
Niels
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150506/fd594852/attachment.sig>