[Gluster-users] 3.1.2 with "No such file" and "Invalid argument" errors

[Steve Wilson]

I'm running a pair of replicated/distributed GlusterFS 3.1.2 servers, 
each with 8 bricks.  Here's the command I used to create the data volume:

    gluster volume create post replica 2 transport tcp
    pablo:/gluster/01/brick stanley:/gluster/01/brick \
    pablo:/gluster/02/brick stanley:/gluster/02/brick \
    pablo:/gluster/03/brick stanley:/gluster/03/brick \
    pablo:/gluster/04/brick stanley:/gluster/04/brick \
    pablo:/gluster/05/brick stanley:/gluster/05/brick \
    pablo:/gluster/06/brick stanley:/gluster/06/brick \
    pablo:/gluster/07/brick stanley:/gluster/07/brick \
    pablo:/gluster/08/brick stanley:/gluster/08/brick

The clients mount the volume using the gluster FUSE client, again 
version 3.1.2.  Since GlusterFS doesn't yet support exporting 
sub-directories directly (it does now for NFS, though, I believe), I 
mount the volume as:
     /net/.post-volume
and then bind mount a couple of lower sub-directories as:
     /net/post (bound to /net/.post-volume/home)
     /net/post-scratch (bound to /net/.post-volume/scratch)

I am using this volume for user home directories.

That's the background information... now for the problem.  There are 
certain files that are almost guaranteed to go missing after less than a 
few hours of a user starting up a login session.  We use Ubuntu with GDM 
as a login greeter and both KDE and Gnome desktops.  Here are a few of 
the files that usually go missing:

    ls: .dmrc: No such file or directory
    ls: .pine-debug2: Invalid argument
    ls: .xsession-errors.old: No such file or directory
    ls: .recently-used.xbel: No such file or directory

Taking .dmrc, for example, I can stop the volume, then remove all 
occurrences of this file on all the bricks on both servers.  Then start 
up the volume again, start the user session again, and after a period of 
time (I'm not sure how long), the file comes up missing again.  When the 
file is missing, this is what it looks like on the bricks themselves:

server 1:

    root at stanley:/gluster# ls -l ./07/brick/home/stevew/.dmrc
    ---------T 1 stevew sysmgr 0 2011-01-28 08:24
    ./07/brick/home/stevew/.dmrc
    root at stanley:/gluster# ls -l ./02/brick/home/stevew/.dmrc
    -rw-r--r-- 1 stevew sysmgr 25 2011-01-28 08:25
    ./02/brick/home/stevew/.dmrc

server 2:

    root at pablo:/gluster# ls -l ./07/brick/home/stevew/.dmrc
    ---------T 1 stevew sysmgr 0 2011-01-28 08:24
    ./07/brick/home/stevew/.dmrc
    root at pablo:/gluster# ls -l ./02/brick/home/stevew/.dmrc
    -rw-r--r-- 1 stevew sysmgr 25 2011-01-28 08:25
    ./02/brick/home/stevew/.dmrc


Pertinent log messages are:

server 1:

    net-.post-volume.log:[2011-01-28 08:16:28.899544] W
    [fuse-bridge.c:2432:fuse_xattr_cbk] glusterfs-fuse: 6318974:
    GETXATTR() /home/stevew/.dmrc => -1 (Invalid argument)


server 2:

    net-.post-volume.log:[2011-01-28 08:10:58.74848] W
    [fuse-bridge.c:2432:fuse_xattr_cbk] glusterfs-fuse: 2428788:
    GETXATTR() /home/stevew/.dmrc => -1 (No such file or directory)


Any thoughts or ideas?

Thanks!

Steve
-- 

Steven M. Wilson, Systems and Network Manager
Markey Center for Structural Biology
Purdue University
(765) 496-1946