[Gluster-devel] GlusterFS vs xfs w/ inode64 (was: Missing files?)

[Matthias Saou]

Liam Slusser <lslusser at gmail.com> wrote:

> I worked on this issue this morning and could find nothing that would
> indicate it wouldn't work.  I was down to 45 free inodes (says
> xfs_db) so i brought down one of the nodes and applied the inode64
> option to /etc/fstab and remount the partition and restarted
> gluster.  Everything appears to be working normally so i applied the
> same option to my other server, and again, everything is working
> normally. I'll let you know after we run with this for a few days but
> so far everything is fine and working normally.  I'm on Centos 5.3
> x86_64 btw.
> 
> An interesting note, after applying the inode64 option the "ifree"
> output after running xfs_db didn't actually change but the filesystem
> is working normally.  I found a bunch of posts on the interweb of
> people who had that exact experience.

It's not mounting with the inode64 option which is an issue, it's once
you get files or directories allocated with inodes > 32bits from our
experience. So what you need to do is test with files or directories
created once the filesystem has been mounted with the inode64 options
and do have their inode beyond the 32bit limit.

Here's an example of what we have on the server :
[...]
 3221490663 drwxrwsr-x  5 flumotion file   46 Mar 11 19:07 cust5
 3221491044 drwxrwsr-x  6 flumotion file   56 May 12 12:27 cust1
 3221495387 drwxrwsr-x  9 flumotion file  109 Mar 26  2008 cust2
 3221500135 drwxrwsr-x  5 flumotion file   46 May 22 11:36 cust8
 3221500510 drwxrwsr-x  3 flumotion file   21 Jan  2  2008 cust23
 3221500956 drwxrwsr-x  5 flumotion file   46 Jun 25 16:23 cust7
72107183169 drwxrwsr-x  3 flumotion file   26 Jul 10 13:38 cust4
98784439192 drwxrwsr-x  3 flumotion file   29 Jul  2 16:55 cust12

The last two directories are the ones which aren't accessible from the
glusterfs clients.

So in your case, if you had only 45 free inodes left, you should be
able to create 46 directories and get the 46th reproduce the problems.

Matthias