[Gluster-users] GlusterFS 3.1.2 - df and du irregularities

[Amar Tumballi]

Hi Nic,

Answers inline.


> I'm having a weird issue where my gluster volumes are being reported as
> full by df but du shows only 18GB of actual data in them (which is less than
> 1% of the capacity of the volumes).  The client shows 2/3 of the volume is
> full, only replicate-3 is currently not "full" and given how this problem
> started I expect it to be full by the time I come into the office tomorrow
> morning.  I am runnung glusterfs 3.1.2 on Ubuntu 10.10. Does anyone have any
> idea why this might be

happening?
>
>
GlusterFS exports in a thin provision way by design, which causes this
behavior. You are seeing this behavior because of the fundamental difference
in the implementation of the two commands. (df internally uses 'statvfs(2)',
and du uses 'readdir(), stat()' recursively to find out the size under a
given directory).

To understand it better, I will give you an example from my laptop

root at supernova:~# gluster volume create test-df
supernova:/tmp/export/test-df
Creation of volume test-df has been successful. Please start the volume to
access data.
root at supernova:~# gluster volume start test-df
Starting volume test-df has been successful
root at supernova:~# df -h /tmp/export/test-df
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             9.2G  6.3G  2.5G  72% /
root at supernova:~# mount -t glusterfs localhost:/test-df /mnt/glusterfs
root at supernova:~# df -h /mnt/glusterfs
Filesystem            Size  Used Avail Use% Mounted on
localhost:/test-df    9.2G  6.3G  2.5G  72% /mnt/glusterfs
root at supernova:~# du -sh /mnt/glusterfs/
4.0K /mnt/glusterfs/

That means the export directory used for the volume resides on a filesystem
which is already 72% full. So, when the filesystem is mounted, the 'df' of
the filesystem shows 72% full, and there are no files in glusterfs volume.

Hope that explains. So, to set up glusterfs properly, make sure your backend
exports are dedicated to GlusterFS and some other process is not filling it
up.

Regards,
Amar

PS: this should be to our FAQ in wiki, so people will be aware of these type
of issues due to thin provision factors.