[Gluster-devel] Three Issues Confused me recently

[Susant Palai]

Comment inline.

----- Original Message -----
> From: christ1016 at sina.com
> To: "gluster-devel" <gluster-devel at gluster.org>
> Sent: Thursday, 25 June, 2015 7:56:45 PM
> Subject: [Gluster-devel] Three Issues Confused me recently
> 
> 
> 
> Hi, everyone!
> 
> 
> 
> 
> Nowadays, t here are three issues confusing me recently when I used the
> glusterfs to save huge datas. Like below:
> 1) Is there any reason for reserving 10% free space of each brick in the
> volume ? And Can I do not reserve the 10% free space of each brick in the
> volume? You know, I will use the glusterfs to save huge surveillance videos,
> so each brick will be set a large disk space. If each brick will be reserved
> 10% free space, it must led to low usage of disk and waster many disk
> spaces.
> 

"10%" is the default and it can be modified by the cluster.min-free-disk option.

e.g gluster v set _VOL_NAME_ min-free-disk 8GB


 *On the question of what should be this cluster.min-free-disk's value?*
  
  Cluster.min-free-disk: The min-free-disk setting establishes a data threshold for each brick in a volume. The primary intention of this is to ensure that there is adequate space to perform self-heal and rebalance operations, both of which require disk overhead. The min-free-disk value is taken into account when it is already exceeded before a file is being written. When that is the case, the DHT algorithm will choose to write the file instead to another brick where min-free-disk is not exceeded, and will write a 0-byte link-to file on the brick where min-free-disk is exceeded and where the file was originally hashed. This link-to file contains metadata to point the client to the brick where the data was actually written.  Because min-free-disk is only considered after it has been exceeded, and because the DHT algorithm makes no other consideration of available space on a brick, it is possible to write a large file that will exceed the space on the brick it is hashed to even while another brick has enough space to hold the file. This would result in an I/O error to the client.

   So if you know you routinely write files up to  nGB size, then min-free-disk can be set to arbitrarily a little larger value than n. For example if your file size is 5GB which is at the high end of the file sizes you will be writing, then 
you might consider setting min-free-disk to be 8GB. Doing this will ensure that the file will go to a brick with enough available space (assuming one exist).


> 2) Will it appear some exceptions when the filesystem, like xfs, ext4, had
> been written fully?
> 
As I already mentioned above, the new file creation will be redirected to a different brick with adequate space considering min-free-disk is exceeded. 

> 3) Is it natural that a very high cpu usage when the directory quota is
> enabled ? (glusterfs 3.6.2)
> 
CCing quota team for this.

> And is there any solution to avoid it ?
> 
> 
> I am very appreciate for your help, thanks very much.
> 
> 
> 
> 
> 
> 
> 
> Best regards.
> 
> 
> 
> 
> 
> 
> 
> Louis
> 
> 2015/6/25
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>