[Gluster-users] Performance with small files (again)

[Count Zero]

In my opinion, you will only see this improved when GlusterFS WAN Replication is available. How on earth is this related, you ask?

Here's what I think is happening in your case:

- NFS is not a distributed FS. There is one host, an NFS Server. When you access a file, the standard getAttr() happens (you can also monitor it with nfsstat), and you then start getting the file data.

- GlusterFS is distributed! There is the case that your file could be re-written on any of the nodes in the cluster, that host this file. In order to make sure you get the most up-to-date file, when you try to access the file, GlusterFS will check with ALL nodes. The more machines you have in the cluster, in theory, the longer this might take.

- How this is related to WAN Replication? In WAN replication this process takes much longer because it happens on the WAN, and makes GlusterFS unsuitable for WAN scenarios. I know some work is being done in this direction, and I have a feeling it will greatly benefit LAN scenarios as well, where very small files are involved. I guess some sort of state caching on the clients will happen, or some of the checks will be skippable (by configuration), with some "eventual replication" being allowed.

Anyway this is all theoretical - I have no knowledge of the inner workings of GlusterFS.


On Jan 27, 2011, at 2:15 AM, David Lloyd wrote:

> So, we're trying to serve home directories, and some directories with
> applications from our gluster cluster, mounting as glusterfs. This is stuff
> we've done quite happily from nfs previously.
> 
> The main use of the gluster is to serve large image files and it's great for
> that but the piddling little files that in the homedirs and applications are
> much slower (~10x) than the previous nfs. This is measured as the time it
> takes for the application to start, rather than anything low-level.
> 
> It feels to me that it's the same kind of thing that's been mentioned here
> before, but it doesn't seem to have been answered.
> 
> Should we expect this behaviour?
> Is there anything we can do to improve things?
> 
> We're on 3.1.1, haven't done much in the way of tuning from defaults.
> 
> Cheers
> David
> 
> -- 
> David Lloyd
> V Consultants
> www.v-consultants.co.uk
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users