[Gluster-devel] glfs vs. unfsd performance figures

[Gordan Bobic]

On 08/01/2010 21:15, Martin Fick wrote:
> --- On Fri, 1/8/10, Gordan Bobic<gordan at bobich.net>  wrote:
> ...
>>>> On writes, NFS gets 4.4MB/s, GlusterFS (server
>> side AFR) gets 4.6MB/s. Pretty even.
>>>> On reads GlusterFS gets 117MB/s, NFS gets 119MB/s
>> (on the first read after flushing the caches, after that it
>> goes up to 600MB/s). The difference in the unbuffered
>> readings seems to be in the sane ball park and the
>> difference on the reads is roughly what I'd expect
>> considering NFS is running UDP and GLFS is running TCP.
>>>>
> ...
>
>> # The machines involved are quad core
>> time make -j8 all
>>
>> 1) pure ext3
>> 6:40    CPU bound
>> 2) ext3
>>      15:15    rootfs (glfs, no
>> cache) I/O bound
>> 3) ext3+knfsd
>> 7:02    mostly network bound
>> 4) ext3+unfsd        16:04
>> 5) glfs
>>      61:54    rootfs (glfs, no
>> cache) I/O bound
>> 6) glfs+cache
>> 32:32    rootfs (glfs, no cache) I/O bound
>> 7) glfs+unfsd        278:30
>> 8) glfs+cache+unfsd    189:15
>> 9) glfs+cache+glfs    186:43
>
> Am I understanding correctly that all the glfs benchmarks are
> using AFR? If so, perhaps that is not a very useful comparison
> since the AFR locking might be your bottleneck with a make?
> If so, it would then not highlight any potential differences
> between your nfs server and pure glfs setup.  I think it
> would be more useful to remove AFR from the picture to get
> a real idea,

I would guess that the key reason for performance deterioration vs bare 
metal is fuse rather than AFR. In all cases, the slave server should 
only be getting writes.

The ext3 tests are there purely as reference points and to get some idea 
of difference in performance between knfsd and unfsd.

The difference between tests 5 and 7, however, is relevant because all 
of that difference (all 400% of it) comes from having the extra hop 
between the client and the server, since in both cases the underlying 
glfs setup is the same.

However, the main point I wanted to get to the bottom of was whether 
using glfs for the server<->client connection has any benefit over 
unfsd, and the test quite clearly shows that it doesn't (there is no 
real difference between results of tests 8 and 9).

Gordan