[Gluster-devel] glfs vs. unfsd performance figures
gordan at bobich.net
Fri Jan 8 22:28:32 UTC 2010
On 08/01/2010 21:15, Martin Fick wrote:
> --- On Fri, 1/8/10, Gordan Bobic<gordan at bobich.net> wrote:
>>>> On writes, NFS gets 4.4MB/s, GlusterFS (server
>> side AFR) gets 4.6MB/s. Pretty even.
>>>> On reads GlusterFS gets 117MB/s, NFS gets 119MB/s
>> (on the first read after flushing the caches, after that it
>> goes up to 600MB/s). The difference in the unbuffered
>> readings seems to be in the sane ball park and the
>> difference on the reads is roughly what I'd expect
>> considering NFS is running UDP and GLFS is running TCP.
>> # The machines involved are quad core
>> time make -j8 all
>> 1) pure ext3
>> 6:40 CPU bound
>> 2) ext3
>> 15:15 rootfs (glfs, no
>> cache) I/O bound
>> 3) ext3+knfsd
>> 7:02 mostly network bound
>> 4) ext3+unfsd 16:04
>> 5) glfs
>> 61:54 rootfs (glfs, no
>> cache) I/O bound
>> 6) glfs+cache
>> 32:32 rootfs (glfs, no cache) I/O bound
>> 7) glfs+unfsd 278:30
>> 8) glfs+cache+unfsd 189:15
>> 9) glfs+cache+glfs 186:43
> Am I understanding correctly that all the glfs benchmarks are
> using AFR? If so, perhaps that is not a very useful comparison
> since the AFR locking might be your bottleneck with a make?
> If so, it would then not highlight any potential differences
> between your nfs server and pure glfs setup. I think it
> would be more useful to remove AFR from the picture to get
> a real idea,
I would guess that the key reason for performance deterioration vs bare
metal is fuse rather than AFR. In all cases, the slave server should
only be getting writes.
The ext3 tests are there purely as reference points and to get some idea
of difference in performance between knfsd and unfsd.
The difference between tests 5 and 7, however, is relevant because all
of that difference (all 400% of it) comes from having the extra hop
between the client and the server, since in both cases the underlying
glfs setup is the same.
However, the main point I wanted to get to the bottom of was whether
using glfs for the server<->client connection has any benefit over
unfsd, and the test quite clearly shows that it doesn't (there is no
real difference between results of tests 8 and 9).
More information about the Gluster-devel