[Gluster-devel] Question about compile performance over GlusterFS

[Craig Tierney]

I have been testing out my GlusterFS setup.  I have been
very happy with the streaming IO performance and scalability.
We have some users on the system now and they are seeing
very good performance (fast and consistent) as compared
to our other filesystem.

I have a test that I created that tries to measure metadata
performance by building the linux kernel.  What I have
found is that GlusterFS is slower than local disk, NFS,
and Panasas.  The compile time on those three systems
is roughly 500 seconds.  For GlusterFS (1.3.7), the
compile time is roughly 1200 seconds.  My GlusterFS filesystem
is using ramdisks on the servers and communicating using
IB-Verbs.  My server and client configs are below.

Note I did not implement both write-behind and not read-behind
based on some benchmarks I saw on the list on how it affects
re-write.

So, is this just because mmap isn't (yet) supported in FUSE?
Or, is there something else I should be looking at.

Thanks,
Craig


server.cfg
----------

volume brick
   type storage/posix                   # POSIX FS translator
   option directory /tmp/scratch/export        # Export this directory
end-volume

volume server
   type protocol/server
   subvolumes brick
   option transport-type ib-sdp/server     # For TCP/IP transport
   option auth.ip.brick.allow *
end-volume

client.cfgvolume client-ns
   type protocol/client
   option transport-type ib-sdp/client
   option remote-host w8-ib0
   option remote-subvolume brick-ns
end-volume



volume client-w8
   type protocol/client
   option transport-type ib-sdp/client
   option remote-host w8-ib0
   option remote-subvolume brick
end-volume

volume unify
         type cluster/unify
         subvolumes  client-w8
         option namespace client-ns
         option scheduler rr
end-volume

volume iot
         type performance/io-threads
         subvolumes unify
         option thread-count 4
end-volume

volume wb
         type performance/write-behind
         subvolumes iot
end-volume

volume ioc
         type performance/io-cache
         subvolumes wb
end-volume

----------




-- 
Craig Tierney (craig.tierney at noaa.gov)