[Gluster-devel] Question about compile performance over GlusterFS

[Amar S. Tumballi]

Hi Craig,
 Thanks for a nice comparison  between GlusterFS and other network file
systems. But sure, before concluding about the performance, I would suggest
few improvement to your GlusterFS setup.

1. Try with client/protocol instead of having unify with only one subvolume.
(Unify makes sense when you have more than one subvolume, but when there is
only one subvolume, its a extra layer which may count as overhead), below
io-thread volume.

2. in io-cache on client side, as during kernel compile lot of *.h files are
re-read, you can give preference to *h files only.

volume ioc
  type performance/io-cache
  subvolumes wb
  option priority *.h:100
end-volume

Regards,
Amar

On Wed, Mar 12, 2008 at 3:55 PM, Craig Tierney <Craig.Tierney at noaa.gov>
wrote:

> I have been testing out my GlusterFS setup.  I have been
> very happy with the streaming IO performance and scalability.
> We have some users on the system now and they are seeing
> very good performance (fast and consistent) as compared
> to our other filesystem.
>
> I have a test that I created that tries to measure metadata
> performance by building the linux kernel.  What I have
> found is that GlusterFS is slower than local disk, NFS,
> and Panasas.  The compile time on those three systems
> is roughly 500 seconds.  For GlusterFS (1.3.7), the
> compile time is roughly 1200 seconds.  My GlusterFS filesystem
> is using ramdisks on the servers and communicating using
> IB-Verbs.  My server and client configs are below.
>
> Note I did not implement both write-behind and not read-behind
> based on some benchmarks I saw on the list on how it affects
> re-write.
>
> So, is this just because mmap isn't (yet) supported in FUSE?
> Or, is there something else I should be looking at.
>
> Thanks,
> Craig
>
>
> server.cfg
> ----------
>
> volume brick
>   type storage/posix                   # POSIX FS translator
>   option directory /tmp/scratch/export        # Export this directory
> end-volume
>
> volume server
>   type protocol/server
>   subvolumes brick
>   option transport-type ib-sdp/server     # For TCP/IP transport
>   option auth.ip.brick.allow *
> end-volume
>
> client.cfgvolume client-ns
>   type protocol/client
>   option transport-type ib-sdp/client
>   option remote-host w8-ib0
>   option remote-subvolume brick-ns
> end-volume
>
>
>
> volume client-w8
>   type protocol/client
>   option transport-type ib-sdp/client
>   option remote-host w8-ib0
>   option remote-subvolume brick
> end-volume
>
> volume unify
>         type cluster/unify
>         subvolumes  client-w8
>         option namespace client-ns
>         option scheduler rr
> end-volume
>
> volume iot
>         type performance/io-threads
>         subvolumes unify
>         option thread-count 4
> end-volume
>
> volume wb
>         type performance/write-behind
>         subvolumes iot
> end-volume
>
> volume ioc
>         type performance/io-cache
>         subvolumes wb
> end-volume
>
> ----------
>
>
>
>
> --
> Craig Tierney (craig.tierney at noaa.gov)
>
>


-- 
Amar Tumballi
Gluster/GlusterFS Hacker
[bulde on #gluster/irc.gnu.org]
http://www.zresearch.com - Commoditizing Supercomputing and Superstorage!