[Gluster-devel] performance improvements

[Hans Einar Gautun]

tir, 23,.10.2007 kl. 18.36 +0200, skrev Vincent Régnard:
> Hi all,
> 
> We are presently trying to tune our non-gluster configuration to improve 
> glusterfs performance. My config is gluster 1.3.7/fuse2.7.0-glfs5, linux 
> 2.6.16.55. We have 3 clients and 3 servers on a 100Mb network with 5ms 
> round trip between clients and servers. The 3 clients replicate with afr 
> on client side over the 3 servers.
> 
> We have a read/write throughput benchmark (dbench) between 2 and 5 MB/s.
> 
> The afr synchronisation using "find -mtime -1 -type f -exec head -c1 
> trick" takes approximately 30 minutes for a 20GB filesystem with 300.000 
> files. Which seems too long to be acceptable for us. I'd like to tune 
> some parameters to increase performance.
> 
> I can imagine that reducing the roundtrip between servers might help ? 
> But I cannot actually do anything for that. The only thing I might be 
> able to do is to configure some QOS. Have you any suggestion about how 
> we should do that ? Would giving priority to tcp/6996 between clients 
> and servers really help ?
> 
> At the (linux) kernel level, could acting on PREMPTION MODEL and 
> CONFIG_HZ produce improvement ?
> 
> Our present config is as follow:
> 
> # CONFIG_PREEMPT_NONE is not set
> # CONFIG_PREEMPT_VOLUNTARY is not set
> CONFIG_PREEMPT=y
> CONFIG_PREEMPT_BKL=y
> 
This is about interrupting the kernel. You like it on a desktop, to feel
responce when you want to do something. Or on a "realtime" system - like
my home firewall with bandwithcontrol on an ADSL link(config=1000Hz).
Need quick responce there. The cost is context switching if you have
many processes trying to get attention. 
On a fileserver you maybe are better of with the kernel timeslice only
(or no other and unneeded processes running). Better adjusting the Hz,
100 or 250 looks ok.


> # CONFIG_HZ_100 is not set
> CONFIG_HZ_250=y
> # CONFIG_HZ_1000 is not set
> CONFIG_HZ=250
> 
> Is it better to prefer SMP to non-SMP kernel builds ? (We presently have 
> SMP eneabled for our dual-cores). What impact on glusterfs performances 
> if we deactivate SMP ?
Having dualcore you need SMP to use both cores - and you want that. On
some CPU's (some Pentium 4) you are better off without hyperthreading -
you have to test it. Multicore kernel runs more processes at the same
time, and thats good also for glusterfs.

> 
> We use linuxthread (glibc2.3) and have no NPTL support, can this 
> influence the performances as well ?
> 
> We naturally already have gluster improvements in the configuration 
> (io-{thread,cache}, readahead and writebehind).b
> 
> Thanks in advance for your comments or suggestions.
> 
> Vincent.
> 
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
-- 
Einar Gautun                     einar.gautun at statkart.no

Statens kartverk            | Norwegian Mapping Authority
3507 Hønefoss               |    NO-3507 Hønefoss, Norway

Ph +47 32118372   Fax +47 32118101       Mob +47 92692662