[Gluster-users] GlusterFS - Abysmal Speed on Fresh Install

[Christopher Michaelis]

Hi there,

I'm new to the GlusterFS project - looks superb and very promising! I'm 
running into massive performance issues, however. I'm using the stock 
default configuration that GlusterFS put in place when I created the 
volume - it seems to reference io-cache, quick-read, etc in the volume 
files, which makes me think it's already pulling in these optimizations.

This is a replicate setup - I've tried with 2, 3, and 4 nodes, and 
performance remains awful on any of them - network communication seems 
fine, with average speeds at 9-15MB/sec.

# gluster --version
glusterfs 3.1.1 built on Nov 29 2010 10:07:45
Repository revision: v3.1.1

With no other activity on the filesystems on any of my nodes:
# time ls -al
total 10072
drwx--x--x 10 root   root       4096 Dec  3 16:50 ./
drwxr-xr-x  3 root   root       4096 Dec  3 14:19 ../
-rw-r--r--  1 root   root   10240000 Dec  3 16:49 testfile

real    0m1.347s
user    0m0.000s
sys    0m0.000s

# time cp testfile testfile2

real    0m11.254s
user    0m0.000s
sys    0m0.000s
# time diff testfile*

real    0m5.792s
user    0m0.004s
sys    0m0.000s

Read speed is marginally faster than write speed, but still horrible - 
e.g. if Apache is serving content off of a glusterfs mountpoint, it 
times out 95% of the time before it can read the files. I'm using mount 
-t glusterfs, with default mount options.

Can anyone point me in the right direction to getting things nice and 
speedy here? I'd appreciate any feedback or help! I can provide any 
configuration files necessary, or even root login access to the box(es) 
via private e-mail if you want to poke around (these are just test boxes 
presently).

Thanks,
--Chris
chris.michaelis at uk2group.com