[Gluster-users] Performance optimization tips Gluster 3.3? (small files / directory listings)

[olav johansen]

Hi,

I'm using Gluster 3.3.0-1.el6.x86_64, on two storage nodes, replicated mode
(fs1, fs2)
Node specs: CentOS 6.2 Intel Quad Core 2.8Ghz, 4Gb ram, 3ware raid, 2x500GB
sata 7200rpm (RAID1 for os), 6x1TB sata 7200rpm (RAID10 for /data), 1Gbit
network

I've it mounted data partition to web1 a Dual Quad 2.8Ghz, 8Gb ram, using
glusterfs. (also tried NFS -> Gluster mount)

We have 50Gb of files, ~800'000 files in 3 levels of directories (max 2000
directories in one folder)

My main problem is speed of directory indexes "ls -alR"  on the gluster
mount takes 23 minutes every time.

It don't seem like any directory listing information cache, with regular
NFS (not gluster) between web1<->fs1, this takes 6m13s first time, and
5m13s there after.

Gluster mount is 4+ times slower for directory indexing performance vs pure
NFS to single server, is this as expected?
I understand there is a lot more calls involved checking both nodes but I'm
just looking for a reality check regarding this.

Any suggestions of how I can speed this up?

Thanks,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120606/b1f93ad8/attachment.html>