[Gluster-users] performance tuning?

[Will Daniels]

On 21/12/11 14:26, Jeff White wrote:
> I'm going to run into the 'lots of small files on replica sucks' problem.

Good luck! I found it is not the performance so much that is the 
problem, but the consequential system load on the bricks with many small 
files.

I found that copying 3 large IMAP accounts simultaneously to a new 
server having Maildirs on gfs (one brick hosted on a different server, 
16-core 32GB machine with two 1.5TB SATA drives configured in RAID 10, 
gigabit ethernet) created such I/O load on the brick server as to bring 
it to it's knees (uptime load numbers around 80-90) despite CPU load 
closer to 5%.

A couple of very small VMs running on the brick server were just 
completely choked. I tried a couple of gfs tuning options with no 
observable difference.

In my case it was an quick conclusion to just use an IMAP proxy and 
stick to local storage, though I did wonder afterwards if perhaps the 
local I/O got choked because there were 16 cores to run gluster threads 
in parallel. If it is possible to restrict gluster's thread count, maybe 
it would have worked better?

And I guess with greater distribution across bricks the load would be 
shared between the disk I/O potential of many more machines. I'd be 
interested to hear more experienced views on these things.

-Will