[Gluster-devel] Re; Caching ...

[Gareth Bult]

Hi, 

Can someone tell me, is the IO cache per mount point, or per process ? 

I suspect it's per mount point, in which case running a number of intensive processes (i.e. >1) on one mount point can make the IO cache pretty pointless. I've not had time to "prove" this just yet, but .. I have one process that runs in 30 seconds ... when I run 3 concurrent processes they complete in around 5 mins .. (each process operates on a different / large file) 

Would it be possible (!) to specify either in the client config or at mount time that an IO cache was per-process rather than per mount point ? 
(or would the solution simply to remount the same partition many times ...?) 
Or even better, that cache memory is tagged so each process gets an even share of the cache and intensive reads by one process do not wipe out cached entries of another process .. i.e. some sort of sharing on a process basis rather than an absolute drop the least recently used basis? 

Just as a matter of interest, I've noticed the following in terms of performance which has surprised me somewhat; 

a. Running XEN instances off disk images on a mounted gluster filesystem provides much better performance than I would have anticipated 
b. Running multi-process / multi-file access off a gluster filesystem is slower than anticipated, indeed performance seems to drop-off exponentially with the number of processes involved in the IO. 

I am using readahead and IO cache and I'm still tuning .. but if there's anything specific performance-wise I should be looking at with regards to multi-process access, an advice would be most welcome. 

Regards, 
Gareth.