[Gluster-users] cache before threads or visa versa

[Ian Rogers]

Can someone familiar with performance/io-threads explain how it actually 
works (or write in up in the wiki)?...

I've been doing some experiments with performance/io-threads and 
performance/io-cache. I'd expect that if io-threads was the root of the 
vol tree then the io-cache would be duplicated along with the memory 
usage. But that doesn't seem to be the case...

E.g. on one test client I have

...

volume iothreads_web_upload_ct
   type performance/io-threads
   option thread-count 64
   subvolumes mirror_web_upload_ct
end-volume

volume io_cache_web_upload_ct
   type performance/io-cache
   option cache-size 256MB
   option cache-timeout 60
   subvolumes iothreads_web_upload_ct
end-volume


and on the other

...

volume io_cache_web_upload_tc
   type performance/io-cache
   option cache-size 256MB
   option cache-timeout 60
   subvolumes mirror_web_upload_tc
end-volume

volume iothreads_web_upload_tc
   type performance/io-threads
   option thread-count 64  # default is 1
   subvolumes io_cache_web_upload_tc
end-volume

I then run a bunch of concurrent greps on each client on a large set of 
files.  On both clients VSZ of glusterfs for that mount never get above 
357492k which is actually great but not what I expected :-)

If one puts io-threads in front of protocol/client will the client be 
making multiple concurrent TCP connections to the server or not?

Ian