[Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

[Pranith Kumar Karampuri]

hi,
      Background: Quick-read + open-behind xlators are developed to help 
in small file workload reads like apache webserver, tar etc to get the 
data of the file in lookup FOP itself. What happens is, when a lookup 
FOP is executed, GF_CONTENT_KEY is added in xdata with max-length and 
posix xlator reads the file and fills the data in xdata response if this 
key is present as long as the file-size is less than max-length given in 
the xdata. So when we do a tar of something like a kernel tree with 
small files, if we look at profile of the bricks all we see are lookups. 
OPEN + READ fops will not be sent at all over the network.

      With dht2 because data is present on a different cluster. We can't 
get the data in lookup. Shyam was telling me that opens are also sent to 
metadata cluster. That will make perf in this usecase back to where it 
was before introducing these two features i.e. 1/3 of current perf 
(Lookup vs lookup+open+read). I suggest that we send some fop at the 
time of open to data cluster and change quick-read to cache this data on 
open (if not already) then we can reduce the perf hit to 1/2 of current 
perf, i.e. lookup+open.

      Sorry if this was already discussed and I didn't pay attention.

Pranith