[Gluster-users] Replicate performance (cluster/replicate, performance/io-cache)

[Alex Craven]

Greetings,

I've just started experimenting with glusterfs and am new to this list, so
apologies if this has all been covered before! (a quick browse through the
archives didn't reveal anything though)

I'm attempting to use glusterfs to manage replication between a pair of
geographically separated test servers (communicating over an ssh tunnel).
I realise this isn't exactly an optimal configuration. However, I need
fast read performance but don't particularly care about write performance,
and it doesn't really matter if I'm reading 30-second-old data (my usage
is mainly for webserver replication, so the data won't be changing very
often anyway).

Now, I've set up gluster as per the attached configurations. I believe all
the appropriate performance translators are configured, notably including
the io-cache. The read-subvolume is set to a posix storage on the local
machine.

Still, in spite of the local read-subvolume and caching, successive read
operations on the same (tiny) file, well within the cache-timeout period
always take a minimum time equivalent to four round trips to complete.

Can anyone explain why this is? I understand there may be one transaction
to validate the cache, but what else is going on here? Furthermore, is it
possible to defer/disable this re-validation (within a timeout)
altogether?

Thanks in advance for any help!

Cheers,
Alex.


-- 
Alex Craven
krei at it.net.au
-------------- next part --------------
volume loopback
  type storage/posix
  option directory /cluster/test
end-volume

volume remote-achaemenes
 	type protocol/client
 	option transport-type tcp
 	option remote-host 127.0.0.1
 	option remote-port 6997
 	option remote-subvolume brick
	option username test_cluster
	option password xxxxxxxx
	option transport-timeout 10
end-volume

volume replicate
	type cluster/replicate
	subvolumes  loopback remote-achaemenes
	option read-subvolume loopback
	option lock-node loopback
	### DANGER BELOW! 
        # the following are set temporarily, in an attempt to identify the
        # cause of poor read performance issues
	option self-heal off
	option data-self-heal off
	option metadata-self-heal off
	option entry-self-heal off
	option data-change-log off
	option metadata-change-log off
	option entry-change-log off
	option data-lock-server-count 0
	option entry-lock-server-count 0
end-volume

volume writebehind
	type performance/write-behind
	option window-size 1MB
	option flush-behind on
	subvolumes replicate
end-volume

volume cache
	type performance/io-cache
	option cache-size 256MB
	subvolumes writebehind
	option priority *.php:3,*.html:2,*:1
	option cache-timeout 10
	option force-revalidate-timeout 30
	option force-atime-update off
end-volume
-------------- next part --------------
volume posix
  type storage/posix
  option directory /cluster/test
end-volume

volume locks
  type features/locks
  subvolumes posix
end-volume

volume brick
  type performance/io-threads
  option thread-count 8
  subvolumes locks
end-volume

volume server
  type protocol/server
  option transport-type tcp
  option auth.login.brick.allow test_cluster
  option auth.login.test_cluster.password xxxxxxxx
  option listen-port 6996
  option bind-address 127.0.0.1
  subvolumes brick
end-volume