[Gluster-users] Coherency problem with file buffer cache

[Jeff Darcy]

On 06/12/2012 08:22 AM, Sabyasachi Ruj wrote:
> I am having a problem with glusterfs and kernel file buffer cache.
> I've been able to reduce the problem to a simple case to demonstrate
> the behavior:
> ...

I tried running this with RHS 2.0+ (what you would know as GlusterFS 3.3+, from
current git HEAD) under RHEL 6.2 on my machines.  What I saw was that FUSE
*doesn't even send us the read* in most cases, not even reliably when we poke
/proc/sys/vm/drop_caches (which seems to me like it should *always* result in a
read).  It shows up in strace because that's before FUSE, but we're after FUSE.
 If I set a breakpoint on fuse_readv_resume, for example, I don't reach it
until we do an open for another process - at which point the original process
does see the new data.  This can be avoided by using attribute-timeout=0 and
entry-timeout=0, which are unfortunately available in the "glusterfs" command
but not in mount.glusterfs AFAICT.  With these options we still don't see the
change immediately, but we do get called when drop_caches is invoked.