[Gluster-users] Coherency problem with file buffer cache

[Sabyasachi Ruj]

Hi,

I am having a problem with glusterfs and kernel file buffer cache.
I've been able to reduce the problem to a simple case to demonstrate
the behavior:

I have an app-1 in client1. This application reads page x of a file.
The content of this page will get stored in kernel file buffer cache
of client1. app-1 periodically checks for update in the content of
page x. Now an app-2 in client2, updates page x of the same file and
does fsync. The problem is app-1 never sees the update made by app-2.
I guess it is because of the page cache is not being marked as dirty
in client-1. Is there any way to guarantee that once a fsync has been
called from any client. All client will see the latest content?

I have written a C program to replicate this problem if that helps.
You can find the program here: http://snipt.org/vaId9.

Download the program into an mounted gluster volume. You have to have
at least two clients.

Complie the program using:

    # g++ -g -o test <filename.cpp>

Then from client1:

    # test samplefile verify

It will output 'A' in the screen, which now the content of the file samplefile.

Then from client2 run:

    # test samplefile update B

This will update the content of the file samplefile with content 'B'.
You will be able to see that client1 will still show 'A'. This will
not happen when you update the file from the same client where verify
is running, that is client1.

I know that direct I/O mode can help to certain extent. But it does
not guarantee a atomic transaction either.

-- 
Sabyasachi