[Gluster-devel] Client side afr, locking, race condition, simultanous writes, out of sync

[gordan at bobich.net]

On Fri, 9 May 2008, Anand Babu Periasamy wrote:

> Now it becomes easier to explain with actual cases and examples.
>
> Your tests are expected to fail. It is a POSIX requirement
> for applications to hold locks or synchronize using some
> form of IPC/RPC before writing to the same file at same
> time.
>
> If you hold a lock and write "one" or "two" to file.txt
> your tests will always succeed. If not, the file will
> get corrupted. In case of a replicated file system,
> corruption can also result in disparate set of data.

Yes, but data being corrupted is not the same as data being inconsistent. 
The latter is an additional problem, which IMO shouldn't happen. It 
doesn't happen on other cluster file systems, and it should be avoided at 
all cost.

> We can introduce atomic-writes as an option for AFR
> translator. It will avoid disparate writes in case
> of corruption, but it still cannot avoid corruption.
> It won't be of much use, except taking some performance
> hit.

I disagree that it wouldn't be of much use. Disperate writes aren't an 
option on other similar solutions (e.g. <GFS|OCFS>+DRBD).

> We are open to implementing this option if you think
> it is useful to you.
>
> /bin/echo command interpreted by the shell is a complete
> process. Shell creates the files without holding any locks.
> If you want to hold locks inside shell scripts, you
> can either create a .lck file and hold on to it..
> or can do a simple trick like this one..
> --------------------------------------
> client1# echo "one" > file.txt.$(hostname)
> client1# mv file.txt.$(hostname) file.txt
>
> client2# echo "two" > file.txt.$(hostname)
> client2# mv file.txt.$(hostname) file.txt
> --------------------------------------
>
> You are always expected to hold locks and issue writes
> in a multi user mode.

This will also re-set versions on files, but what happens when both sides 
are appending to the file? Logs being an obvious example - they are not 
always locked.

Gordan