[Gluster-devel] Client side AFR race conditions?

[Derek Price]

Kevan Benson wrote:

> I'm not saying I don't want to see a more robust solution for client 
> side AFR, just that each configuration has it's place, and client side 
> AFR isn't currently (and may never be) capable of serving a share that 
> requires high data integrity.
> 
> If you think fixing this current issue will solve your problems, maybe 
> you haven't considered the implications of connectivity problems between 
> some clients and some (not all) servers...  Add in some clients with 
> slightly off timestamps and you might have some major problems WITHOUT 
> any reboots.

Am I getting this straight?  Even with server-side AFR, you get mirrors, 
but if all the clients aren't talking to the same server then there is 
no forced synchronization going on?  How hard would it be to implement 
some sort of synchronization/locking layer over AFR such that reads and 
writes could still go to the nearest (read: fastest) possible server yet 
still be guaranteed to be in sync?

In other words, the majority of servers would know of new version 
numbers being written anywhere and yet reads would always serve local 
copies (potentially after waiting for synchronization).  The application 
I'm thinking of is virtualized read/write storage.  For example, say you 
want to share some sort of data repository with offices in Europe, 
India, and the U.S. and you only have slow links connecting the various 
offices.  You would want all client access to happen against a local 
mirror, and you would want to restrict traffic between the mirrors to 
that absolutely required for locking and data synchronization.

The only thing I'm not quite sure of in this model is what to do if the 
server processing a write operation crashes before the write finishes. 
I wouldn't want reads against the other mirrors to have to wait 
indefinitely for the crashed server to return, so the best I can come up 
with is that "write locks" for any files that hadn't been mirrored to at 
least one available server before a crash would need to be revoked on 
the first subsequent attempted access of the unsynchronized file.  Then 
when the crashed server came back up and tried to synchronize, it would 
find that its file wasn't the current version and sync in the other 
direction.

?

Derek
-- 
Derek R. Price
Solutions Architect
Ximbiot, LLC <http://ximbiot.com>
Get CVS and Subversion Support from Ximbiot!

v: +1 248.835.1260
f: +1 248.246.1176