[Gluster-devel] Splitbrain Resolution

[Joey Novak]

Hi Gordan,

 I answered your questions about meta data, and striping in the wiki, here:
http://www.gluster.org/docs/index.php/GlusterFS_FAQ#Metadata_Storage_-_When_using_striping_.28unify.29.2C_how_does_the_file_data_get_split.3F

 As for the other questions...  Some of them already have answers in the
wiki, and some of them I don't know the answer to, but I am sure one of the
awesome developers will chime in and answer your questions.

  As for AFR, the client takes care of the work.  The servers are
essentially just network accessible file systems, that aren't aware of each
other, and thus, don't replicate between each others at all.

  SplitBrain...  While it is possible for this to happen, Gluster is not
designed like Coda for offline operation.  If a client can't hit a server,
it is unlikely another client can hit it, and thus create a splitbrain
scenario...  But to answer your question, I don't think there is any
conflict resolution method.

  Fencing and Quorum...  I don't think you will find those terms anywhere in
the documentation.

  Metadata change detection...  That is a good question, I hope someone
provides a good explanation.

    Joey
On Thu, Apr 17, 2008 at 7:53 PM, Gordan Bobic <gordan at bobich.net> wrote:

> Hi,
>
> I'm experimenting with GlusterFS and I have a few questions that the
> documentation seems to leave unanswered.
>
> 1) Replication (AFR)
> How does this work? I can see from my test setup that the mounted FS has
> the same content (files) as the backing directory on the server. The server
> configuration only lists the local node and not the other peer servers. This
> implies that all replication/syncing is done by the clients. This in turn
> implies that the read load can be shared between the servers (scales to 1*n
> where n is the number of servers), but the write load gets sent to each
> server. This implies that the write performance scales inversely when using
> mirroring (1/n). This seems quite poor. Am I misunderstanding how this
> works? Do the servers replicate between themselves? Or does all replication
> really happen on the client nodes? How would this handle the condition of
> writes happening on the server directly to the backing directory while the
> client is trying to write to the same directory/files? Would this work the
> same as NFS would or is there a definitive requirement to always access the
> data via the glusterfs mount point? (I understand that this is only possible
> when using AFR, and not with striping.)
>
> 2) Splitbrain
> How does the recovery from this situation get handled? Which file wins,
> and which file gets clobbered? Is there any scope for conflict resolution
> (e.g. as in Coda)?
>
> 3) Metadata Storage
> When using striping, how does the file data get split, and how/where is
> the metadata kept?
>
> 4) Fencing and Quorum
> Is there error/desync detection, and are there such concepts as fencing of
> dead server nodes and quorum to prevent splitbrain operation?
>
> 5) Metadata Change Detection
> I understand from the documentation tha replication/sync (where required)
> happens when opening a file for reading. What about metadata? Are metadata
> changes (e.g. file size, modification time) visible to the clients when the
> file changes on the server and the client issues an ls? Or is it necessary
> to read the file before issuing ls to get the size/timestamp metadata? What
> about touching a file? Does this cause the file to be synced? Would it cause
> the file to get corrupted if another node updated the file's content it in
> the meantime?
>
> Thanks.
>
> Gordan
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>



-- 
-Joey

C) 803-409-9969
W) 757-233-0834