[Gluster-devel] File integrity and consistency in geo-replication

[Natale Vinto]

Hi Jeff,
2012/12/19 Jeff Darcy <jdarcy at redhat.com>:
> On 12/19/2012 12:57 PM, Natale Vinto wrote:
>>
>> I saw the Server Quorum feature for the next version, I was wondering
>> if it is the one from the Duvvuri theory and if could be useful for
>> that case killing unconsistent bricks.
>
>
> I hadn't heard of Duvvuri, and all I could find in a quick search was a
> couple of old papers about adaptive leasing.  Do you have any other
> references?  The server quorum feature allows us to avoid inconsistency from
> writes done without local quorum, but has practically no effect on
> geo-replication.  What are your expectations about quorum and consistency in
> a wide-area environment?

I'm sorry I put the wrong name of the author, he was Jalote [1] and in
his (old) "Fault tolerance in distribuited system" he was talking a
quorum-based protocol ROWA that I thought it would be the basic idea
behind the new feature that could maybe help in the consistency
avoiding contradictory changes due any network outage. This was what I
supposed because I didn't understood good the scenario.
>
>> And, what about using Hadoop with the Gluster connector?
>
>
> Um . . . it works?  Not sure what you're getting at here.

I don't know. I know it from lecterature as being well used in large
data sets across clusters and I since I found it as a Gluster
connector I was wondering if using HDFS would be a right thing.
>
>> I think that this work would require a massive study and testing (for
>> me at least!), but it would be very nice do this research trying to
>> get an international cultural needing working thanks to a big
>> opensource project, "in perpetuum" :)
>
>
> I agree.  It would definitely be good for us to understand what your needs
> are with respect to consistency or data integrity, and discuss how our
> modular architecture might allow us to add features that address those
> needs.

I think that what is crucial is the fact to ensure in some way that
the geo-replication on very big data won't freeze or fail, or at least
there is nothing irreversibly corrupted and file is whole. Then in
your opinion, it would be possible, or has it any sense, using for the
geo-replication a torrent-like approach with a module that makes all
nodes seeders and let them update itself at the needing?
Thanks for explanations,
Bye

>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel

[1] http://books.google.it/books?id=KuRQAAAAMAAJ

-- 
Natale Vinto
http://www.natalevinto.it
FSF Member #8163
gpg keyserver: keys.gnupg.net  recv-keys 55260343
Key fingerprint = 71F1 12C2 035D 7082 0C0A  E677 8A85 5F78 5526 0343