[Gluster-devel] Crawling and indexing hardware

[Kevan Benson]

gordan at bobich.net wrote:
> On Fri, 9 May 2008, Daniel Maher wrote:
> 
>>>>> Do you plan to do any AFR (automatic file replication) ?  If so,
>>>>> consider that even a one-byte change to your "big index files" will
>>>>> cause the /entire/ file to be AFR'd between all participating
>>>>> nodes.
>>>
>>> Marcus, what do you mean by this?
>>
>> You've mis-attributed the quote, Krishna.  What i meant was exactly
>> what i said : if you have a 5GB file on an AFR cluster, and a client
>> makes a one-byte change to that file, then the entire 5GB file is
>> re-copied to all members of the AFR cluster.
>>
>> At least, that's my understanding of the situation - please correct me
>> if i'm wrong. :)
> 
> In a word - no. If the nodes are connected, then only the actual 1 byte 
> change will be replicated. If the nodes are disconnected, then the "self 
> heal" will copy the entire 5GB file when the nodes connect.

This can be mitigated with striping, if the striping AFR bug was ever 
fixed (?).

It does increase the complexity of the configuration and the complexity 
of manual recovery (if ever needed) though.

-- 

-Kevan Benson
-A-1 Networks