[Gluster-devel] Re; Load balancing ...

[Gareth Bult]

Sorry, I'm trying to follow this but I'm coming a little unstuck ..

Am I right in thinking the rolling hash / rsync solution would involve syncing the file "on open" as per the current system .. and in order to do this, the server would have to read through the entire file in order to create the hashes?
(indeed it would need to do this on two servers to create hashes for comparison?)

So .. as a rough benchmark .. assume 50Mb/sec for a standard / modern SATA drive, opening a crashed 20G file is going to take 400 seconds or six minutes ... ? (which would also flatten two servers for the duration)

Whereas a journal replay of 10M is going to take < 1s and be effectively transparent.
(I'm guessing this could also be done at open time ??)

I know I'm missing something here .. could anyone clarify or detail the rolling hash solution for me?

tia

Gareth.



----- Original Message -----
From: "Krishna Srinivas" <krishna at zresearch.com>
To: "Gordan Bobic" <gordan at bobich.net>
Cc: "gluster-devel" <gluster-devel at nongnu.org>
Sent: Wednesday, April 30, 2008 10:02:00 AM GMT +00:00 GMT Britain, Ireland, Portugal
Subject: Re: [Gluster-devel] Re; Load balancing ...

On Wed, Apr 30, 2008 at 1:41 PM, Gordan Bobic <gordan at bobich.net> wrote:
> Martin Fick wrote:
>
>
> >
> > > A better solution would be to maintain a list of
> > > dirty blocks and use it during selfheal.
> > >
> >
> > Agreed, but why not make it infinitely granular and
> > keep a list of dirty file spans instead of blocks? This should be
> extremely space efficient.
> >
>
>  Is this complication and extra effort realy worth the benefit over straight
> rolling hash rsync approach? It seems to me that applying the rsync method
> at read-time would be a fairly minor mod that would solve 99% of the
> problem. No extra book-keeping would be required, only a change from copying
> the whole file to rsyncing the file.

correct, rsync algorithm is the best choice right now.

Krishna

>
>  Gordan
>
>
>
>
>  _______________________________________________
>  Gluster-devel mailing list
>  Gluster-devel at nongnu.org
>  http://lists.nongnu.org/mailman/listinfo/gluster-devel
>


_______________________________________________
Gluster-devel mailing list
Gluster-devel at nongnu.org
http://lists.nongnu.org/mailman/listinfo/gluster-devel