[Gluster-devel] Online codes

[Anand Avati]

> >
> > I am using GlusterFS since a few weeks. Surfing the net I stumbled into
> this:
> >
> > http://en.wikipedia.org/wiki/Online_codes
> >
> > I am wondering if it would be a good idea to make a xlator for GlusterFS
> using
> > this algorithm.
> >
> > The advantage of this solution is that you can split a file in N size
> blocks,
> > and spread them around the bricks. If you lose one brick, you can still
> > reconstruct files combining the remaining blocks. It's also quite
> efficient.
> >
> > Kind regards to everybody and compliments for the great job already
> done!
>
> I'm not sure how you would extrapolate this to a client/server
> architecture, even one where there may be many servers.
>
> It seems optimized for a network where everyone participates in the file
> sharing (indeed it was developed for P2P usage), maximizing the
> possibility that new nodes get data other nodes are less likely to be
> sharing.
>
> I'm not sure how well it would apply to glusterFS, which is
> client/server based, as it seems to require P2P type connections.  Even
> though you CAN run the client and server on the same system in
> GlsuterFS, there's a distinction between where data is served from and
> where it's mounted to, which isn't usually the case in P2P (indeed, it
> can't be the case for this algorithm to work).
>
> Although, maybe GlusterFS is modular enough that there's some real
> interesting translators that can be made to work with this in useful
> ways.  I can't seem to wrap my head around a useful way though.


One use I see (and from what I understand of the paper) is that the
algorithm can provide a RAID5/RAID6 like translator where redundancy factor
is arbitrary, and this redundancy factor can be increased/decreased on the
fly (adding nodes/deleting nodes for redundancy of any N nodes).

Also calculation of each block is based upon a pseudorandom number which
decides the probability of being able to reconstruct, which makes it
necessary to have (1 + e)*n blocks to reconstruct n blocks, where "usually"
e is 0. I need to read the paper in greater detail I must admit.

avati


-- 
If I traveled to the end of the rainbow
As Dame Fortune did intend,
Murphy would be there to tell me
The pot's at the other end.