[Gluster-devel] bit rot support for glusterfs

[shishir gowda]

On 8 January 2014 01:15, Paul Cuzner <pcuzner at redhat.com> wrote:
> Hi Shishir,
>
> Great to see this being progressed!
>
> Would it make sense to compare the checksums for a file across replica's
> too? ie. in a replicated volume, one the local checksum compare is complete,
> compare the checksum against the file's replica on another brick node. This
> would account for 'funnies' in the data path during the write phase
> introducing corruption.
>
Paul, that would introduce dependency of replication on bit-rot
functionality. Bit rot functionality would not be taking into the
replication/cluster setup into account.

On the other-hand, replication could always use bit-rot checksum if
bit rot is enabled on the given replicated volume

> Cheers,
>
> Paul C
>
>
>
> ________________________________
>
> From: "shishir gowda" <gowda.shishir at gmail.com>
> To: gluster-devel at nongnu.org
> Sent: Tuesday, 24 December, 2013 8:35:08 PM
> Subject: [Gluster-devel] bit rot support for glusterfs
>
>
> Hi All,
>
> I will be starting to work on bit rot detection for glusterfs.
>
> This work will be primarily based on Doug Willam's prototype
> (https://github.com/williamsdoug/glusterfs_bitrot_proto).
>
> The existing design can be found @
> http://gluster.org/community/documentation/index.php/Arch/BitRot_Detection#Current_Prototype
>
> Few design updates:
>
> 1. Depend on change-log to recompute checksum. This eliminates
> periodic crawl of brick/volume to update the checksum.
>
> 2. Policy to determine when checksum to recomputed. If a file is under
> going active I/O, then compute checksum only after a delay
>
> 3. Ability to turn off/on bit rot detection in volumes.
>
> 4. If bit rot is turned on for a volume, a crawl would be necessary in
> this case to compute checksum.
>
> This just a initial draft, and will update the design doc, and the thread
> soon.
>
> Please feel free to comment.
>
> With regards,
> Shishir
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>
>