[Gluster-devel] bit rot support for glusterfs

[Paul Cuzner]

Hi Chris, 

I'm glad someone other than me asked this question :) 

I think there are two things at play here. 

1) when btrfs is adopted then we inherited block level checksuming from the filesystem scrub. Since this operates below the file level this type of bit rot detection will be workload agnostic - i.e. open files, large media files will be more easily dealt with and less of an overhead. The implementation then becomes an exercise of integrating the btrfs scrub with a remedial action from, gluster's perspective. I haven't looked too deep into this, but from what I understand this could simply be a logwatcher that catches the scrub error and hands of to gluster to repair - pretty much as you describe. 

2)however, for the current implementations out there bit rot is still a risk and we need to do something. By utilsing the new changlog framework with the checksum process, we can enable bit-rot detection on the more standard and more widely accepted LVM/XFS deployments. 

I guess we need one eye on the now, and another on the near-future! 

Cheers, 

PC 

----- Original Message -----

> From: "Chris Murphy" <lists at colorremedies.com>
> To: gluster-devel at nongnu.org
> Sent: Friday, 10 January, 2014 11:06:47 AM
> Subject: Re: [Gluster-devel] bit rot support for glusterfs

> Is an optimization possible for Btrfs that avoids GlusterFS level checksums
> entirely? Even with data raid0, by default it writes metadata with raid1,
> and can determine file corruption. Such corruptions are reported by
> pathtofile by Btrfs. If that can be communicated to glusterfs, glusterfs
> could initiate repair by getting a good copy from another brick to forward
> to the application layer, and then initiate an overwrite for that file on
> the brick with the bad copy.

> ?

> Chris Murphy
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140110/5a8133c8/attachment-0001.html>