[Gluster-devel] Bitrot/Tering : Bad files get migrated and hence corruption goes undetected.

[Venky Shankar]

On Sat, Feb 27, 2016 at 06:32:38AM -0500, Joseph Fernandes wrote:
> Yep Agree! :)
> 
> Lets hear from the bitrot folks, what they have to propose.

Apologies for late reply.

> 
> ~Joe 
> 
> ----- Original Message -----
> From: "Niels de Vos" <ndevos at redhat.com>
> To: "Joseph Fernandes" <josferna at redhat.com>
> Cc: "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Saturday, February 27, 2016 4:28:43 PM
> Subject: Re: [Gluster-devel] Bitrot/Tering : Bad files get migrated and hence corruption goes undetected.
> 
> On Fri, Feb 26, 2016 at 11:01:28PM -0500, Joseph Fernandes wrote:
> > Well correctly we dont migrate the existing signature, the file starts
> > it life fresh in the new tier(i.e get the bit rot version 1 on the new
> > tier),
> > Now this is also the case with any special xattr/attributes of the
> > file.
> > Again we rely heavily on the dht rebalance mechanism for migrations,
> > which also doesnt carry special attributes/xattr.
> 
> Is there a good reason to not migrate the bitrot signature? Relying on
> an existing functionality is fine, but if it does not address all your
> needs, you have a valid use-case to improve it.

That could be done. However, an I/O operation on the object during migration
should invalidate the signature and the object should be signed again.

AFAICS, there needs to be some infrastructure to avoid (re)signing of an
object if it's fresh after migration.

Thoughts?

> 
> Niels
> 
> > 
> > 
> > ----- Original Message -----
> > From: "Niels de Vos" <ndevos at redhat.com>
> > To: "Joseph Fernandes" <josferna at redhat.com>
> > Cc: "Gluster Devel" <gluster-devel at gluster.org>
> > Sent: Friday, February 26, 2016 10:33:11 PM
> > Subject: Re: [Gluster-devel] Bitrot/Tering : Bad files get migrated and hence corruption goes undetected.
> > 
> > On Fri, Feb 26, 2016 at 09:32:46AM -0500, Joseph Fernandes wrote:
> > > Hi All,
> > > 
> > > This is a discussion mail on the following issue, 
> > > 
> > > 1. Object is corrupted before it could be signed: In this case, the corrupted
> > >    object is signed and get migrated upon I/O. There's no way to identify corruption
> > >    for this set of objects.
> > > 
> > > 2. Object is signed (but not scrubbed) and corruption happens thereafter:
> > >    In this case, as of now, integrity checking is not done on the fly
> > >    and the object would get migrated (and signed again in the hot tier).
> > > 
> > > 
> > > The (1) is definitely not a issue with bitrot with tiering. But (2) we can do something to avoid
> > > corrupted file from getting migrated. Before we migrate files we can scrub it, but its just a naive
> > > thought, any better suggestions? 
> > 
> > Is there a reason the existing signature can not be migrated? Why does
> > it become invalid?
> > 
> > Niels
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel