[Gluster-devel] Bitrot/Tering : Bad files get migrated and hence corruption goes undetected.

[Joseph Fernandes]

Yep Agree! :)

Lets hear from the bitrot folks, what they have to propose.

~Joe 

----- Original Message -----
From: "Niels de Vos" <ndevos at redhat.com>
To: "Joseph Fernandes" <josferna at redhat.com>
Cc: "Gluster Devel" <gluster-devel at gluster.org>
Sent: Saturday, February 27, 2016 4:28:43 PM
Subject: Re: [Gluster-devel] Bitrot/Tering : Bad files get migrated and hence corruption goes undetected.

On Fri, Feb 26, 2016 at 11:01:28PM -0500, Joseph Fernandes wrote:
> Well correctly we dont migrate the existing signature, the file starts
> it life fresh in the new tier(i.e get the bit rot version 1 on the new
> tier),
> Now this is also the case with any special xattr/attributes of the
> file.
> Again we rely heavily on the dht rebalance mechanism for migrations,
> which also doesnt carry special attributes/xattr.

Is there a good reason to not migrate the bitrot signature? Relying on
an existing functionality is fine, but if it does not address all your
needs, you have a valid use-case to improve it.

Niels

> 
> 
> ----- Original Message -----
> From: "Niels de Vos" <ndevos at redhat.com>
> To: "Joseph Fernandes" <josferna at redhat.com>
> Cc: "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Friday, February 26, 2016 10:33:11 PM
> Subject: Re: [Gluster-devel] Bitrot/Tering : Bad files get migrated and hence corruption goes undetected.
> 
> On Fri, Feb 26, 2016 at 09:32:46AM -0500, Joseph Fernandes wrote:
> > Hi All,
> > 
> > This is a discussion mail on the following issue, 
> > 
> > 1. Object is corrupted before it could be signed: In this case, the corrupted
> >    object is signed and get migrated upon I/O. There's no way to identify corruption
> >    for this set of objects.
> > 
> > 2. Object is signed (but not scrubbed) and corruption happens thereafter:
> >    In this case, as of now, integrity checking is not done on the fly
> >    and the object would get migrated (and signed again in the hot tier).
> > 
> > 
> > The (1) is definitely not a issue with bitrot with tiering. But (2) we can do something to avoid
> > corrupted file from getting migrated. Before we migrate files we can scrub it, but its just a naive
> > thought, any better suggestions? 
> 
> Is there a reason the existing signature can not be migrated? Why does
> it become invalid?
> 
> Niels