[Gluster-users] how to verify bitrot signed file manually?

[Kotresh Hiremath Ravishankar]

Hi Amudhan,

Sorry for the late response as I was busy with other things. You are right
bitrot uses sha256 for checksum.
If file-1, file-2 are marked bad, the I/O should be errored out with EIO.
If that is not happening, we need
to look further into it. But what's the file contents of file-1 and file-2
on the replica bricks ? Are they
matching ?

Thanks and Regards,
Kotresh HR

On Mon, Sep 25, 2017 at 4:19 PM, Amudhan P <amudhan83 at gmail.com> wrote:

> resending mail.
>
>
> On Fri, Sep 22, 2017 at 5:30 PM, Amudhan P <amudhan83 at gmail.com> wrote:
>
>> ok, from bitrot code I figured out gluster using sha256 hashing algo.
>>
>>
>> Now coming to the problem, during scrub run in my cluster some of my
>> files were marked as bad in few set of nodes.
>> I just wanted to confirm bad file. so, I have used "sha256sum" tool in
>> Linux to manually get file hash.
>>
>> here is the result.
>>
>> file-1, file-2 marked as bad by scrub and file-3 is healthy.
>>
>> file-1 sha256 and bitrot signature value matches but still it's been
>> marked as bad.
>>
>> file-2 sha256 and bitrot signature value don't match, could be a victim
>> of bitrot or bitflip.file is still readable without any issue and no errors
>> found in the drive.
>>
>> file-3 sha256 and bitrot signature matches and healthy.
>>
>>
>> file-1 output from
>>
>> "sha256sum" = "71eada9352b1352aaef0f806d3d56
>> 1768ce2df905ded1668f665e06eca2d0bd4"
>>
>>
>> "getfattr -m. -e hex -d "
>> # file: file-1
>> trusted.bit-rot.bad-file=0x3100
>> trusted.bit-rot.signature=0x01020000000000000071eada9352b135
>> 2aaef0f806d3d561768ce2df905ded1668f665e06eca2d0bd4
>> trusted.bit-rot.version=0x020000000000000058e4f3b40006793d
>> trusted.ec.config=0x0000080a02000200
>> trusted.ec.dirty=0x00000000000000000000000000000000
>> trusted.ec.size=0x0000000718996701
>> trusted.ec.version=0x0000000000038c4c0000000000038c4d
>> trusted.gfid=0xf078a24134fe4f9bb953eca8c28dea9a
>>
>> output scrub log:
>> [2017-09-02 13:02:20.311160] A [MSGID: 118023]
>> [bit-rot-scrub.c:244:bitd_compare_ckum] 0-qubevaultdr-bit-rot-0:
>> CORRUPTION DETECTED: Object /file-1 {Brick: /media/disk16/brick16 | GFID:
>> f078a241-34fe-4f9b-b953-eca8c28dea9a}
>> [2017-09-02 13:02:20.311579] A [MSGID: 118024]
>> [bit-rot-scrub.c:264:bitd_compare_ckum] 0-qubevaultdr-bit-rot-0: Marking
>> /file-1 [GFID: f078a241-34fe-4f9b-b953-eca8c28dea9a | Brick:
>> /media/disk16/brick16] as corrupted..
>>
>> file-2 output from
>>
>> "sha256sum" = "c41ef9c81faed4f3e6010ea67984c
>> 3cfefd842f98ee342939151f9250972dcda"
>>
>>
>> "getfattr -m. -e hex -d "
>> # file: file-2
>> trusted.bit-rot.bad-file=0x3100
>> trusted.bit-rot.signature=0x0102000000000000009162cb17d4f0be
>> e676fcb7830c5286d05b8e8940d14f3d117cb90b7b1defc129
>> trusted.bit-rot.version=0x020000000000000058e4f3b400019bb2
>> trusted.ec.config=0x0000080a02000200
>> trusted.ec.dirty=0x00000000000000000000000000000000
>> trusted.ec.size=0x00000000403433f6
>> trusted.ec.version=0x000000000000201a000000000000201b
>> trusted.gfid=0xa50012b0a632477c99232313928d239a
>>
>> output scrub log:
>> [2017-09-02 05:18:14.003156] A [MSGID: 118023]
>> [bit-rot-scrub.c:244:bitd_compare_ckum] 0-qubevaultdr-bit-rot-0:
>> CORRUPTION DETECTED: Object /file-2 {Brick: /media/disk13/brick13 | GFID:
>> a50012b0-a632-477c-9923-2313928d239a}
>> [2017-09-02 05:18:14.006629] A [MSGID: 118024]
>> [bit-rot-scrub.c:264:bitd_compare_ckum] 0-qubevaultdr-bit-rot-0: Marking
>> /file-2 [GFID: a50012b0-a632-477c-9923-2313928d239a | Brick:
>> /media/disk13/brick13] as corrupted..
>>
>>
>> file-3 output from
>>
>> "sha256sum" = "a590735b3c8936cc7ca9835128a19
>> c38a3f79c8fd53fddc031a9349b7e273f27"
>>
>>
>> "getfattr -m. -e hex -d "
>> # file: file-3
>> trusted.bit-rot.signature=0x010200000000000000a590735b3c8936
>> cc7ca9835128a19c38a3f79c8fd53fddc031a9349b7e273f27
>> trusted.bit-rot.version=0x020000000000000058e4f3b400019bb2
>> trusted.ec.config=0x0000080a02000200
>> trusted.ec.dirty=0x00000000000000000000000000000000
>> trusted.ec.size=0x000000003530fc96
>> trusted.ec.version=0x0000000000001a980000000000001a99
>> trusted.gfid=0x10d8920e42cd42cf9448b8bf3941c192
>>
>>
>>
>> most of the bitrot bad files are in the set of new nodes and data were
>> uploaded using gluster 3.10.1. no drive issues are any kind of error msgs
>> in logs.
>>
>> what could be gone wrong?
>>
>> regards
>> Amudhan
>>
>> On Thu, Sep 21, 2017 at 1:23 PM, Amudhan P <amudhan83 at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I have a file in my brick which was signed by bitrot and latter when
>>> running scrub it was marked as bad.
>>>
>>> Now, I want to verify file again manually. just to clarify my doubt
>>>
>>> how can I do this?
>>>
>>>
>>> regards
>>> Amudhan
>>>
>>
>>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>



-- 
Thanks and Regards,
Kotresh H R
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170929/514e7a0c/attachment.html>