[Gluster-users] bitrot log messages

[Jackie Tung]

Looks great, thank you!

FWIW, I think this is really useful info to have on the main gluster documentation.  Actual bitrot is a pretty rare occurrence, especially for newer deployments.

> On Oct 26, 2016, at 10:04 PM, Kotresh Hiremath Ravishankar <khiremat at redhat.com> wrote:
> 
> Hi Jackie,
> 
> Here is the sample output of scrub status where two files are corrupted.
> 
> root at FEDORA2:$gluster vol bitrot master scrub status
> 
> Volume name : master
> 
> State of scrub: Active (Idle)
> 
> Scrub impact: lazy
> 
> Scrub frequency: biweekly
> 
> Bitrot error log location: /var/log/glusterfs/bitd.log
> 
> Scrubber error log location: /var/log/glusterfs/scrub.log
> 
> 
> =========================================================
> 
> Node: localhost
> 
> Number of Scrubbed files: 4
> 
> Number of Skipped files: 0
> 
> Last completed scrub time: 2016-10-27 05:03:23
> 
> Duration of last scrub (D:M:H:M:S): 0:0:0:5
> 
> Error count: 2
> 
> Corrupted object's [GFID]:
> 
> b912c776-7253-4418-a35f-a37661d673b6
> 
> f63cb6fe-123d-4e4b-a6f1-d830f7f5392a
> 
> =========================================================
> 
> 
> Thanks and Regards,
> Kotresh H R
> 
> ----- Original Message -----
>> From: "Jackie Tung" <jackie at drive.ai>
>> To: "Kotresh Hiremath Ravishankar" <khiremat at redhat.com>
>> Cc: gluster-users at gluster.org
>> Sent: Wednesday, October 26, 2016 8:34:07 PM
>> Subject: Re: [Gluster-users] bitrot log messages
>> 
>> That’s great, thank you!  We plan to automatically look for these signs.
>> 
>> For “scrub status”, do you have a sample output for a positive hit you could
>> share easily?
>> 
>>> On Oct 26, 2016, at 12:05 AM, Kotresh Hiremath Ravishankar
>>> <khiremat at redhat.com> wrote:
>>> 
>>> Correcting the command..I had missed 'scrub' keyword.
>>> 
>>> "gluster vol bitrot <volname> scrub status"
>>> 
>>> Thanks and Regards,
>>> Kotresh H R
>>> 
>>> ----- Original Message -----
>>>> From: "Kotresh Hiremath Ravishankar" <khiremat at redhat.com>
>>>> To: "Jackie Tung" <jackie at drive.ai>
>>>> Cc: gluster-users at gluster.org
>>>> Sent: Wednesday, October 26, 2016 10:53:50 AM
>>>> Subject: Re: [Gluster-users] bitrot log messages
>>>> 
>>>> Hi Jackie,
>>>> 
>>>> "gluster vol bitrot <volname> status" should show the corrupted files
>>>> gfid.
>>>> If you want to get the info from logs, it will be logged as below.
>>>> 
>>>> [2016-10-26 05:21:20.767774] A [MSGID: 118023]
>>>> [bit-rot-scrub.c:246:bitd_compare_ckum] 0-master-bit-rot-0: CORRUPTION
>>>> DETECTED: Object /dir1/file1 {Brick: /bricks/brick1/b10 | GFID:
>>>> b912c776-7253-4418-a35f-a37661d673b6}
>>>> [2016-10-26 05:21:20.770294] A [MSGID: 118024]
>>>> [bit-rot-scrub.c:266:bitd_compare_ckum] 0-master-bit-rot-0: Marking
>>>> /dir1/file1 [GFID: b912c776-7253-4418-a35f-a37661d673b6 | Brick:
>>>> /bricks/brick1/b10] as corrupted..
>>>> 
>>>> 
>>>> Thanks and Regards,
>>>> Kotresh H R
>>>> 
>>>> ----- Original Message -----
>>>>> From: "Jackie Tung" <jackie at drive.ai>
>>>>> To: gluster-users at gluster.org
>>>>> Sent: Wednesday, October 26, 2016 2:14:28 AM
>>>>> Subject: [Gluster-users] bitrot log messages
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> Redhat documentation says that things will get logged to bitd.log, and
>>>>> scrub.log. These files are pretty big - even when we only take the “ E “
>>>>> log
>>>>> level lines.
>>>>> 
>>>>> https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/chap-Detecting_Data_Corruption.html
>>>>> 
>>>>> Anyone know which particular message patterns we should be actively
>>>>> looking
>>>>> for to detect bitrot?
>>>>> 
>>>>> Thanks,
>>>>> Jackie
>>>>> 
>>>>> 
>>>>> 
>>>>> The information in this email is confidential and may be legally
>>>>> privileged.
>>>>> It is intended solely for the addressee. Access to this email by anyone
>>>>> else
>>>>> is unauthorized. If you are not the intended recipient, any disclosure,
>>>>> copying, distribution or any action taken or omitted to be taken in
>>>>> reliance
>>>>> on it, is prohibited and may be unlawful.
>>>>> 
>>>>> _______________________________________________
>>>>> Gluster-users mailing list
>>>>> Gluster-users at gluster.org
>>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>> 
>> 
>> --
>> 
>> 
>> The information in this email is confidential and may be legally
>> privileged. It is intended solely for the addressee. Access to this email
>> by anyone else is unauthorized. If you are not the intended recipient, any
>> disclosure, copying, distribution or any action taken or omitted to be
>> taken in reliance on it, is prohibited and may be unlawful.
>> 


-- 
 

The information in this email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this email 
by anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be 
taken in reliance on it, is prohibited and may be unlawful.