[Gluster-users] self-heal not working

Mon Aug 21 21:49:10 UTC 2017

Sure, it doesn't look like a split brain based on the output:

Brick node1.domain.tld:/data/myvolume/brick
Status: Connected
Number of entries in split-brain: 0

Brick node2.domain.tld:/data/myvolume/brick
Status: Connected
Number of entries in split-brain: 0

Brick node3.domain.tld:/srv/glusterfs/myvolume/brick
Status: Connected
Number of entries in split-brain: 0

> -------- Original Message --------
> Subject: Re: [Gluster-users] self-heal not working
> Local Time: August 21, 2017 11:35 PM
> UTC Time: August 21, 2017 9:35 PM
> From: bturner at redhat.com
> To: mabi <mabi at protonmail.ch>
> Gluster Users <gluster-users at gluster.org>
>
> Can you also provide:
>
> gluster v heal <my vol> info split-brain
>
> If it is split brain just delete the incorrect file from the brick and run heal again. I haven"t tried this with arbiter but I assume the process is the same.
>
> -b
>
> ----- Original Message -----
>> From: "mabi" <mabi at protonmail.ch>
>> To: "Ben Turner" <bturner at redhat.com>
>> Cc: "Gluster Users" <gluster-users at gluster.org>
>> Sent: Monday, August 21, 2017 4:55:59 PM
>> Subject: Re: [Gluster-users] self-heal not working
>>
>> Hi Ben,
>>
>> So it is really a 0 kBytes file everywhere (all nodes including the arbiter
>> and from the client).
>> Here below you will find the output you requested. Hopefully that will help
>> to find out why this specific file is not healing... Let me know if you need
>> any more information. Btw node3 is my arbiter node.
>>
>> NODE1:
>>
>> STAT:
>> File:
>> ‘/data/myvolume/brick/data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png’
>> Size: 0 Blocks: 38 IO Block: 131072 regular empty file
>> Device: 24h/36d Inode: 10033884 Links: 2
>> Access: (0644/-rw-r--r--) Uid: ( 33/www-data) Gid: ( 33/www-data)
>> Access: 2017-08-14 17:04:55.530681000 +0200
>> Modify: 2017-08-14 17:11:46.407404779 +0200
>> Change: 2017-08-14 17:11:46.407404779 +0200
>> Birth: -
>>
>> GETFATTR:
>> trusted.afr.dirty=0sAAAAAQAAAAAAAAAA
>> trusted.bit-rot.version=0sAgAAAAAAAABZhuknAAlJAg==
>> trusted.gfid=0sGYXiM9XuTj6lGs8LX58q6g==
>> trusted.glusterfs.d99af2fa-439b-4a21-bf3a-38f3849f87ec.xtime=0sWZG9sgAGOyo=
>>
>> NODE2:
>>
>> STAT:
>> File:
>> ‘/data/myvolume/brick/data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png’
>> Size: 0 Blocks: 38 IO Block: 131072 regular empty file
>> Device: 26h/38d Inode: 10031330 Links: 2
>> Access: (0644/-rw-r--r--) Uid: ( 33/www-data) Gid: ( 33/www-data)
>> Access: 2017-08-14 17:04:55.530681000 +0200
>> Modify: 2017-08-14 17:11:46.403704181 +0200
>> Change: 2017-08-14 17:11:46.403704181 +0200
>> Birth: -
>>
>> GETFATTR:
>> trusted.afr.dirty=0sAAAAAQAAAAAAAAAA
>> trusted.bit-rot.version=0sAgAAAAAAAABZhu6wAA8Hpw==
>> trusted.gfid=0sGYXiM9XuTj6lGs8LX58q6g==
>> trusted.glusterfs.d99af2fa-439b-4a21-bf3a-38f3849f87ec.xtime=0sWZG9sgAGOVE=
>>
>> NODE3:
>> STAT:
>> File:
>> /srv/glusterfs/myvolume/brick/data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png
>> Size: 0 Blocks: 0 IO Block: 4096 regular empty file
>> Device: ca11h/51729d Inode: 405208959 Links: 2
>> Access: (0644/-rw-r--r--) Uid: ( 33/www-data) Gid: ( 33/www-data)
>> Access: 2017-08-14 17:04:55.530681000 +0200
>> Modify: 2017-08-14 17:04:55.530681000 +0200
>> Change: 2017-08-14 17:11:46.604380051 +0200
>> Birth: -
>>
>> GETFATTR:
>> trusted.afr.dirty=0sAAAAAQAAAAAAAAAA
>> trusted.bit-rot.version=0sAgAAAAAAAABZe6ejAAKPAg==
>> trusted.gfid=0sGYXiM9XuTj6lGs8LX58q6g==
>> trusted.glusterfs.d99af2fa-439b-4a21-bf3a-38f3849f87ec.xtime=0sWZG9sgAGOc4=
>>
>> CLIENT GLUSTER MOUNT:
>> STAT:
>> File:
>> "/mnt/myvolume/data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png"
>> Size: 0 Blocks: 0 IO Block: 131072 regular empty file
>> Device: 1eh/30d Inode: 11897049013408443114 Links: 1
>> Access: (0644/-rw-r--r--) Uid: ( 33/www-data) Gid: ( 33/www-data)
>> Access: 2017-08-14 17:04:55.530681000 +0200
>> Modify: 2017-08-14 17:11:46.407404779 +0200
>> Change: 2017-08-14 17:11:46.407404779 +0200
>> Birth: -
>>
>> > -------- Original Message --------
>> > Subject: Re: [Gluster-users] self-heal not working
>> > Local Time: August 21, 2017 9:34 PM
>> > UTC Time: August 21, 2017 7:34 PM
>> > From: bturner at redhat.com
>> > To: mabi <mabi at protonmail.ch>
>> > Gluster Users <gluster-users at gluster.org>
>> >
>> > ----- Original Message -----
>> >> From: "mabi" <mabi at protonmail.ch>
>> >> To: "Gluster Users" <gluster-users at gluster.org>
>> >> Sent: Monday, August 21, 2017 9:28:24 AM
>> >> Subject: [Gluster-users] self-heal not working
>> >>
>> >> Hi,
>> >>
>> >> I have a replicat 2 with arbiter GlusterFS 3.8.11 cluster and there is
>> >> currently one file listed to be healed as you can see below but never gets
>> >> healed by the self-heal daemon:
>> >>
>> >> Brick node1.domain.tld:/data/myvolume/brick
>> >> /data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png
>> >> Status: Connected
>> >> Number of entries: 1
>> >>
>> >> Brick node2.domain.tld:/data/myvolume/brick
>> >> /data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png
>> >> Status: Connected
>> >> Number of entries: 1
>> >>
>> >> Brick node3.domain.tld:/srv/glusterfs/myvolume/brick
>> >> /data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png
>> >> Status: Connected
>> >> Number of entries: 1
>> >>
>> >> As once recommended on this mailing list I have mounted that glusterfs
>> >> volume
>> >> temporarily through fuse/glusterfs and ran a "stat" on that file which is
>> >> listed above but nothing happened.
>> >>
>> >> The file itself is available on all 3 nodes/bricks but on the last node it
>> >> has a different date. By the way this file is 0 kBytes big. Is that maybe
>> >> the reason why the self-heal does not work?
>> >
>> > Is the file actually 0 bytes or is it just 0 bytes on the arbiter(0 bytes
>> > are expected on the arbiter, it just stores metadata)? Can you send us the
>> > output from stat on all 3 nodes:
>> >
>> > $ stat <file on back end brick>
>> > $ getfattr -d -m - <file on back end brick>
>> > $ stat <file from gluster mount>
>> >
>> > Lets see what things look like on the back end, it should tell us why
>> > healing is failing.
>> >
>> > -b
>> >
>> >>
>> >> And how can I now make this file to heal?
>> >>
>> >> Thanks,
>> >> Mabi
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Gluster-users mailing list
>> >> Gluster-users at gluster.org
>> >> http://lists.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170821/bd2bb68d/attachment.html>