[Gluster-users] No healing, errno 22

[Zenon Panoussis]

> -Was this an upgraded setup or a fresh v9.0 install? 

It was freshly installed 8.3 Centos rpms and upgraded to 9.0.
I enabled granular after the upgrade.

> - When there are entries yet to be healed, the CLI should 
> have prevented you toggling this option - was that not the 
> case?

Indeed, enabling granular was only possible when there were
0 files to heal. Re-disabling it, however, did not impose this
limitation.

> - Can you find the directory name corresponding to the gfid 
> 011fcc1b-4d90-4c36-86ec-488aaa4db3b8 (use 
> https://github.com/gluster/glusterfs/blob/master/extras/gfid-to-dirname.sh 
> if needed) and see if all files/ sub directories (first level 
> only) inside it are same on all 3 bricks?

Nifty little script!

[root at node03 ~]# ./gfid2dirname.sh /gfs/gv0 011fcc1b-4d90-4c36-86ec-488aaa4db3b8
Location of the directory corresponding to gfid:011fcc1b-4d90-4c36-86ec-488aaa4db3b8 is /gfs/gv0/vmail/net/provocation/oracle/Maildir/.Sent/cur/

I get the same answer on all three nodes. This directory contains
no subdirectories, only files.

[root at node01 ~]# find /gfs/gv0/vmail/net/provocation/oracle/Maildir/.Sent/cur/ -type f |wc -l
10264

[root at node02 ~]# find /gfs/gv0/vmail/net/provocation/oracle/Maildir/.Sent/cur/ -type f |wc -l
10604

[root at node03 ~]# find /gfs/gv0/vmail/net/provocation/oracle/Maildir/.Sent/cur/ -type f |wc -l
10603

The figures don't fully add up to 4/343/344, but are very close.
Nothing is is in split-brain, so it simply looks like node01 is
lagging behind the other two.