[Gluster-users] Entries in heal pending

[Strahil Nikolov]

 You are the second person (excluding me) who observes this behaviour.The easiest way to resolve this is to:1. Check which file is newest (there is a timestamp in the file) for :/gluster_bricks/data/data/7ac28c32-947b-4ad5-8d69-213a205f06e8/images/d7c11f2e-58e4-4fe1-8236-6ded0f4dd757/e566f230-df72-4073-aecf-7e5a8d6b569b.metaAnd for :/gluster_bricks/data/data/7ac28c32-947b-4ad5-8d69-213a205f06e8/images/079904a4-71af-492c-bb2f-b45a918e8a2e/fce4b64d-2444-4f11-b226-db75bb2960c2.meta

Let's assume node3storage.ovirt.local has the newest data.
2.Then you can just backup (just in case you change your mind) the files locally and rsync from node3storage.ovirt.local (actually replace with the node with newest timestamp in the file) to the other bricks.
3. Run a gluster heal just to notify gluster that the issue is resolved.
In my case one of the nodes had a newer version of the file (I am using replica 2 arbiter 1 volume) and the gfid was different and this prevented Gluster from healing that.
Usually , oVirt just updates the timestamp in the meta files - so even an older version is not a problem.
P.S.: What version of gluster are you using. I suppose v6.5 or v6.6 , right ?
Best Regards,Strahil Nikolov



    В сряда, 18 декември 2019 г., 16:18:12 ч. Гринуич+2, Szilágyi Balázs <szilagyi.balazs at pcvilag.hu> написа:  
 
 Dear Gluster Users,

I'm a newbie to gluster storage and during the stability testing I made 
a node reboot and I've got some heal issues after that, that i'm unable 
to fix.
Anyway the vm's are running fine from the storage, and I did not 
discovered any data corruption.
The system is oVirt version 4.3.7. I have 3 nodes with a replica 3. 
Please let me know what to do with the pending heals that are unable to 
finish.
Also let me know if some more details are necessary.

Thanks,
   Balazs

[root at node2 ~]# gluster volume status data
Status of volume: data
Gluster process                             TCP Port  RDMA Port Online  Pid
------------------------------------------------------------------------------
Brick node1storage.ovirt.local:/gluster_bri
cks/data/data                               49152     0 Y       4187
Brick node2storage.ovirt.local:/gluster_bri
cks/data/data                               49154     0 Y       6163
Brick node3storage.ovirt.local:/gluster_bri
cks/data/data                               49154     0 Y       19439
Self-heal Daemon on localhost               N/A       N/A Y       3136
Self-heal Daemon on node3storage.ovirt.loca
l                                           N/A       N/A Y       5876
Self-heal Daemon on node1storage.ovirt.loca
l                                           N/A       N/A Y       15479

Task Status of Volume data
------------------------------------------------------------------------------
There are no active volume tasks

[root at node2 ~]#
[root at node2 ~]# gluster volume heal data info summary
Brick node1storage.ovirt.local:/gluster_bricks/data/data
Status: Connected
Total Number of entries: 2
Number of entries in heal pending: 2
Number of entries in split-brain: 0
Number of entries possibly healing: 0

Brick node2storage.ovirt.local:/gluster_bricks/data/data
Status: Connected
Total Number of entries: 2
Number of entries in heal pending: 2
Number of entries in split-brain: 0
Number of entries possibly healing: 0

Brick node3storage.ovirt.local:/gluster_bricks/data/data
Status: Connected
Total Number of entries: 0
Number of entries in heal pending: 0
Number of entries in split-brain: 0
Number of entries possibly healing: 0

[root at node2 ~]# gluster volume heal data info
Brick node1storage.ovirt.local:/gluster_bricks/data/data
/7ac28c32-947b-4ad5-8d69-213a205f06e8/images/d7c11f2e-58e4-4fe1-8236-6ded0f4dd757/e566f230-df72-4073-aecf-7e5a8d6b569b.meta
/7ac28c32-947b-4ad5-8d69-213a205f06e8/images/079904a4-71af-492c-bb2f-b45a918e8a2e/fce4b64d-2444-4f11-b226-db75bb2960c2.meta
Status: Connected
Number of entries: 2

Brick node2storage.ovirt.local:/gluster_bricks/data/data
/7ac28c32-947b-4ad5-8d69-213a205f06e8/images/d7c11f2e-58e4-4fe1-8236-6ded0f4dd757/e566f230-df72-4073-aecf-7e5a8d6b569b.meta
/7ac28c32-947b-4ad5-8d69-213a205f06e8/images/079904a4-71af-492c-bb2f-b45a918e8a2e/fce4b64d-2444-4f11-b226-db75bb2960c2.meta
Status: Connected
Number of entries: 2

Brick node3storage.ovirt.local:/gluster_bricks/data/data
Status: Connected
Number of entries: 0

________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users at gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20191218/234d4a01/attachment.html>