[Gluster-users] Healing Delays

Pranith Kumar Karampuri pkarampu at redhat.com
Mon Oct 3 06:45:53 UTC 2016


On Sun, Oct 2, 2016 at 5:49 AM, Lindsay Mathieson <
lindsay.mathieson at gmail.com> wrote:

> On 2/10/2016 12:48 AM, Lindsay Mathieson wrote:
>
>> Only the heal count does not change, it just does not seem to start. It
>> can take hours before it shifts, but once it does, its quite rapid. Node 1
>> has restarted and the heal count has been static at 511 shards for 45
>> minutes now. Nodes 1 & 2 have low CPU load, node 3 has glusterfsd pegged at
>> 800% CPU.
>>
>
> Ok, had a try at systematically reproducing it this morning and was
> actually unable to do so - quite weird. Testing was the same as last night
> - move all the VM's off a server and reboot it, wait for the healing to
> finish. This time I tried it with various different settings.
>
>
> Test 1
> ------
> cluster.granular-entry-heal no
> cluster.locking-scheme full
> Shards / Min: 350 / 8
>
>
> Test 2
> ------
> cluster.granular-entry-heal yes
> cluster.locking-scheme granular
> Shards / Min:  391 / 10
>
> Test 3
> ------
> cluster.granular-entry-heal yes
> cluster.locking-scheme granular
> heal command issued
> Shards / Min: 358 / 11
>
> Test 3
> ------
> cluster.granular-entry-heal yes
> cluster.locking-scheme granular
> heal full command issued
> Shards / Min: 358 / 27
>
>
> Best results were with cluster.granular-entry-heal=yes,
> cluster.locking-scheme=granular but they were all quite good.
>
>
> Don't know why it was so much worse last night - i/o load, cpu and memory
> were the same. However one thin that is different which I can't easily
> reproduce was that the cluster had been running for several weeks, but last
> night I rebooted all nodes. Could gluster be developing an issue after
> running for some time?


>From the algorithm point of view, the only thing that matters is load that
it needs to heal. Doesn't depend on age. So whether the load to heal is
100GB in very less time or in few months, the time to heal should be same.


>
>
>
> --
> Lindsay Mathieson
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>



-- 
Pranith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161003/7fbc8a77/attachment.html>


More information about the Gluster-users mailing list