[Gluster-users] canceling full heal 3.8

[David Gossage]

On Sat, Aug 27, 2016 at 11:01 PM, David Gossage <dgossage at carouselchecks.com
> wrote:

> On Sat, Aug 27, 2016 at 9:55 PM, David Gossage <
> dgossage at carouselchecks.com> wrote:
>
>> On Sat, Aug 27, 2016 at 5:35 PM, David Gossage <
>> dgossage at carouselchecks.com> wrote:
>>
>>> On Aug 27, 2016 4:37 PM, "Lindsay Mathieson" <
>>> lindsay.mathieson at gmail.com> wrote:
>>> >
>>> > On 28/08/2016 6:07 AM, David Gossage wrote:
>>> >>
>>> >> 7 hours after starting full heal shards still haven't started
>>> healing, and count from heal statistics heal-count has only reached 1800
>>> out of 19000 shards.  shards dir hasn't even been recreated yet.  Creation
>>> of the non sharded stubs (do they have a more official term?) in the
>>> visible mount point was as speedy as expected.  shards are painfully slow.
>>> >
>>> >
>>> >
>>> > Is your CPU usage through the roof?
>>>
>>> Currently it has almost no activity.  First node yesterday got a bit
>>> high.  But 2nd node today that has issues is pretty low.
>>> >
>>> > If you haven't already, I'd suggest
>>> >
>>> > - changing "cluster.data-self-heal-algorithm" to "full"
>>> >
>>> > - And restarting the gluster volume if possible
>>> >
>>>
>>> I'll shut down vm's later tonight and see if that helps at all.
>>>
>>
>> applied  "cluster.data-self-heal-algorithm" to "full"
>>
>> stopped volume - started volume
>>
>> cpu activity barely noticeable. heal count crawling at 1 new addition to
>> list every minute or 2 and stil just building a list it hasnt started
>> making a .shard directory
>>
>
> logging into each linux vm and running from / 'find . | xargs stat ' seeme
> dto make count ump a bit faster. wasnt quite sure best way to repeatthat
> for windows vm's so just ran full system virus scans.
>
> Still after 15+ plus hours now its listed 3600/19000 shards in to be
> healed list and started healing none.
>

24hours and its added 25% of shards to list and not started any.  I sense
this will be a pleasant monday morning tomorrrow.

> >
>>> > I have a suspicion something changed recently with heal, I've noticed
>>> that it takes a long time (hours) to kick in when the diff algorithm is
>>> used. I don't recall it doing this with 3.7.11
>>> >
>>> >
>>> > --
>>> > Lindsay Mathieson
>>> >
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160828/f03f8afd/attachment.html>