<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 02/24/2017 01:25 PM,
<a class="moz-txt-link-abbreviated" href="mailto:max.degraaf@kpn.com">max.degraaf@kpn.com</a> wrote:<br>
</div>
<blockquote
cite="mid:BD35B08176463944A213E3C94AD3CEFE17FC2DA7@CPEMS-KPN510.KPNCNL.LOCAL"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;
mso-fareast-language:EN-US;}
pre
{mso-style-priority:99;
mso-style-link:"HTML - vooraf opgemaakt Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;
mso-fareast-language:NL;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Ballontekst Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
color:black;
mso-fareast-language:EN-US;}
span.HTML-voorafopgemaaktChar
{mso-style-name:"HTML - vooraf opgemaakt Char";
mso-style-priority:99;
mso-style-link:"HTML - vooraf opgemaakt";
font-family:Consolas;
color:black;
mso-fareast-language:EN-US;}
span.E-mailStijl20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.E-mailStijl21
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BallontekstChar
{mso-style-name:"Ballontekst Char";
mso-style-priority:99;
mso-style-link:Ballontekst;
font-family:"Tahoma","sans-serif";
color:black;
mso-fareast-language:EN-US;}
span.E-mailStijl24
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Any
advice on the sequence of updating? Server of client first?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">I
assume it’s a simple in-place update where configuration is
preserved. Right?</span></p>
</div>
</blockquote>
<br>
Configuration will be preserved. I don't know actual procedure for
rolling upgrade (other than servers first with one after other). May
be somebody else in this ML can provide you more info or just a
google search will find you some blogs.<br>
<br>
<br>
Regards<br>
Rafi KC<br>
<br>
<br>
<blockquote
cite="mid:BD35B08176463944A213E3C94AD3CEFE17FC2DA7@CPEMS-KPN510.KPNCNL.LOCAL"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:NL">Van:</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:NL">
Mohammed Rafi K C [<a class="moz-txt-link-freetext" href="mailto:rkavunga@redhat.com">mailto:rkavunga@redhat.com</a>] <br>
<b>Verzonden:</b> vrijdag 24 februari 2017 08:49<br>
<b>Aan:</b> Graaf, Max de; <a class="moz-txt-link-abbreviated" href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
<b>Onderwerp:</b> Re: [Gluster-users] self heal failed,
on /<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On 02/24/2017 11:47 AM, <a
moz-do-not-send="true" href="mailto:max.degraaf@kpn.com">
<a class="moz-txt-link-abbreviated" href="mailto:max.degraaf@kpn.com">max.degraaf@kpn.com</a></a> wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">The
version on the server of this specific mount is 3.7.11.
The client is running version 3.4.2.
</span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-language:NL"><br>
It is always better to have everything in one version, all
clients and all servers. In this case there is huge gap
between the versions, 3.7 and 3.4 .<br>
<br>
An additional thing is the code running on 3.4 is replicaV1
code and on 3.7 it v2, meaning there is huge difference to
the logic of replication/healing. So I recommend to keep all
the gluster instances to the same version<br>
<br>
<br>
~Rafi <br>
<br>
<br>
<br>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">There
is more to that. This client is actually mounting to volumes
where the other server is running 3.4.2 as well. What’s your
advice, update that other server to 3.7.11 (or higher)
first? Of start with the client update?</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:NL">Van:</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:NL">
Mohammed Rafi K C [<a moz-do-not-send="true"
href="mailto:rkavunga@redhat.com">mailto:rkavunga@redhat.com</a>]
<br>
<b>Verzonden:</b> vrijdag 24 februari 2017 07:02<br>
<b>Aan:</b> Graaf, Max de; <a moz-do-not-send="true"
href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
<b>Onderwerp:</b> Re: [Gluster-users] self heal failed,
on /</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<p class="MsoNormal">On 02/23/2017 12:18 PM, <a
moz-do-not-send="true" href="mailto:max.degraaf@kpn.com">
<a class="moz-txt-link-abbreviated" href="mailto:max.degraaf@kpn.com">max.degraaf@kpn.com</a></a> wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span lang="EN-US">Hi,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">We have a 4 node
glusterfs setup that seems to be running without any
problems. We can’t find any problems with replication or
whatever.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">We also have 4
machines running the glusterfs client. On all 4 machines
we see the following error in the logs at random moments:</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">[2017-02-23
00:04:33.168778] I
[afr-self-heal-common.c:2869:afr_log_self_heal_completion_status]
0-aab-replicate-0: metadata self heal is successfully
completed, metadata self heal from source aab-client-0
to aab-client-1, aab-client-2, aab-client-3, metadata -
Pending matrix: [ [ 0 0 0 0 ] [ 0 0 0 0 ] [ 0 0 0 0 ] [ 0
0 0 0 ] ], on /</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">[2017-02-23
00:09:34.431089] E
[afr-self-heal-common.c:2869:afr_log_self_heal_completion_status]
0-aab-replicate-0: metadata self heal failed, on /</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">[2017-02-23
00:14:34.948975] I
[afr-self-heal-common.c:2869:afr_log_self_heal_completion_status]
0-aab-replicate-0: metadata self heal is successfully
completed, metadata self heal from source aab-client-0
to aab-client-1, aab-client-2, aab-client-3, metadata -
Pending matrix: [ [ 0 0 0 0 ] [ 0 0 0 0 ] [ 0 0 0 0 ] [ 0
0 0 0 ] ], on /</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">The content within the
glusterfs filesystems is rather static with only minor
changes on it. This “self heal failed” is printed
randomly in the logs on the glusterfs client. It’s printed
even at moment where nothing has changed within the
glusterfs filesystem. When it is printed, its never on
multiple servers at the same time. What we also don’t
understand : the error indicates self heal failed on root
“/”. In the root of this glusterfs mount there only 2
folders and no files are ever written at the root level.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Any thoughts?</span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:12.0pt"><br>
From the logs, It looks like an older version of gluster ,
probably 3.5 . Please confirm your glusterfs version. The
version is pretty old and it may be moved End of Life. And
this is AFR v1 , where the latest stable version runs with
AFRv2.<br>
<br>
So I would suggest you to upgrade to a later version may be
3.8 .<br>
<br>
If you still want to go with this version, I can give it a
try. Let me know the version, volume info and volume status.
Still I will suggest to upgrade ;)<br>
<br>
<br>
Regards<br>
Rafi KC<br>
<br>
<br>
<br>
<br>
<br>
</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><br>
<br>
<br>
<br>
</span><o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Gluster-users mailing list<o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a><o:p></o:p></pre>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-language:NL"><o:p> </o:p></span></p>
</div>
</blockquote>
<br>
</body>
</html>