<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 07/20/2017 03:42 PM, yayo (j) wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAGK=3kycgTSLxRLZSPNZi+ERh_=OhDFH29hXHzWEPnsV3vmhAQ@mail.gmail.com">
<div dir="ltr">
<div class="gmail_extra"><br>
<div class="gmail_quote">2017-07-20 11:34 GMT+02:00
Ravishankar N <span dir="ltr"><<a
href="mailto:ravishankar@redhat.com" target="_blank"
moz-do-not-send="true">ravishankar@redhat.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span
class="gmail-m_7883186325552308780gmail-"> <br>
<div
class="gmail-m_7883186325552308780gmail-m_9132661499198415129moz-cite-prefix">Could
you check if the self-heal daemon on all nodes is
connected to the 3 bricks? You will need to check
the glustershd.log for that.<br>
</div>
</span> If it is not connected, try restarting the shd
using `gluster volume start engine force`, then launch
the heal command like you did earlier and see if heals
happen.<br>
<br>
</div>
</blockquote>
<div><br>
</div>
<div>I've executed the command on all 3 nodes (Know is
enougth only one) , after that the "heal" command report
elements between 6 and 10 ... (sometime 6, sometime 8,
sometime 10)</div>
<div><br>
</div>
<div><br>
</div>
<div>Log on glustershd.log don't say anything :</div>
</div>
</div>
</div>
</blockquote>
<br>
But it does say something. All these gfids of completed heals in
the log below are the for the ones that you have given the getfattr
output of. So what is likely happening is there is an intermittent
connection problem between your mount and the brick process, leading
to pending heals again after the heal gets completed, which is why
the numbers are varying each time. You would need to check why that
is the case.<br>
Hope this helps,<br>
Ravi<br>
<br>
<blockquote type="cite"
cite="mid:CAGK=3kycgTSLxRLZSPNZi+ERh_=OhDFH29hXHzWEPnsV3vmhAQ@mail.gmail.com">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
</div>
</div>
<blockquote style="margin:0px 0px 0px
40px;border:none;padding:0px">
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>[2017-07-20 09:58:46.573079] I [MSGID: 108026]
[afr-self-heal-common.c:1254:<wbr>afr_log_selfheal]
0-engine-replicate-0: Completed data selfheal on
e6dfd556-340b-4b76-b47b-<wbr>7b6f5bd74327.
sources=[0] 1 sinks=2</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>[2017-07-20 09:59:22.995003] I [MSGID: 108026]
[afr-self-heal-metadata.c:51:_<wbr>_afr_selfheal_metadata_do]
0-engine-replicate-0: performing metadata selfheal
on f05b9742-2771-484a-85fc-<wbr>5b6974bcef81</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>[2017-07-20 09:59:22.999372] I [MSGID: 108026]
[afr-self-heal-common.c:1254:<wbr>afr_log_selfheal]
0-engine-replicate-0: Completed metadata selfheal on
f05b9742-2771-484a-85fc-<wbr>5b6974bcef81.
sources=[0] 1 sinks=2</i></div>
</div>
</div>
</div>
</blockquote>
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> If it doesn't, please provide the
getfattr outputs of the 12 files from all 3 nodes using
`getfattr -d -m . -e hex <i>/gluster/engine/brick/</i>path-to-<wbr>file`
?<br>
<br>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
<blockquote style="margin:0px 0px 0px
40px;border:none;padding:0px">
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><b><i>NODE01:</i></b></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.68</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000120000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xe3565b5014954e5bae883bceca47<wbr>b7d9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.48</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x0000000e0000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x676067891f344c1586b8c0d05b07<wbr>f187</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/19d71267-52a4-42a3-<wbr>bb1e-e3145361c0c2/7a215635-<wbr>02f3-47db-80db-8b689c6a8f01</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000550000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x8aa745646740403ead51f56d9ca5<wbr>d7a7</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000c80000000000000000000<wbr>00000000000000d4f2290000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.60</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000070000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x4e33ac33dddb4e29b4a351770b81<wbr>166a</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>dom_md/ids</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x0f0000000000000059647d5b0004<wbr>47e9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x2581cb9ac2b74bd9ac17a09bd2f0<wbr>01b3</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/__DIRECT_<wbr>IO_TEST__</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xf05b97422771484a85fc5b6974bc<wbr>ef81</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000000000000000000000<wbr>000000000000000000000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/88d41053-a257-4272-<wbr>9e2e-2f3de0743b81/6573ed08-<wbr>d3ed-4d12-9227-2c95941e1ad6</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000010000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x0f0000000000000059647d5b0004<wbr>47e9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xe6dfd556340b4b76b47b7b6f5bd7<wbr>4327</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.64</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x0000000a0000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x9ef88647cfe64a35a38ca5173c9e<wbr>8fc0</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><i><br>
</i></div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><b><i><br>
</i></b></div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><b><i>NODE02:</i></b></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.68</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x0000001a0000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xe3565b5014954e5bae883bceca47<wbr>b7d9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.48</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000010000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x0000000c0000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x676067891f344c1586b8c0d05b07<wbr>f187</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/19d71267-52a4-42a3-<wbr>bb1e-e3145361c0c2/7a215635-<wbr>02f3-47db-80db-8b689c6a8f01</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x0000008e0000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x8aa745646740403ead51f56d9ca5<wbr>d7a7</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000c80000000000000000000<wbr>00000000000000d4f2290000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.60</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000090000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x4e33ac33dddb4e29b4a351770b81<wbr>166a</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>dom_md/ids</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000010000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x2581cb9ac2b74bd9ac17a09bd2f0<wbr>01b3</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/__DIRECT_<wbr>IO_TEST__</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xf05b97422771484a85fc5b6974bc<wbr>ef81</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000000000000000000000<wbr>000000000000000000000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/88d41053-a257-4272-<wbr>9e2e-2f3de0743b81/6573ed08-<wbr>d3ed-4d12-9227-2c95941e1ad6</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000020000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xe6dfd556340b4b76b47b7b6f5bd7<wbr>4327</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.64</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.engine-client-2=<wbr>0x000000120000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x9ef88647cfe64a35a38ca5173c9e<wbr>8fc0</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><i><br>
</i></div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><i><br>
</i></div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><i><br>
</i></div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><i><br>
</i></div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><b>NODE04</b>:</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.68</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xe3565b5014954e5bae883bceca47<wbr>b7d9</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.48</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x676067891f344c1586b8c0d05b07<wbr>f187</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/19d71267-52a4-42a3-<wbr>bb1e-e3145361c0c2/7a215635-<wbr>02f3-47db-80db-8b689c6a8f01</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x8aa745646740403ead51f56d9ca5<wbr>d7a7</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000c80000000000000000000<wbr>00000000000000d4f2290000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.60</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x4e33ac33dddb4e29b4a351770b81<wbr>166a</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>dom_md/ids</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x2581cb9ac2b74bd9ac17a09bd2f0<wbr>01b3</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/__DIRECT_<wbr>IO_TEST__</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x0200000000000000596484e20006<wbr>237b</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xf05b97422771484a85fc5b6974bc<wbr>ef81</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000000000000000000000<wbr>000000000000000000000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/88d41053-a257-4272-<wbr>9e2e-2f3de0743b81/6573ed08-<wbr>d3ed-4d12-9227-2c95941e1ad6</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0xe6dfd556340b4b76b47b7b6f5bd7<wbr>4327</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i><br>
</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>getfattr: Removing leading '/' from absolute
path names</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.64</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>trusted.gfid=<wbr>0x9ef88647cfe64a35a38ca5173c9e<wbr>8fc0</i></div>
</div>
</div>
</div>
</blockquote>
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>hum.... Is selinux the problem? but on node04 was
disabled (AFTER GLUSTER JOIN, I hope to remember) ... You
think I needs to relabel? how?</div>
<div><br>
</div>
</div>
</div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>[root@node01 ~]# sestatus</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>SELinux status: disabled</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><i><br>
</i></div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>[root@node02 ~]# sestatus</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>SELinux status: disabled</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><i><br>
</i></div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>[root@node04 ~]# sestatus</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>SELinux status: disabled</i></div>
</div>
</div>
</div>
</blockquote>
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div><br>
</div>
<div>Thank you </div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> Thanks,<br>
Ravi
<div>
<div class="gmail-m_7883186325552308780gmail-h5"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> 2. Are these 12
files also present in the 3rd data
brick?<br>
</div>
</blockquote>
<div><br>
</div>
<div>I've checked right now: all files
exists in all 3 nodes </div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> 3. Can you provide
the output of `gluster volume info` for
the this volume?</div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
<blockquote style="margin:0px 0px 0px
40px;border:none;padding:0px">
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Volume Name: engine</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Type: Replicate</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Volume ID:
d19c19e3-910d-437b-8ba7-4f2a23<wbr>d17515</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Status: Started</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Snapshot Count: 0</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Number of Bricks: 1 x 3 = 3</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Transport-type: tcp</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Bricks:</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Brick1:
node01:/gluster/engine/brick</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Brick2:
node02:/gluster/engine/brick</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Brick3:
node04:/gluster/engine/brick</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>Options Reconfigured:</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>nfs.disable: on</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>performance.readdir-ahead: on</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>transport.address-family: inet</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>storage.owner-uid: 36</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>performance.quick-read: off</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>performance.read-ahead: off</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>performance.io-cache: off</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>performance.stat-prefetch: off</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>performance.low-prio-threads: 32</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>network.remote-dio: off</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>cluster.eager-lock: enable</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>cluster.quorum-type: auto</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>cluster.server-quorum-type:
server</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>cluster.data-self-heal-algorit<wbr>hm:
full</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>cluster.locking-scheme: granular</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>cluster.shd-max-threads: 8</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>cluster.shd-wait-qlength: 10000</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>features.shard: on</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>user.cifs: off</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>storage.owner-gid: 36</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>features.shard-block-size: 512MB</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>network.ping-timeout: 30</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>performance.strict-o-direct: on</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>cluster.granular-entry-heal: on</i></div>
</div>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><i>auth.allow: *</i></div>
</div>
</div>
</div>
</blockquote>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div> server.allow-insecure: on</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-"><br>
<blockquote type="cite">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>Some extra info:</div>
<div><br>
</div>
<div>We have recently
changed the gluster from:
2 (full repliacated) + 1
arbiter to 3 full
replicated cluster </div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</span> Just curious, how did you do
this? `remove-brick` of arbiter brick
followed by an `add-brick` to increase
to replica-3?<br>
<br>
</div>
</blockquote>
<div><br>
</div>
<div>Yes</div>
<br>
<div>
<div
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs
gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-nginx"
style="display:block;overflow-x:auto;padding:0.5em;color:rgb(51,51,51);background:rgb(248,248,248);font-family:monospace">
<div><br>
</div>
<div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-attribute">#gluster</span>
volume remove-brick engine replica <span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-number"
style="color:rgb(136,0,0)">2</span>
node03:/gluster/data/brick force <b>(OK!)</b></div>
<div><br>
</div>
<div>#gluster volume heal engine <span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-literal"
style="color:rgb(120,169,96)">info</span>
<b>(<span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-literal"
style="color:rgb(120,169,96)">no</span>
entries!)</b></div>
<div><br>
</div>
<div>#gluster volume add-brick engine
replica <span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-number"
style="color:rgb(136,0,0)">3</span>
node04:/gluster/engine/brick <b>(OK!)</b></div>
<div><br>
</div>
<div><b>After some minutes</b></div>
<div><br>
</div>
<div>[root<span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-variable"
style="color:rgb(188,96,96)">@node01</span>
~]<span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">#
gluster volume heal engine info</span></div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Brick
node01:/gluster/engine/brick</span></div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Status:
Connected</span></div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Number
of entries: 0</span></div>
<div><br>
</div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Brick
node02:/gluster/engine/brick</span></div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Status:
Connected</span></div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Number
of entries: 0</span></div>
<div><br>
</div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Brick
node04:/gluster/engine/brick</span></div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Status:
Connected</span></div>
<div><span
class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment"
style="color:rgb(136,136,136)">Number
of entries: 0</span></div>
</div>
</div>
</div>
<br>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> Thanks,<br>
Ravi</div>
</blockquote>
<div><br>
</div>
<div>Another extra info (I don't know if
this can be the problem): Five days ago A
black out has suddenly shut down the
networks switch (also gluster network) of
node 03 and 04 ... But I don't know this
problem is in place after this black out </div>
<div><br>
</div>
<div>Thank you!</div>
</div>
<div><br>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail-m_7883186325552308780gmail_signature">Linux
User: 369739 <a href="http://counter.li.org"
target="_blank" moz-do-not-send="true">http://counter.li.org</a></div>
</div>
</div>
</blockquote>
<br>
</body>
</html>