<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 07/19/2017 08:02 PM, Sahina Bose
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CACjzOvcxmzxTxynS5fUs5HyEGN-AYF3P6Y34qLJ6MjktQh9djA@mail.gmail.com">
<div dir="ltr">[Adding gluster-users]<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Jul 19, 2017 at 2:52 PM, yayo
(j) <span dir="ltr"><<a href="mailto:jaganz@gmail.com"
target="_blank" moz-do-not-send="true">jaganz@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi all,
<div><br>
</div>
<div>We have an ovirt cluster hyperconverged with hosted
engine on 3 full replicated node . This cluster have 2
gluster volume:</div>
<div><br>
</div>
<div>- data: volume for the Data (Master) Domain (For vm)</div>
<div>- engine: volume fro the hosted_storage Domain (for
hosted engine)</div>
<div><br>
</div>
<div>We have this problem: "engine" gluster volume have
always unsynced elements and we cant' fix the problem,
on command line we have tried to use the "heal" command
but elements remain always unsynced .... </div>
<div><br>
</div>
<div>Below the heal command "status":</div>
<div><br>
</div>
<div>
<div class="m_6699510126617823271gmail-hljs
m_6699510126617823271gmail-objectivec"
style="display:block;overflow-x:auto;padding:0.5em;color:rgb(51,51,51);background:rgb(248,248,248);font-family:monospace">
<div>[root@node01 ~]<span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"># gluster volume
heal engine info</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Brick
node01:/gluster/engine/brick</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/.shard/8aa74564-6740-403e-<wbr>ad51-f56d9ca5d7a7.48</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/.shard/8aa74564-6740-403e-<wbr>ad51-f56d9ca5d7a7.64</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/.shard/8aa74564-6740-403e-<wbr>ad51-f56d9ca5d7a7.60</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/.shard/8aa74564-6740-403e-<wbr>ad51-f56d9ca5d7a7.2</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/.shard/8aa74564-6740-403e-<wbr>ad51-f56d9ca5d7a7.68</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/8f215dd2-8531-4a4f-b6ed-<wbr>ea789dd8821b/images/19d71267-<wbr>52a4-42a3-bb1e-e3145361c0c2/<wbr>7a215635-02f3-47db-80db-<wbr>8b689c6a8f01</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/8f215dd2-8531-4a4f-b6ed-<wbr>ea789dd8821b/images/88d41053-<wbr>a257-4272-9e2e-2f3de0743b81/<wbr>6573ed08-d3ed-4d12-9227-<wbr>2c95941e1ad6</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/.shard/8aa74564-6740-403e-<wbr>ad51-f56d9ca5d7a7.61</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/.shard/8aa74564-6740-403e-<wbr>ad51-f56d9ca5d7a7.1</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/8f215dd2-8531-4a4f-b6ed-<wbr>ea789dd8821b/dom_md/ids</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/.shard/8aa74564-6740-403e-<wbr>ad51-f56d9ca5d7a7.20</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/__DIRECT_IO_TEST__</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Status: Connected</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Number of entries:
12</span></div>
<div><br>
</div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Brick
node02:/gluster/engine/brick</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/8f215dd2-8531-4a4f-b6ed-<wbr>ea789dd8821b/images/19d71267-<wbr>52a4-42a3-bb1e-e3145361c0c2/<wbr>7a215635-02f3-47db-80db-<wbr>8b689c6a8f01</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"><span
class="m_6699510126617823271gmail-hljs-meta-string"
style="color:rgb(77,153,191)"><gfid:9a601373-bbaa-44d8-b396-<wbr>f0b9b12c026f></span></span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/8f215dd2-8531-4a4f-b6ed-<wbr>ea789dd8821b/dom_md/ids</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"><span
class="m_6699510126617823271gmail-hljs-meta-string"
style="color:rgb(77,153,191)"><gfid:1e309376-c62e-424f-9857-<wbr>f9a0c3a729bf></span></span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"><span
class="m_6699510126617823271gmail-hljs-meta-string"
style="color:rgb(77,153,191)"><gfid:e3565b50-1495-4e5b-ae88-<wbr>3bceca47b7d9></span></span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"><span
class="m_6699510126617823271gmail-hljs-meta-string"
style="color:rgb(77,153,191)"><gfid:4e33ac33-dddb-4e29-b4a3-<wbr>51770b81166a></span></span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/__DIRECT_IO_TEST__</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"><span
class="m_6699510126617823271gmail-hljs-meta-string"
style="color:rgb(77,153,191)"><gfid:67606789-1f34-4c15-86b8-<wbr>c0d05b07f187></span></span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"><span
class="m_6699510126617823271gmail-hljs-meta-string"
style="color:rgb(77,153,191)"><gfid:9ef88647-cfe6-4a35-a38c-<wbr>a5173c9e8fc0></span></span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">/8f215dd2-8531-4a4f-b6ed-<wbr>ea789dd8821b/images/88d41053-<wbr>a257-4272-9e2e-2f3de0743b81/<wbr>6573ed08-d3ed-4d12-9227-<wbr>2c95941e1ad6</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"><span
class="m_6699510126617823271gmail-hljs-meta-string"
style="color:rgb(77,153,191)"><gfid:9ad720b2-507d-4830-8294-<wbr>ec8adee6d384></span></span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)"><span
class="m_6699510126617823271gmail-hljs-meta-string"
style="color:rgb(77,153,191)"><gfid:d9853e5d-a2bf-4cee-8b39-<wbr>7781a98033cf></span></span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Status: Connected</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Number of entries:
12</span></div>
<div><br>
</div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Brick
node04:/gluster/engine/brick</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Status: Connected</span></div>
<div><span
class="m_6699510126617823271gmail-hljs-meta"
style="color:rgb(31,113,153)">Number of entries: 0</span></div>
<div><br>
</div>
</div>
</div>
<br>
<div> </div>
<div>running the "gluster volume heal engine" don't solve
the problem...<br>
</div>
<div><br>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
1. What does the glustershd.log say on all 3 nodes when you run the
command? Does it complain anything about these files?<br>
2. Are these 12 files also present in the 3rd data brick?<br>
3. Can you provide the output of `gluster volume info` for the this
volume?<br>
<blockquote type="cite"
cite="mid:CACjzOvcxmzxTxynS5fUs5HyEGN-AYF3P6Y34qLJ6MjktQh9djA@mail.gmail.com">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>Some extra info:</div>
<div><br>
</div>
<div>We have recently changed the gluster from: 2 (full
repliacated) + 1 arbiter to 3 full replicated cluster </div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
Just curious, how did you do this? `remove-brick` of arbiter brick
followed by an `add-brick` to increase to replica-3?<br>
<br>
Thanks,<br>
Ravi<br>
<blockquote type="cite"
cite="mid:CACjzOvcxmzxTxynS5fUs5HyEGN-AYF3P6Y34qLJ6MjktQh9djA@mail.gmail.com">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>but i don't know this is the problem...</div>
<div><br>
</div>
<div>The "data" volume is good and healty and have no
unsynced entry.</div>
<div><br>
</div>
<div>Ovirt refuse to put the node02 and node01 in
"maintenance mode" and complains about "unsynced
elements"<br>
</div>
<div><br>
</div>
<div>How can I fix this?</div>
<div>Thank you </div>
</div>
<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" moz-do-not-send="true">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users"
rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>