<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hello,</p>
<p>I am stuck with failure of my gluster 2x replica heal with
messages at glustershd.log as :</p>
<p><b><font size="-1">[2018-11-21 05:28:07.813003] E [MSGID: 114031]
[client-rpc-fops.c:1646:client3_3_entrylk_cbk] 0-gv1-client-0:
remote operation failed [Transport endpoint is not connected]</font></b></p>
<p>When the log hits here In either of the replica nodes; I can see
that my command: <b>watch glsuter volume heal <my volume>
statistics </b>returns no more progress and status unchanged
afterward. I am running glusterfs on top of zfs and it is
basically a storage for small read-only files. There was a thread
with Shyam Ranganathan and Reiner Keller in here where the core of
the problem was the storage going out of inods and no space left
error which obviously can not be my case as am on top of ZFS.
However, the similarity between us is that we were previously on
3.10 and upon various issues with that version we upgraded to 3.12
on Ubuntu 16.04 with kernel 4.4.0-116-generic.</p>
<p>Anybody faced issue as above?Can you advise what can be done as
it is for over a month with no effective self-heal process
completed...</p>
<p>Here is my gluster cluster info:</p>
<p><b><font size="-1">Volume Name: gv1<br>
Type: Replicate<br>
Volume ID: f1c955a1-7a92-4b1b-acb5-8b72b41aaace<br>
Status: Started<br>
Snapshot Count: 0<br>
Number of Bricks: 1 x 2 = 2<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: IMG-01:/images/storage/brick1<br>
Brick2: IMG-02:/images/storage/brick1<br>
Options Reconfigured:<br>
cluster.self-heal-daemon: enable<br>
cluster.eager-lock: off<br>
client.event-threads: 4<br>
performance.cache-max-file-size: 8<br>
features.scrub: Inactive<br>
features.bitrot: off<br>
network.inode-lru-limit: 50000<br>
nfs.disable: true<br>
performance.readdir-ahead: on<br>
server.statedump-path: /tmp<br>
cluster.background-self-heal-count: 32<br>
performance.md-cache-timeout: 30<br>
cluster.readdir-optimize: on<br>
cluster.shd-max-threads: 4<br>
cluster.lookup-optimize: on<br>
features.cache-invalidation: on<br>
features.cache-invalidation-timeout: 600<br>
performance.stat-prefetch: on<br>
performance.cache-invalidation: on<br>
server.event-threads: 4</font></b><br>
<br>
</p>
<p>Thank you<br>
</p>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Hamid Safe
<a class="moz-txt-link-abbreviated" href="http://www.devopt.net">www.devopt.net</a>
+989361491768
</pre>
</body>
</html>