<div dir="ltr">Hi,<div><br></div><div>The email is talking about many issues. Let me ask a few questions to get a whole picture.</div><div>1. are the peers are in the connected state now? or they still in the rejected state?</div><div>2. What led you to see "locking failed" messages? We would like to if there is a reproducer and fix the issue if any.</div><div>3. Another transaction in progress message appears when there is already a operation going on. Are you seeing this when there is no such transaction going on?</div><div>4. When did you hit the timedouts? Did you tried to look at the pstack output of glusterd process? If so, please share the pstack output.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 21, 2020 at 7:08 PM <<a href="mailto:nico@furyweb.fr">nico@furyweb.fr</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi all.<br>
<br>
We're using 3 nodes Gluster 7.3 (2 + 1 arbiter), yesterday node 2 was rejected from cluster and I applied following steps to fix : <a href="https://staged-gluster-docs.readthedocs.io/en/release3.7.0beta1/Administrator%20Guide/Resolving%20Peer%20Rejected/" rel="noreferrer" target="_blank">https://staged-gluster-docs.readthedocs.io/en/release3.7.0beta1/Administrator%20Guide/Resolving%20Peer%20Rejected/</a><br>
I also saw <a href="https://docs.gluster.org/en/latest/Troubleshooting/troubleshooting-glusterd/" rel="noreferrer" target="_blank">https://docs.gluster.org/en/latest/Troubleshooting/troubleshooting-glusterd/</a> but solution isn't compatible as cluster.max-op-version doesn't exist and all op-version are the same on all 3 nodes.<br>
<br>
After renewing SSL certs and several restart all volumes came back online but glusterd log file on all 3 nodes is filled with nothing else than following 3 lines :<br>
<br>
[2020-04-21 13:05:19.478913] I [socket.c:4347:ssl_setup_connection_params] 0-socket.management: SSL support on the I/O path is ENABLED<br>
[2020-04-21 13:05:19.478972] I [socket.c:4350:ssl_setup_connection_params] 0-socket.management: SSL support for glusterd is ENABLED<br>
[2020-04-21 13:05:19.478986] I [socket.c:4360:ssl_setup_connection_params] 0-socket.management: using certificate depth 1<br>
<br>
Moreover, I have "Locking failed", "Another transaction is in progress" and "Error : Request timed out" on gluster volume status volxxx command.<br>
All SSL certs on clients have also been renewed and all volumes were remounted. All 3 nodes were alternatively restarted (glusterd) and rebooted.<br>
<br>
The cluster is not in production environment but there's about ~250 clients for ~75 volumes, I don't know how to troubleshoot and fix this problem, if anyone has an idea.<br>
________<br>
<br>
<br>
<br>
Community Meeting Calendar:<br>
<br>
Schedule -<br>
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div>Thanks,<br></div>Sanju<br></div></div>