[Gluster-users] How to check running transactions in gluster?

[Atin Mukherjee]

On Mon, Nov 26, 2018 at 8:21 AM Atin Mukherjee <amukherj at redhat.com> wrote:

>
>
> On Sun, Nov 25, 2018 at 8:40 PM Jeevan Patnaik <g1patnaik at gmail.com>
> wrote:
>
>> Hi,
>>
>> I am getting output Another transaction is in progress with few gluster
>> volume commands including stop command. And with gluster volume status
>> command, it's just hung and fails with timeout error.
>>
>
> This is primarily because of not allowing glusterd to complete it's
> handshake with others when concurrent restart of glusterd services are done
> (as I could understand from your previous email in the list). With GlusterD
> (read as GD1) this is a current challenge w.r.t it's design where due to
> its N X N handshaking mechanism at the restart sequence to bring all the
> configuration data into inconsistent what we've seen is the overall
> recovery time of the cluster can take very long if N is on the higher side
> (in your case N = 72 which is certainly high) and hence the recommendation
> is not to restart the glusterd services concurrently and wait for the
> handshaking to complete.
>

Forgot to mention that GlusterD2 ( https://github.com/gluster/glusterd2)
which is in development phase addresses this design problem.


>
>> So, I want to find out which transaction is hung and how to know this? I
>> ran volume statedump command, but didn't wait till it's completed to check
>> if it's hung or giving any resut, as it is also taking time.
>>
>
> kill -SIGUSR1 $(pidof glusterd) should get you a glusterd statedump file
> in /var/run/gluster which can point to a backtrace dump at the bottom to
> understand which transaction is currently in progress. In case this
> transaction is queued up for more than 180 seconds (which is not usual) the
> unlock timer kicks out such locks.
>
>
>> Please help me with this.. I'm struggling with these gluster timeout
>> errors :(
>>
>> Besides, I have also tuned
>> transport.listen-backlog gluster to 200 and following kernel parameters
>> to avoid syn overflow rejects:
>> net.core.somaxconn = 1024
>> net.ipv4.tcp_max_syn_backlog = 20480
>>
>> Regards,
>> Jeevan.
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181126/4abba43c/attachment.html>