[Gluster-users] Gluster 3.12.14: slow ls -l command + "Another transaction is in progress. Please try again after sometime" message

[Mauro Tridici]

Dear All,

during last two days, our distributed dispersed gluster volume is showing some strange behaviors:

1) "df -h" and "ls -l" linux commands are very slow
2) “TAB” key usage causes the hang of the shell
3) nagios monitoring system is showing some alerts related to QUOTA and VOLUME UTILIZATION similar to the following one:

Notification Type: PROBLEM

Service: Volume Quota - tier2
Host: tier2-gluster
Address: tier2-gluster
State: WARNING

Date/Time: Wed Oct 24 16:27:13 CEST 2018

Additional Info:

QUOTA: Quota status could not be determined.

4) on some gluster servers, “df -h” command shows a new mount “/run/gluster/tier2_quota_list”
5) gluster vol status returns this message: “Another transaction is in progress. Please try again after sometime”.

I just restarted the “glusterd” service on each server, one by one.
Also, I executed “umount -l /run/gluster/tier2_quota_list” on each server.
Now, everything seems to be ok, but I would like to know if there is a way to avoid this kind of problem.

Last question: do you think that it is a good practice executing “service glusterd restart” command on each server when the problem appears? 
In attachment you can find the glusterd.log file

Thank you in advance,
Mauro




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181024/fc99694c/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: glusterd.log
Type: application/octet-stream
Size: 499572 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181024/fc99694c/attachment-0001.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181024/fc99694c/attachment-0003.html>