[Gluster-users] glusterfs and glusterfsd process utilization extremely high

[Krutika Dhananjay]

----- Original Message -----

> From: "Kyle Harris" <kyle.harris98 at gmail.com>
> To: gluster-users at gluster.org
> Sent: Thursday, December 18, 2014 4:47:35 AM
> Subject: [Gluster-users] glusterfs and glusterfsd process utilization
> extremely high

> This is an extenuation of a problem that I posted about last month that I am
> still experiencing. The original post with more detail can be found at
> http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019587.html
> . To sum up my problem, I have a freshly created 3 node replicated cluster.
> It contains roughly 135 GB of files, many of which are small. It is home to
> several web sites hosted with Apache. I am using Gluster version 3.6.1-1
> installed from RPMs mounted from the server via the Fuse client (I have
> tried NFS but it makes no difference).

> When I posted last about the problem of extreme processor utilization, the
> solution I was given by Parnith was to utilize another file system other
> than EXT4 and to turn off cluster.entry-self-heal. I am now using XFS and
> cluster.entry-self-heal is turned off and I even turned off
> cluster.self-heal-daemon but it made absolutely no difference. All is fine
> during the entire time the cluster is loaded via rsync however the minute I
> point Apache traffic at the sites hosted on the cluster, glusterfs and
> glusterfsd begin to climb to levels so high that in a matter of minutes it
> is not even possible to log on to the system. No modification have been made
> to any of the other Gluster settings.

> Any additional help resolving this matter would be greatly appreciated.

Hello, 

First of all, do the logs suggest anything useful? 

Could you perform the following steps while the I/O is going on (this is assuming the nodes are not thrashed to the extent that it is impossible to execute these commands): 

1) On the shell, on one of the nodes in the cluster, execute `gluster volume profile <volname> start` 
Wait for a minute or two. And then execute `gluster volume profile <volname> info` and collect its output. 
Wait for another minute or so. And execute `gluster volume profile <volname> info` and collect its output too, and share them? 
You can stop the profiling once you are done using `gluster volume profile <volname> stop`. 
2) Assuming it is the brick processes (glusterfsd) that are showing high CPU utilisation, is it possible to get the core of the processes when this is happening? 

-Krutika 

> --
> Regards,

> Kyle

> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20141217/37e04684/attachment.html>