[Gluster-users] gluster CPU cycles at 100%

[Pranith Kumar Karampuri]

Could you please also capture fop profiling output using the following link please:
http://gluster.org/community/documentation/index.php/Gluster_3.2:_Running_Gluster_Volume_Profile_Command

That should give what fops are executing and what are their latencies which will be helpful.

Pranith

----- Original Message -----
> From: "Jay Vyas" <jayunit100 at gmail.com>
> To: "Paul Robert Marino" <prmarino1 at gmail.com>
> Cc: "Gluster-users at gluster.org" <gluster-users at gluster.org>
> Sent: Wednesday, January 29, 2014 6:07:14 AM
> Subject: Re: [Gluster-users] gluster CPU cycles at 100%
> 
> the workload : copying 21,000 1KB files.... small files, into gluster.
> The CPU is spiked the whole time and it takes a long time to do the copying.
> 
> There are some other moving parts, but Im pretty sure gluster shouldnt spike
> so high for just 21 MB of data, right?
> 
> there might be an error in my numbers above, so dont lose sleep - i still am
> trying to reproduce this "bug"... if thats what it is.
> 
> 
> On Tue, Jan 28, 2014 at 7:11 PM, Paul Robert Marino < prmarino1 at gmail.com >
> wrote:
> 
> 
> Well what do you mean by %100 CPU do you mean just 1 core or multiple also
> what's your IO wait percentage on your gluster servers.
> 
> Its not uncommon for gluster to spike up to use more than a whole core.
> 
> You may simply be exhausting the effective write cache on the gluster servers
> in which case unless you are striping your data across many raids you may
> just be hitting the limits of your hardware.
> 
> You may also have client(s) that are asymetricly connecting to only one or a
> small subset of servers due to a connectivity issue which would cause that
> one node or group of nodes to bear the brunt of replicating the entire
> workload via constant self heal.
> 
> To help you I would rather know more about your configuration before giving
> advice.
> 
> 
> 
> 
> -- Sent from my HP Pre3
> 
> 
> On Jan 28, 2014 14:08, Harshavardhana < harsha at harshavardhana.net > wrote:
> 
> You should take a 'gluster' statedump of the process - and open a
> bugzilla for analysis with logs.
> 
> On Tue, Jan 28, 2014 at 10:30 AM, Jay Vyas < jayunit100 at gmail.com > wrote:
> > Hi folks :
> > 
> > Im running mahout on top of gluster using the GlusterFileSystem hadoop
> > plugin.
> > 
> > It works well, but Im noticing that glusterfsd is at 100% CPU, and a copy
> > action seems to be stalling after a long workload.
> > 
> > Any ways to rescue the system? or should i restart it ?
> > 
> > --
> > Jay Vyas
> > http://jayunit100.blogspot.com
> > 
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://supercolony.gluster.org/mailman/listinfo/gluster-users
> 
> 
> 
> --
> Religious confuse piety with mere ritual, the virtuous confuse
> regulation with outcomes
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
> 
> 
> 
> --
> Jay Vyas
> http://jayunit100.blogspot.com
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users