[Gluster-users] Optimizing Gluster (gfapi) for high IOPS

[Andrew Lau]

On Sun, Mar 23, 2014 at 6:10 AM, Josh Boon <gluster at joshboon.com> wrote:

> Thanks for those options. My machines tend to be self-healing rather
> frequently.  Doing a gluster volume heal VMARRAY info the file list cycles
> through most of my high IOPS machines
>
> Also what's the best way to apply those options with out bricking the
> running VM's?  I just made a rough stab and took the cluster down.
>
>
The CPU problem thing sounds a lot like what I ran into with my ovirt on
gluster deployment (same boxes). What I did to solve that was use cgroups
to limit the CPU usage glusterd and glusterfsd is allowed to use. [1]

I'm not completely sure if libgfapi uses the glusterd process to access the
storage, could someone else comment? However, I know by limiting glusterfsd
we can slow down the replication process by limiting the CPU it sees, thus
not bringing the entire system to a halt.

[1]
http://www.andrewklau.com/controlling-glusterfsd-cpu-outbreaks-with-cgroups/



 ----- Original Message -----
> From: "Vijay Bellur" <vbellur at redhat.com>
> To: "Josh Boon" <gluster at joshboon.com>, "Nick Majeran" <nmajeran at gmail.com
> >
> Cc: "Gluster-users at gluster.org List" <gluster-users at gluster.org>
> Sent: Saturday, March 22, 2014 1:36:09 PM
> Subject: Re: [Gluster-users] Optimizing Gluster (gfapi) for high IOPS
>
> On 03/21/2014 09:50 PM, Josh Boon wrote:
> > Hardware RAID 5 on SSD's using LVM formatted with XFS default options
> > mounted with noatime
> >
> > Also I don't a lot of history for this current troubled machine but the
> > sysctl additions don't appear to have made a significant difference
>
> Performance tunables in [1] are normally recommended for qemu -
> libgfapi. The last two options are related to quorum and the remaining
> tunables are related to performance. It might be worth a check to see if
> these options help provide better performance.
>
> Do you happen to know if self-healing was in progress when the machines
> stall?
>
> -Vijay
>
> [1]
> https://github.com/gluster/glusterfs/blob/master/extras/group-virt.example
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140331/923d4e72/attachment.html>