<div dir="ltr"><div>It worries me how many threads talk about low performance. I'm about to build out a replica 3 setup and run Ovirt with a bunch of Windows VMs. </div><div><br></div><div>Are the issues Tony is experiencing "normal" for Gluster? Does anyone here have a system with windows VMs and have good performance? </div><div><br></div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div><div dir="ltr" style="font-size:12.8px"><font size="4" face="arial narrow, sans-serif"><b>Vincent Royer</b></font></div><div dir="ltr"><div style="font-size:12.8px"><font size="4" face="arial narrow, sans-serif"><b>778-825-1057</b></font></div><div style="font-size:12.8px"><font size="4" face="arial narrow, sans-serif"><b><br></b></font></div><div style="font-size:12.8px"><a href="http://www.epicenergy.ca/" target="_blank"><img src="https://lh3.googleusercontent.com/ezD9JxeN67lTscselHjW4ZEqvQRj_i_T_Ky_s__GkR0AtanG1nK52CD1Rdhmyzbk7XRPf-Rhhnr2-SLPmhtd0J8xVMp0xOiR20ebQVb6jU2xwJPKN4lePRq5G2p93ybqA4SnqjCWCvEsNlK37a9FtStdFcTJyIoS2aHKiL69L3bZg1g7mEAMr7IHXVYvHgq3MX9rNaqIQx9T3uPuIHEDIfxrgpCrsBoEamgaRHj5hcpEobnogqCqpa1sRcKy927TSeR5BbVLM1-wn0h-4LvE8CMqj30jpQULxZ9uXEg_S6sR3ZeZjtkkRvgK4l_wiivWlyV_SjCDQ5QREG3jkuCjXC04i1NNmziGZZNWYsaohE6khrN4JGVtvaSOiS2woYJNz3Gt0EucxJp3eUBYspzTl9iZia7bLvFEGIODvmfjRVZJmnCrFucJxGkAtWn_EUIbxJXS5NWAFnvBoixQP563Fbu7dFKL77lDcb4PHwrnJVjqcgU0OTeiMBlMvEt97YgMpMvdi7rx9L4DeEbrIGUovgx4QLfRRoF_9u2mxFu8FHupCxfocSyB3vs8gqOLvVdx42qwiLN0sGLAg8K3G_vFwVNddXQ_yb3cGeCEAOTnk7jfiP4O4jsPIW2RRbnKKL-usL11Tp7gkxbgRouLOoyqpbisjA4vRWDeylj1yDGyBg=w400-h38-no" width="200" height="18" class="hoverZoomLink"><br></a></div><div><b><font face="arial narrow, sans-serif" size="2">SUSTAINABLE MOBILE <font color="#6aa84f">ENERGY </font>SOLUTIONS</font></b></div></div></div><div><br></div><div><br><div><div><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr">On Wed, May 2, 2018 at 7:52 AM Tony Hoyle <<a href="mailto:tony@hoyle.me.uk">tony@hoyle.me.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 01/05/2018 02:27, Thing wrote:<br>
> Hi,<br>
> <br>
> So is the KVM or Vmware as the host(s)? I basically have the same setup<br>
> ie 3 x 1TB "raid1" nodes and VMs, but 1gb networking. I do notice with<br>
> vmware using NFS disk was pretty slow (40% of a single disk) but this<br>
> was over 1gb networking which was clearly saturating. Hence I am moving<br>
> to KVM to use glusterfs hoping for better performance and bonding, it<br>
> will be interesting to see which host type runs faster.<br>
<br>
1gb will always be the bottleneck in that situation - that's going too<br>
max out at the speed of a single disk or lower. You need at minimum to<br>
bond interfaces and preferably go to 10gb to do that.<br>
<br>
Our NFS actually ends up faster than local disk because the read speed<br>
of the raid is faster than the read speed of the local disk.<br>
<br>
> Which operating system is gluster on? <br>
<br>
Debian Linux. Supermicro motherboards, 24 core i7 with 128GB of RAM on<br>
the VM hosts.<br>
<br>
> Did you do iperf between all nodes?<br>
<br>
Yes, around 9.7Gb/s<br>
<br>
It doesn't appear to be raw read speed but iowait. Under nfs load with<br>
multiple VMs I get an iowait of around 0.3%. Under gluster, never less<br>
than 10% and glusterfsd is often the top of the CPU usage. This causes<br>
a load average of ~12 compared to 3 over NFS, and absolutely kills VMs<br>
esp. Windows ones - one machine I set booting and it was still booting<br>
30 minutes later!<br>
<br>
Tony<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailman/listinfo/gluster-users</a></blockquote></div>