<div dir="ltr"><div><br></div>My take is that unless you have loads of data and are trying to optimize for cost/TB, HDDs are probably not the right choice. This is particularly true for random I/O workloads for which HDDs are really quite bad.<div><br></div><div>I'd recommend a recent gluster release, and some tuning because the default settings are not optimized for performance. Some options to consider:</div><div>client.event-threads</div><div>server.event-threads</div><div>cluster.choose-local</div><div>performance.client-io-threads</div><div><br></div><div>You can toggle the last two and see what works for you. You'd probably need to set event-threads to 4 or more. Ideally you'd tune some of the thread pools based on observed bottlenecks in collected stats. top (top -bHd 10 > top_threads.out.txt) is great for this. Using 6 small drives/bricks instead of 3 is also a good idea to reduce likelihood of rpc bottlenecks.</div><div><br></div><div>There has been an effort to improve gluster performance over fast SSDs. Hence the recommendation to try with a recent release. You can also check in on some of the issues being worked on:</div><div><a href="https://github.com/gluster/glusterfs/issues/412">https://github.com/gluster/glusterfs/issues/412</a><br></div><div><a href="https://github.com/gluster/glusterfs/issues/410">https://github.com/gluster/glusterfs/issues/410</a><br></div><div><br></div><div>-- Manoj</div><div><br></div><div><div><div class="gmail_extra"><div class="gmail_quote">On Sat, Mar 24, 2018 at 4:14 AM, Jayme <span dir="ltr"><<a href="mailto:jaymef@gmail.com" target="_blank">jaymef@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Do you feel that SSDs are worth the extra cost or am I better off using regular HDDs? I'm looking for the best performance I can get with glusterFS<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Mar 23, 2018 at 12:03 AM, Manoj Pillai <span dir="ltr"><<a href="mailto:mpillai@redhat.com" target="_blank">mpillai@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Thu, Mar 22, 2018 at 3:31 PM, Sahina Bose <span dir="ltr"><<a href="mailto:sabose@redhat.com" target="_blank">sabose@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 19, 2018 at 5:57 PM, Jayme <span dir="ltr"><<a href="mailto:jaymef@gmail.com" target="_blank">jaymef@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I'm spec'ing a new oVirt build using three Dell R720's w/ 256GB. I'm considering storage options. I don't have a requirement for high amounts of storage, I have a little over 1TB to store but want some overhead so I'm thinking 2TB of usable space would be sufficient. <div><br></div><div>I've been doing some research on Micron 1100 2TB ssd's and they seem to offer a lot of value for the money. I'm considering using smaller cheaper SSDs for boot drives and using one 2TB micron SSD in each host for a glusterFS replica 3 setup (on the fence about using an arbiter, I like the extra redundancy replicate 3 will give me).</div><div><br></div><div>My question is, would I see a performance hit using only one drive in each host with glusterFS or should I try to add more physical disks. Such as 6 1TB drives instead of 3 2TB drives? </div></div></blockquote></div></div></div></blockquote><div><br></div></span><div>It is possible. With SSDs the rpc layer can become the bottleneck with some workloads, especially if there are not enough connections out to the server side. We had experimented with a multi-connection model for this reason: <a href="https://review.gluster.org/#/c/19133/" target="_blank">https://review.gluste<wbr>r.org/#/c/19133/</a>.</div><span class="gmail-m_-3997065906060665138HOEnZb"><font color="#888888"><div><br></div><div>-- Manoj</div></font></span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>[Adding gluster-users for inputs here]</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Also one other question. I've read that gluster can only be done in groups of three. Meaning you need 3, 6, or 9 hosts. Is this true? If I had an operational replicate 3 glusterFS setup and wanted to add more capacity I would have to add 3 more hosts, or is it possible for me to add a 4th host in to the mix for extra processing power down the road?</div></div></blockquote><div><br></div><div>In oVirt, we support replica 3 or replica 3 with arbiter (where one of the 3 bricks is a low storage arbiter brick). To expand storage, you would need to add in multiples of 3 bricks. However if you only want to expand compute capacity in your HC environment, you can add a 4th node.</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Thanks!</div><div><br></div></div>
<br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br></blockquote></div><br></div></div>
<br></span><span>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><br></span></blockquote></div><br></div></div>
</blockquote></div><br></div>
</blockquote></div><br></div></div></div></div>