<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Vijay,<div><br></div><div>The Volume has been created using heketi-cli blockvolume create command. The block config is the config applied by heketi out of the box and in my case ended up to be:</div><div>- 3 nodes each with 1 brick</div><div>- the brick is carved from a VG with a single PV</div><div>- the PV consists of a 1.2TB SSD, not partitioned and no HW RAID behind</div><div>- the volume does not have any custom setting aside what configured in /etc/glusterfs/group-gluster-block by default</div><div><font face="monospace, monospace">performance.quick-read=off</font></div><div><font face="monospace, monospace">performance.read-ahead=off</font></div><div><font face="monospace, monospace">performance.io-cache=off</font></div><div><font face="monospace, monospace">performance.stat-prefetch=off</font></div><div><font face="monospace, monospace">performance.open-behind=off</font></div><div><font face="monospace, monospace">performance.readdir-ahead=off</font></div><div><font face="monospace, monospace">performance.strict-o-direct=on</font></div><div><font face="monospace, monospace">network.remote-dio=disable</font></div><div><font face="monospace, monospace">cluster.eager-lock=enable</font></div><div><font face="monospace, monospace">cluster.quorum-type=auto</font></div><div><font face="monospace, monospace">cluster.data-self-heal-algorithm=full</font></div><div><font face="monospace, monospace">cluster.locking-scheme=granular</font></div><div><font face="monospace, monospace">cluster.shd-max-threads=8</font></div><div><font face="monospace, monospace">cluster.shd-wait-qlength=10000</font></div><div><font face="monospace, monospace">features.shard=on</font></div><div><font face="monospace, monospace">features.shard-block-size=64MB</font></div><div><font face="monospace, monospace">user.cifs=off</font></div><div><font face="monospace, monospace">server.allow-insecure=on</font></div><div><font face="monospace, monospace">cluster.choose-local=off</font></div><div><br></div><div>Kernel: 3.10.0-862.11.6.el7.x86_64<br></div><div>OS: Centos 7.5.1804</div><div>tcmu-runner: 0.2rc4.el7</div><div><br></div><div>Each node has 32 cores and 128GB RAM and 10Gb connection.</div><div><br></div><div>What i am trying to understand is what should be performance expectations with gluster-block since i couldnt find many benchmarks online.</div><div><br></div><div>Regards</div><div>Davide</div><div><br></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Nov 9, 2018 at 7:07 AM Vijay Bellur <<a href="mailto:vbellur@redhat.com">vbellur@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Davide,<div><br></div><div>Can you please share the block hosting volume configuration?</div><div><br></div><div>Also, more details about the kernel and tcmu-runner versions could help in understanding the problem better.</div><div><br></div><div>Thanks,</div><div>Vijay</div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Nov 6, 2018 at 6:16 AM Davide Obbi <<a href="mailto:davide.obbi@booking.com" target="_blank">davide.obbi@booking.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi,<div><br></div><div>i am testing gluster-block and i am wondering if someone has used it and have some feedback regarding its performance.. just to set some expectations... for example:</div><div>- i have deployed a block volume using heketi on a 3 nodes gluster4.1 cluster. it's a replica3 volume.</div><div>- i have mounted via iscsi using multipath config suggested, created vg/lv and put xfs on it</div><div>- all done without touching any volume setting or customizing xfs parameters etc..</div><div>- all baremetal running on 10Gb, gluster has a single block device, SSD in use by heketi</div><div><br></div><div>so i tried a dd and i get a 4.7 MB/s? </div><div>- on the gluster nodes i have in write ~200iops, ~15MB/s, 75% util steady and spiky await time up to 100ms alternating between the servers. CPUs are mostly idle but there is some waiting...</div><div>- Glusterd and fsd utilization is below 1%</div><div><br></div><div>The thing is that a gluster fuse mount on same platform does not have this slowness so there must be something wrong with my understanding of gluster-block? </div><div> </div><div><br></div><div><br></div><div><br></div><div><br></div></div></div>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a></blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><span style="display:block;font-size:11.0pt;font-family:Century Gothic;color:#003580"><div style="color:rgb(0,53,128);font-family:Arial,Helvetica,sans-serif;font-weight:bold;font-size:15px"><div>Davide Obbi</div><div style="font-weight:normal;font-size:13px;color:rgb(0,174,239)">System Administrator<br><br></div><div style="font-weight:normal;font-size:13px;color:rgb(102,102,102)">Booking.com B.V.<br>Vijzelstraat 66-80 Amsterdam 1017HL Netherlands</div><div style="font-weight:normal;font-size:13px;color:rgb(102,102,102)"><span style="color:rgb(0,174,239)">Direct </span>+31207031558<br></div><div style="font-weight:normal;font-size:13px;color:rgb(102,102,102)"><div style="font-weight:bold;font-size:16px;color:rgb(0,53,128)"><a href="https://www.booking.com/" style="color:rgb(0,127,255);background-image:initial;background-position:initial;background-repeat:initial" target="_blank"><img src="https://bstatic.com/static/img/siglogo.jpg" alt="Booking.com" title="Booking.com"></a></div><div><span style="font-size:11px">Empowering People to experience the world since 1996</span></div><div><span style="font-size:11px">43 languages, 214+ offices worldwide, 141,000+ global destinations, 29 million reported listings <br></span><span style="font-size:11px">Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)</span></div></div></div></span></div>