<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">Why don’t you set LSI to passtrough mode and set one brick per HDD?<br><br><div>Regards,<div>Bartosz</div></div><div><br>Wiadomość napisana przez Brandon Bates <<a href="mailto:brandon@brandonbates.com">brandon@brandonbates.com</a>> w dniu 27.10.2017, o godz. 08:47:<br><br></div><blockquote type="cite"><div>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<title>Message</title>
<meta name="GENERATOR" content="MSHTML 11.00.10570.1001">
<div>
<div><font face="Arial"><font size="2"><span class="468513406-27102017">Hi gluster
users,</span></font></font></div>
<div><font face="Arial"><font size="2"><span class="468513406-27102017"><span class="468513406-27102017">I've spent several months trying to get any kind
of high performance out of gluster. <span class="468513406-27102017">The
current XFS/samba array is used for video editing and 300-400MB/s for at least 4
clients is minimum (currently a single windows client gets at least 700/700 for
a single client over samba, peaking to 950 at times using blackmagic speed
test). Gluster has been getting me as low as 200MB/s when the server
can do well over 1000MB/s. </span></span></span>I have really been
counting on <span class="468513406-27102017">/ touting
</span>Gluster <span class="468513406-27102017">as </span>being the way of
the future for us<span class="468513406-27102017">. However</span> I
can't justify cutting our performance to a mere 13% of non-gluster<span class="468513406-27102017"> speeds</span>. I've started to reach a give up
point and really need some help/hope otherwise I'll just have to migrate the
data from server 1 to server 2 just like I've been doing for the last
decade.<span class="468513406-27102017"> </span><span class="468513406-27102017">:(</span></font></font></div>
<div><font face="Arial"><font size="2"><span class="468513406-27102017"></span></font></font> </div>
<div><font face="Arial"><font size="2"><span class="468513406-27102017"><span class="468513406-27102017">If anyone can please help me understand where I might
be going wrong it would be absolutely
wonderful!</span></span></font></font></div>
<div><font face="Arial"><font size="2"><span class="468513406-27102017"></span></font></font> </div></div>
<div><font size="2" face="Arial">Server 1:<br>Single E5-1620 v2<br>Ubuntu
14.04<br>glusterfs 3.10.5<br>16GB Ram<br>24 drive array on LSI raid<br>Sustained
>1.5GB/s to XFS (77TB)</font></div>
<div> </div>
<div><font size="2" face="Arial">Server 2:<br>Single E5-2620 v3<br>Ubuntu
16.04<br>glusterfs 3.10.5<br>32GB Ram<br>36 drive array on LSI raid<br>Sustained
>2.5GB/s to XFS (164TB)</font></div>
<div> </div>
<div><font size="2" face="Arial">Speed tests are done with local with single thread
(dd) or 4 threads (iozone) using my standard 64k io size to 20G or 5G files (20G
for local drives, 5G for gluster) files.</font></div>
<div><font size="2" face="Arial"></font> </div>
<div><font size="2" face="Arial">Servers have Intel X520-DA2 dual port 10Gbit NICS
bonded together with 802.11ad LAG to a Quanta LB6-M switch. Iperf
throughput numbers are single stream >9000Mbit/s</font></div>
<div> </div>
<div><font size="2" face="Arial">Here is my current gluster
performance:</font></div>
<div> </div>
<div><font size="2" face="Arial">Single brick on server 1 (server 2 was
similar):<br>Fuse mount:<br>1000MB/s write<br>325MB/s read</font></div>
<div> </div>
<div><font size="2" face="Arial">Distributed only servers 1+2:<br>Fuse mount on
server 1:<br>900MB/s write iozone 4 streams<br>320MB/s read iozone 4
streams<br>single stream read 91MB/s @64K, 141MB/s @1M<br>simultaneous iozone 4
stream 5G files<br>Server 1: 1200MB/s write, 200MB/s read<br>Server 2: 950MB/s
write, 310MB/s read</font></div>
<div><font size="2" face="Arial"></font> </div>
<div><font size="2" face="Arial">I did some earlier single brick tests with samba
VFS and 3 workstations and got up to 750MB/s write and 800MB/s read aggregate
but that's still not good.</font></div>
<div> </div>
<div><font size="2" face="Arial">These are the only volume settings tweaks I have
made <span class="468513406-27102017">(</span>after much single box
testing<span class="468513406-27102017"> </span>to find what actually made a
difference<span class="468513406-27102017">)</span>:<br>performance.cache-size
1GB (Default 23MB)<br>performance.client-io-threads
on<br><a href="http://performance.io">performance.io</a>-thread-count
64<br>performance.read-ahead-page-count
16<br>performance.stat-prefetch on<br>server.event-threads 8
(default?)<br>client.event-threads 8</font></div>
<div> </div>
<div><font size="2" face="Arial"><span class="468513406-27102017">Any help given is
appreciated!</span></font></div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>Gluster-users mailing list</span><br><span><a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a></span><br><span><a href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a></span></div></blockquote></body></html>