<div dir="ltr"><div>Hi,</div><div><br></div><div dir="ltr">On Mon, Apr 8, 2019 at 8:50 AM PSC <<a href="mailto:1173701037@qq.com">1173701037@qq.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US">Hi,
I am a storage software coder who is interested in Gluster. I am
trying to improve the read/write performance of it. </span></font></font>
</p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US">I
noticed that gluster is using Vandermonde matrix in erasure code
encoding and decoding process. However, it is quite complicate to
generate inverse matrix of a Vandermonde matrix, which is necessary
for decode. The cost is O(n<font face="Liberation Serif, serif">³</font>).</span></font></font></p></blockquote><div><br></div>That's not true, actually. A Vandermonde matrix can be inverted in O(n^2), as the code currently does (look at ec_method_matrix_inverse() in ec-method.c). Additionally, current code does caching of inverted matrices, so in normal circumstances there shouldn't be many inverse computations. Only when something changes (a brick dies or comes online), a new inverted matrix could be needed.<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US">
</span></font></font>
</p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US">Use
a Cauchy matrix, can greatly cut down the cost of the process to find
an inverse matrix. Which is O(n<font face="Liberation Serif, serif">²).</font></span></font></font></p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">I
use intel storage accelerate library to replace the original ec
encode/decode part of gluster. And it reduce the encode and decode
time to about 50% </font><font face="Liberation Serif, serif">of the</font><font face="Liberation Serif, serif">
original one.</font></span></font></font></p></blockquote><div><br></div><div>How do you test that ? I also did some tests long ago and I didn't observe that difference.</div><div><br></div><div>Doing a raw test of encoding/decoding performance of the current code using Intel AVX2 extensions, it's able to process 7.6 GiB/s on a single core of an Intel Xeon Silver 4114 when L1 cache is used. Without relying on internal cache, it performs at 3.9 GiB/s. Does ISA-L provide better performance for a matrix of the same size (4+2 non-systematic matrix) ?</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">However,
when I test the whole system. The read/write performance is almost
the same as the original </font><font face="Liberation Serif, serif">gluster.</font></span></font></font></p></blockquote><div><br></div><div>Yes, there are many more things involved in the read and write operations in gluster. For the particular case of EC, having to deal with many bricks simultaneously (6 in this case) means that it's very sensitive to network latency and communications delays, and this is probably one of the biggest contributors. There some other small latencies added by other xlators.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">
</font></span></font></font>
</p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">I
test it on three machines as servers. Each one had two bricks, both
of them are SSD. So the total amount of bricks is 6. Use two of them
as coding bricks. That is a 4+2 disperse volume configure.</font></span></font></font></p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">The
capability of network card is 10000Mbps. Theoretically it can support
read and write with the speed faster than 1000MB/s.</font></span></font></font></p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">The
actually performance of read is about </font><font face="Liberation Serif, serif">492</font><font face="Liberation Serif, serif">MB/s.</font></span></font></font></p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">The
actually performance of write is about 3</font><font face="Liberation Serif, serif">36</font><font face="Liberation Serif, serif">MB/s.</font></span></font></font></p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">While
the original one read at 4</font><font face="Liberation Serif, serif">61</font><font face="Liberation Serif, serif">MB/s,
</font><font face="Liberation Serif, serif">write at 322MB/s</font></span></font></font></p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">Is
there someone who can give me some advice about how to improve its
performance? Which part is the critical defect on its performance if
it’s not the ec translator? </font></span></font></font>
</p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">I
did a time count on translators. It show me EC translator just take
7% in the whole read\write process. Even though I knew that some
translators are run </font><font face="Liberation Serif, serif">asynchronous</font><font face="Liberation Serif, serif">,
so the real percentage can be some how lager than that. </font></span></font></font>
</p>
<p style="margin-bottom:0cm;line-height:100%"><br>
</p>
<p style="margin-bottom:0cm;line-height:100%"><font face="Liberation Serif, serif"><font style="font-size:12pt" size="3"><span lang="en-US"><font face="Liberation Serif, serif">Sincerely
thank you for your patient to read my question!</font></span></font></font></p>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-devel</a></blockquote></div></div>