[Gluster-users] Gluster linear scale-out performance

[Qing Wang]

Hi Yaniv,

Thanks for the quick response. I forget to mention I am testing the writing
performance, not reading. In this case, would the client cache hit rate
still be a big issue?

I'll use fio to run my test once again, thanks for the suggestion.

Thanks,
Qing

On Tue, Jul 21, 2020 at 2:38 PM Yaniv Kaul <ykaul at redhat.com> wrote:

>
>
> On Tue, 21 Jul 2020, 21:30 Qing Wang <qw at g.clemson.edu> wrote:
>
>> Hi,
>>
>> I am trying to test Gluster linear scale-out performance by adding more
>> storage server/bricks, and measure the storage I/O performance. To vary the
>> storage server number, I create several "stripe" volumes that contain 2
>> brick servers, 3 brick servers, 4 brick servers, and so on. On gluster
>> client side, I used "dd if=/dev/zero of=/mnt/glusterfs/dns_test_data_26g
>> bs=1M count=26000" to create 26G data (or larger size), and those data will
>> be distributed to the corresponding gluster servers (each has gluster brick
>> on it) and "dd" returns the final I/O throughput. The Internet is 40G
>> infiniband, although I didn't do any specific configurations to use
>> advanced features.
>>
>
> Your dd command is inaccurate, as it'll hit the client cache. It is also
> single threaded. I suggest switching to fio.
> Y.
>
>
>> What confuses me is that the storage I/O seems not to relate to the
>> number of storage nodes, but Gluster documents said it should be linear
>> scaling. For example, when "write-behind" is on, and when Infiniband "jumbo
>> frame" (connected mode) is on, I can get ~800 MB/sec reported by "dd", no
>> matter I have 2 brick servers or 8 brick servers -- for 2 server case, each
>> server can have ~400 MB/sec; for 4 server case, each server can have
>> ~200MB/sec. That said, each server I/O does aggregate to the final storage
>> I/O (800 MB/sec), but this is not "linear scale-out".
>>
>> Can somebody help me to understand why this is the case? I certainly can
>> have some misunderstanding/misconfiguration here. Please correct me if I
>> do, thanks!
>>
>> Best,
>> Qing
>> ________
>>
>>
>>
>> Community Meeting Calendar:
>>
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://bluejeans.com/441850968
>>
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200721/a535ccd0/attachment.html>