[Gluster-users] [External] Re: Samba+Gluster: Performance measurements for small files

Tue Jan 22 08:23:03 UTC 2019

Hi David,

i haven't tested samba but glusterfs fuse, i have posted the results few
months ago, tests conducted using gluster 4.1.5:

*Options Reconfigured:*
client.event-threads 3
performance.cache-size 8GB
performance.io-thread-count 24
network.inode-lru-limit 1048576
performance.parallel-readdir on
performance.cache-invalidation on
performance.md-cache-timeout 600
features.cache-invalidation on
features.cache-invalidation-timeout 600
performance.client-io-threads on

nr of clients 6
Network 10Gb
Clients Mem 128GB
Clients Cores 22
Centos 7.5.1804
Kernel 3.10.0-862.14.4.el7.x86_64

nr of servers/bricks per volume 3
Network 100Gb
*node to node is 100Gb, cleints 10Gb
Server Mem 377GB
Server Cores 56 *Intel 5120 CPU
Storage 4x8TB NVME
Centos 7.5.1804
Kernel 3.10.0-862.14.4.el7.x86_64

This for example are FOPS with 128K IO size (cnsidered sweet spot for
glusterfs according to documentation). In Blue 8threads per client and red
4threads for client
[image: image.png]
Below 4K
[image: image.png]
and 1MB
[image: image.png]

On Tue, Jan 22, 2019 at 9:09 AM David Spisla <spisla80 at gmail.com> wrote:

> Hello Amar,
> thank you for the advice. We already use nl-cache option and a bunch of
> other settings. At the moment we try the samba-vfs-glusterfs plugin to
> access a gluster volume via samba. The performance increase now.
> Additionally we are looking for some performance measurements to compare
> with. Maybe someone in the community also does performance tests. Does
> Redhat has some official reference measurement?
>
> Regards
> David Spisla
>
> Am Di., 22. Jan. 2019 um 07:14 Uhr schrieb Amar Tumballi Suryanarayan <
> atumball at redhat.com>:
>
>> For Samba usecase, please make sure you have nl-cache (ie,
>> 'negative-lookup cache') enabled. We have seen some improvements from this
>> value.
>>
>> -Amar
>>
>> On Tue, Dec 18, 2018 at 8:23 PM David Spisla <spisla80 at gmail.com> wrote:
>>
>>> Dear Gluster Community,
>>>
>>> it is a known fact that Samba+Gluster has a bad smallfile performance.
>>> We now have some test measurements created by this setup: 2-Node-Cluster on
>>> real hardware with Replica-2 Volume (just one subvolume), Gluster v.4.1.6,
>>> Samba v4.7. Samba writes to Gluster via FUSE. Files created by fio. We used
>>> a Windows System as Client which is in the same network like the servers.
>>>
>>> The measurements are as follows. In each test case 400 files were
>>> written:
>>>
>>>                        64KiB_x_400 files            1MiB_x_400 files
>>>         10MiB_x_400 files
>>> 1 Thread          0,77 MiB/s                       8,05
>>> MiB/s                    72,67 MiB/s
>>> 4 Threads        0,86 MiB/s                       8,92 MiB/s
>>>         90,38 MiB/s
>>> 8 Threads        0,87 MiB/s                       8,92
>>> MiB/s                     94,75 MiB/s
>>>
>>> Does anyone have measurements that are in a similar range or are significantly different?
>>> We do not know which values can still be considered "normal" and which are not.
>>> We also know that there are options to improve performance. But first of all we are interested
>>> in whether there are reference values.
>>> Regards
>>> David Spisla
>>>
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>
>>
>>
>> --
>> Amar Tumballi (amarts)
>>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users

-- 
Davide Obbi
Senior System Administrator

Booking.com B.V.
Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
Direct +31207031558
[image: Booking.com] <https://www.booking.com/>
Empowering people to experience the world since 1996
43 languages, 214+ offices worldwide, 141,000+ global destinations, 29
million reported listings
Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190122/a564f10b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 6494 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190122/a564f10b/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 6574 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190122/a564f10b/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 6299 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190122/a564f10b/attachment-0005.png>