[Gluster-users] Small files performance on SSD

[Amar Tumballi]

On Fri, Apr 28, 2017 at 3:44 PM, Szymon Miotk <szymon.miotk at gmail.com>
wrote:

> Dear Gluster community,
>
> I have problems with tuning Gluster for small files performance on SSD.
>
> My usage scenario is, as I've learned, worst possible scenario, but
> it's not up to me to change it:
> - small 1KB files
> - at least 20M of those
> - approx. 10 files/directory
> - mostly writes
> - average speed 1000 files/sec with peaks up to 10K files/sec.
>
> I'm doing something wrong, because I cannot get past performance metrics
> 4K files/sec for distributed volume (2 bricks)
> 2K files/sec for replicated volume (2 bricks).
> I've been experimenting with various XFS formatting and mounting
> options and with Gluster tuning (md-cache, lookup optimize, thread,
> writeback, tiering), but no luck.
>
> I've learned that it's not disk IO that is the bottleneck (direct
> tests on mounted XFS partition show waaay better results, like 100K
> files/sec).
>
> As I've learned from
> http://blog.gluster.org/2014/03/experiments-using-ssds-with-gluster/
> it's possible to get 24K files/sec performance (and that was three years
> ago).
>
>
How many clients are you running? Considering Gluster is a distributed
solution, the performance should be measured as aggregate of all the
clients.


> Test I'm using, run on one server (2 x Xeons, 256 GB RAM, 10GbE network):
> smallfile_cli.py --operation create --threads 32 --file-size 1 --files
> 15625 --top /mnt/testdir/test
>
> Setup:
> 2 servers with 2 Xeons each, 256GB RAM, 8 x 800GB SSD drives in RAID6,
> 10GbE network
> Ubuntu 14.04
> Gluster 3.7.3
>
> Do you have any hints where I should start investigating for the
> bottleneck?
>
>
Currently just the fuse mount may be the bottleneck, but I recommend
running multiple clients (from different machines) doing these operations
in parallel to get the best results out of Gluster.

-Amar

Best regards,
> Szymon
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>



-- 
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170428/9af0f914/attachment.html>