[Gluster-users] FW: Performance with Gluster+Fuse is 60x slower then Gluster+NFS ?
Dan Mons
dmons at cuttingedge.com.au
Thu Feb 18 00:14:59 UTC 2016
Without knowing the details, I'm putting my money on cache.
Choosing how to mount Gluster is workload dependent. If you're doing
a lot of small files with single threaded writes, I suggest NFS. Your
client's nfscache will dramatically improve performance from the
end-user's point of view.
If you're doing heavy multi-threaded reads and writes, and you have
very good bandwidth from your client (e.g.: 10GbE) FUSE+GlusterFS is
better, as it allows your client to talk to all Gluster nodes.
If you are using FUSE+GlusterFS, on the gluster nodes themselves,
experiment with the "performance.write-behind-window-size" and
"performance.cache-size" options. Note that these will affect the
cache used by the clients, so don't set them so high as to exhaust the
RAM of any client connecting (or, for low-memory clients, use NFS
instead).
Gluster ships with conservative defaults for cache, which is a good
thing. It's up to the user to tweak for their optimal needs.
There's no right or wrong answer here. Experiment with NFS and
various cache allocations with FUSE+GlusterFS, and see how you go.
And again, consider your workloads, and whether or not they're taking
full advantage of the FUSE client's ability to deal with highly
parallel workloads.
-Dan
----------------
Dan Mons - VFX Sysadmin
Cutting Edge
http://cuttingedge.com.au
On 18 February 2016 at 08:56, Stefan Jakobs <stefan at localside.net> wrote:
> Van Renterghem Stijn:
>> Interval2
>> Block Size: 1b+ 16b+ 32b+
>> No. of Reads: 0 0
>> 0 No. of Writes: 342 25
>> 575
>>
>> Block Size: 64b+ 128b+
>> 256b+ No. of Reads: 0 0
>> 0 No. of Writes: 143 898
>> 118
>>
>> Block Size: 512b+ 1024b+
>> 2048b+ No. of Reads: 1 4
>> 11 No. of Writes: 82 0
>> 0
>>
>> Block Size: 4096b+ 8192b+
>> 16384b+ No. of Reads: 11 31
>> 39 No. of Writes: 0 0
>> 0
>>
>> Block Size: 32768b+ 65536b+
>> 131072b+ No. of Reads: 59 148
>> 555 No. of Writes: 0 0
>> 0
>>
>> %-latency Avg-latency Min-Latency Max-Latency No. of calls
>> Fop --------- ----------- ----------- ----------- ------------
>> ---- 0.00 0.00 us 0.00 us 0.00 us 1
>> FORGET 0.00 0.00 us 0.00 us 0.00 us 201
>> RELEASE 0.00 0.00 us 0.00 us 0.00 us 54549
>> RELEASEDIR 0.00 47.00 us 47.00 us 47.00 us 1
>> REMOVEXATTR 0.00 94.00 us 74.00 us 114.00 us 2
>> XATTROP 0.00 191.00 us 191.00 us 191.00 us 1
>> TRUNCATE 0.00 53.50 us 35.00 us 74.00 us 4
>> STATFS 0.00 79.67 us 70.00 us 91.00 us 3
>> RENAME 0.00 37.33 us 27.00 us 68.00 us 15
>> INODELK 0.00 190.67 us 116.00 us 252.00 us 3
>> UNLINK 0.00 28.83 us 8.00 us 99.00 us 30
>> ENTRYLK 0.00 146.33 us 117.00 us 188.00 us 6
>> CREATE 0.00 37.63 us 12.00 us 73.00 us 84
>> READDIR 0.00 23.75 us 8.00 us 75.00 us 198
>> FLUSH 0.00 65.33 us 42.00 us 141.00 us 204
>> OPEN 0.01 45.78 us 11.00 us 191.00 us 944
>> FINODELK 0.01 80.34 us 31.00 us 211.00 us 859
>> READ 0.02 96.74 us 50.00 us 188.00 us 944
>> FXATTROP 0.02 55.84 us 24.00 us 140.00 us 1707
>> FSTAT 0.02 52.89 us 21.00 us 175.00 us 2183
>> WRITE 0.02 59.69 us 11.00 us 235.00 us 2312
>> GETXATTR 0.03 51.18 us 8.00 us 142.00 us 3091
>> STAT 0.46 48.66 us 1.00 us 179.00 us 54549
>> OPENDIR 1.13 135.93 us 18.00 us 16362.00 us 48124
>> READDIRP 98.29 70.46 us 16.00 us 2903.00 us 8104385
>> LOOKUP
>>
>> Duration: 7560 seconds
>> Data Read: 91208567 bytes = 91MB
>> Data Written: 292007 bytes = 0,292MB
>
> How did you collect these statistics?
>
> Thanks
> Stefan
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
More information about the Gluster-users
mailing list