[Gluster-users] GlusterFS with FUSE slow vs ZFS volume

Thu Feb 5 11:17:18 UTC 2015

+Kiran Patil may know about this.

Pranith
On 02/03/2015 12:56 AM, ML mail wrote:
> Hello,
>
> I am testing GlusterFS for the first time and have installed the latest GlusterFS 3.5 stable version on Debian 7 on brand new SuperMicro hardware with ZFS instead of hardware RAID. My ZFS pool is a RAIDZ-2 with 6 SATA disks of 2 TB each.
>
> After setting up a first and single test brick on my currently single test node I wanted first to see how much slower will GlusterFS be compared to writting directly to the ZFS volume. For that purpose I have mounted my GlusterFS volume locally on the same server using FUSE.
>
> For my tests I have used bonnie++ with the command "bonnie++ -n16 -b" and I must say I am quite shocked to see that with this current setup GlusterFS slows down the whole file system with a factor of approximately 6 to 8. For example:
>
> ZFS volume
>
> Sequential output by block (read): 936 MB/sec
> Sequential input by block (write): 1520 MB/sec
>
>
> GlusterFS on top of same ZFS volume mounted with FUSE
> Sequential output by block (read): 114 MB/sec
> Sequential input by block (write): 312 MB/sec
>
>
> Now I was wondering if such a performance drop on a single GlusterFS node is expected? If not is it maybe ZFS which is messing up things?
>
> bonnie++ took 3 minutes to rune on the ZFS volume and 18 minutes on the GlusterFS mount. I have copied the bonnie++ results below just in case in CVS format:
>
> 1.96,1.96,ZFS,1,1422907597,31960M,,170,99,936956,94,484417,74,463,99,1520120,98,815.4,41,16,,,,,3376,26,+++++,+++,3109,22,3261,21,+++++,+++,3305,20,66881us,15214us,84887us,23648us,53641us,93322us,39607us,363us,298ms,136ms,18us,176ms
> 1.96,1.96,GFS,1,1422897979,31960M,,16,17,114223,20,92610,20,+++++,+++,312557,14,444.5,6,16,,,,,385,3,5724,5,916,4,357,3,2044,4,750,4,550ms,9715us,23094us,3334us,125ms,90070us,154ms,8609us,17570us,67180us,4116us,7879us
>
> Maybe they are a few performance tuning trick that I am not aware of?
>
> Let me know if I should provide any more information. In advance thanks for your comments.
>
> Best regards
> ML
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users