[Gluster-devel] Small File Performance - Strange results

[Shyam]

On 08/04/2015 12:55 PM, Hafeez Bana wrote:
> All,
>
> We've been evaluating glusterfs 3.2.7 on ubuntu 14.04 LTS. All tests
> were run with event-thread matching cpu-cores and lookup-unhashed turned off

I think you are referring to lookup-optimize rather than 
lookup-unhashed, please use lookup-optimize which is meant for the small 
file performance improvement.

>
> We noted a couple of issues and some interesting results which might be
> of benefit to the developers
>
> Setup
> - bare metal machine with SSD running ZFS - 2 LXC container running
> glusterd and another running a samba instance which is serving a share
> backed by a fuse mounted volume.

There are too many things in the stack here to reliably measure 
performance is what I would think.

I would first, run atop the FUSE mount and compare results with 3.6.x to 
see if the options provided improve the performance or not.

> Observations
> 1 )  the fuse mounted volume has to be done in the host and LXC run on
> top of it. If you try to fuse mount within in LXC - posix acls group
> persmissions are not respected completed by gluster
> 2) small file performance - tests were run unzipping from a windows box
> a 45M and 150M  (expanded size - contains 8000 files) archive. 45M
> performance is reasonable. 150M tests takes 10 times as long(about 18
> minutes) as the same samba instance serving of a straight zfs filesystem
> 3) As a whim we logged into the container containing the samba instance
> and copied files from the ZFS folder to the folder under gluster.
> 14seconds for the same transfer of 8000 files!!
>
> We don't know if there is a mismatch in the config of samba that is
> causing issues with gluster. Any ideas? And also is the issue with posix
> acls groups being broken if you fuse_mount a gluster volume within LXC
> known?
>
> Finally, are there any gluster developers in the UK - we would love to
> talk to them.
>
> Thanks,
> Hafeez
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>