[Gluster-devel] Small File Performance - Strange results

[Joe Julian]

On 08/04/2015 05:53 PM, Shyam wrote:
> On 08/04/2015 12:55 PM, Hafeez Bana wrote:
>> All,
>>
>> We've been evaluating glusterfs 3.2.7 on ubuntu 14.04 LTS. All tests
>> were run with event-thread matching cpu-cores and lookup-unhashed 
>> turned off
>
> I think you are referring to lookup-optimize rather than 
> lookup-unhashed, please use lookup-optimize which is meant for the 
> small file performance improvement.
>

No, he means unhashed. lookup-optimize isn't in 3.2. One of his biggest 
problems will be the lack of readdirp support.

>>
>> We noted a couple of issues and some interesting results which might be
>> of benefit to the developers
>>
>> Setup
>> - bare metal machine with SSD running ZFS - 2 LXC container running
>> glusterd and another running a samba instance which is serving a share
>> backed by a fuse mounted volume.
>
> There are too many things in the stack here to reliably measure 
> performance is what I would think.
>
> I would first, run atop the FUSE mount and compare results with 3.6.x 
> to see if the options provided improve the performance or not.
>
>> Observations
>> 1 )  the fuse mounted volume has to be done in the host and LXC run on
>> top of it. If you try to fuse mount within in LXC - posix acls group
>> persmissions are not respected completed by gluster
>> 2) small file performance - tests were run unzipping from a windows box
>> a 45M and 150M  (expanded size - contains 8000 files) archive. 45M
>> performance is reasonable. 150M tests takes 10 times as long(about 18
>> minutes) as the same samba instance serving of a straight zfs filesystem
>> 3) As a whim we logged into the container containing the samba instance
>> and copied files from the ZFS folder to the folder under gluster.
>> 14seconds for the same transfer of 8000 files!!
>>
>> We don't know if there is a mismatch in the config of samba that is
>> causing issues with gluster. Any ideas? And also is the issue with posix
>> acls groups being broken if you fuse_mount a gluster volume within LXC
>> known?
>>
>> Finally, are there any gluster developers in the UK - we would love to
>> talk to them.
>>
>> Thanks,
>> Hafeez
>>
>>