[Gluster-users] Write performance tragically slow

[Pavel Snajdr]

I'd use NFS in first place, but I can't due to 16 secondary gids limit 
while using AUTH_SYS.
I can't use kerberos in my environment (OpenVZ containers above this 
storage).

So are there any plans to improve performance for small writes?


On 20.11.2010 00:36, Harshavardhana wrote:
> On 11/18/2010 07:28 AM, Barry Jaspan wrote:
>> On Thu, Nov 18, 2010 at 9:41 AM, Anand Avati<anand.avati at gmail.com>
>> wrote:
>>
>>> FUSE based filesystems have to face the unfortunate overhead of context
>>> switches between application and the filesystem process.
>>
>> Gluster 2.x had a library named "booster" that bypassed the FUSE layer by
>> providing implementations of all the filesystem syscalls that recognized
>> glusterfs file descriptors and went directly to the glusterfs code
>> instead
>> of the kernel for those operations.
>>
>> Why was booster removed from Gluster 3.x?
>>
>> Barry
> Barry,
>
> Booster was unstable by many reasons. I have did extensive tests to see
> the difference in throughput based with booster. It revealed that
> bypassing FUSE in itself didn't give much performance improvement,
> actually there were no improvements at all. There were calls like
> OPENDIR, READDIR which were slow and snaggy to respond, where as native
> FUSE was far better.
>
> Context switch overhead for small file performance from a network
> perspective was high enough
> to mask the context with FUSE and VFS layers. So it doesn't matter if
> you bypass FUSE it still hits high on network latency, but in generality
> CPU speed matters for context switching performance.
>
> We used taskset to some extent for small file performance to provide
> more CPU affinity for network i/o for glusterfsd if the user is
> primarily a small file workload. It helps to an extent, but things can
> be different many times under different workloads.
>
> NFS native solves it to large extent with its aggressive client side
> caching of many attributes, since stats are served locally for NFS client.
>
> Booster never evolved much more and we saw the side effects. Since the
> problem is elsewhere, so eventually it has to be dropped.
>
> Regards
>