[Gluster-users] GlusterFS performance

[Nikita A Kardashin]

Hello all!

This problem is solved by me today.
Root of all in the incompatibility of gluster cache and kvm cache.

Bug reproduces if KVM virtual machine created with cache=writethrough
(default for OpenStack) option and hosted on GlusterFS volume. If any other
(cache=writeback or cache=none with direct-io) cacher used, performance of
writing to existing file inside VM is equal to bare storage (from host
machine) write performance.

I think, it must be documented in Gluster and maybe filled a bug.

Other question. Where I can read something about gluster tuning (optimal
cache size, write-behind, flush-behind use cases and other)? I found only
options list, without any how-to or tested cases.


2013/3/5 Toby Corkindale <toby.corkindale at strategicdata.com.au>

> On 01/03/13 21:12, Brian Candler wrote:
>
>> On Fri, Mar 01, 2013 at 03:30:07PM +0600, Nikita A Kardashin wrote:
>>
>>>     If I try to execute above command inside virtual machine (KVM), first
>>>     time all going right - about 900MB/s (cache effect, I think), but if
>>> I
>>>     run this test again on existing file - task (dd) hungs up and can be
>>>     stopped only by Ctrl+C.
>>>     Overall virtual system latency is poor too. For example, apt-get
>>>     upgrade upgrading system very, very slow, freezing on "Unpacking
>>>     replacement" and other io-related steps.
>>>     Does glusterfs have any tuning options, that can help me?
>>>
>>
>> If you are finding that processes hang or freeze indefinitely, this is not
>> a question of "tuning", this is simply "broken".
>>
>> Anyway, you're asking the wrong person - I'm currently in the process of
>> stripping out glusterfs, although I remain interested in the project.
>>
>> I did find that KVM performed very poorly, but KVM was not my main
>> application and that's not why I'm abandoning it.  I'm stripping out
>> glusterfs primarily because it's not supportable in my environment,
>> because
>> there is no documentation on how to analyse and recover from failure
>> scenarios which can and do happen. This point in more detail:
>> http://www.gluster.org/**pipermail/gluster-users/2013-**
>> January/035118.html<http://www.gluster.org/pipermail/gluster-users/2013-January/035118.html>
>>
>> The other downside of gluster was its lack of flexibility, in particular
>> the
>> fact that there is no usage scaling factor on bricks, so that even with a
>> simple distributed setup all your bricks have to be the same size.  Also,
>> the object store feature which I wanted to use, has clearly had hardly any
>> testing (even the RPM packages don't install properly).
>>
>> I *really* wanted to deploy gluster, because in principle I like the idea
>> of
>> a virtual distribution/replication system which sits on top of existing
>> local filesystems.  But for storage, I need something where operational
>> supportability is at the top of the pile.
>>
>
> I have to agree; GlusterFS has been in use here in production for a while,
> and while it mostly works, it's been fragile and documentation has been
> disappointing. Despite 3.3 being in beta for a year, it still seems to have
> been poorly tested. For eg, I can't believe almost no-one else noticed that
> the log files were busted.. nor that the bug report has been around for
> quarter of a year without being responded to or fixed.
>
> I have to ask -- what are you moving to now, Brian?
>
> -Toby
>
>
> ______________________________**_________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.**org/mailman/listinfo/gluster-**users<http://supercolony.gluster.org/mailman/listinfo/gluster-users>
>



-- 
With best regards,
differentlocal (www.differentlocal.ru | differentlocal at gmail.com),
System administrator.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130305/b5b5f19d/attachment.html>