[Gluster-devel] One client can effectively hang entire gluster array

[Glomski, Patrick]

Hello, Jeff.

Thanks for responding so quickly. I'm not familiar with the codebase, so if
you don't mind me asking, how much would that list reordering slow things
down for, say, a queue of 1500 client machines? i.e. round-about how long
of a client list would significantly affect latency?

I only ask because we have quite a few clients and you explicitly call out
that the queue reordering method used may have problems for lots of clients.

Thanks again,

Patrick


On Tue, Jul 12, 2016 at 11:18 AM, Jeff Darcy <jdarcy at redhat.com> wrote:

> > > * We might be able to tweak io-threads (which already runs on the
> > > bricks and already has a global queue) to schedule requests in a
> > > fairer way across clients. Right now it executes them in the
> > > same order that they were read from the network.
> >
> > This sounds to be an easier fix. We can make io-threads to factor in
> another
> > input i.e., the client through which request came in (essentially
> > frame->root->client) before scheduling. That should make the problem
> > bearable at-least if not crippling. As to what algorithm to use, I think
> we
> > can consider leaky bucket of bit-rot implementation or dmclock. I've not
> > really thought deeper about the algorithm part. If the approach sounds
> ok,
> > we can discuss more about algos.
>
> I've created a patch to address the most basic part of this, in the
> simplest
> way I could think of.
>
> http://review.gluster.org/#/c/14904/
>
> It's still running through basic tests, so I don't even know if it's really
> correct yet, but it should give an idea of the conceptual direction.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160712/1449bdc2/attachment.html>