[Gluster-users] glusterfs unusable?

[richard lucassen]

On Wed, 27 Mar 2019 07:53:55 -0700
Joe Julian <joe at julianfamily.org> wrote:

Ok Joe, this is the situation: I have a glusterfs cluster using R630
Dell servers with 256GB of memory, a bunch of 3.4TB SSD's and Intel Xeon
E5-2667 beasts. Using such power and seeing glusterfs taking 5
seconds for a simple "ls -alR" on a client directly connected over a
1Gbit cable to these servers is rather slow (this will be nominated
for The Understement Of The Week)  Rather slow, not unusable (and I
haven't even added an arbiter to these two servers yet)

OTOH, contrary to what you suggest, I'm not using a brick at home, it is
just a linux client connecting to these two servers, ok, I admit, over a
slow line. I was just looking how long it would take
before a simple "ls -alR" would take. And when this takes almost an
hour consuming 2GB upload then I think I can say it's quite unusable.

So I'm sorry Joe, I don't want to spoil your day, but I have to say
that Glusterfs (sorry for the wrong abbreviation) did spoil my day
because of this issue. Such a bad behaviour would certainly be a show
stopper.

I hope the patches will resolve these issues.

R.

> First, your statement and subject is hyperbolic and combative. In
> general it's best not to begin any approach for help with an
> uneducated attack on a community.
> 
> GFS (Global File System) is an entirely different project but I'm
> going to assume you're in the right place and actually asking about
> GlusterFS.
> 
> You haven't described your use case so I'll make an assumption that
> your intent is to sync files from your office to your home. I'll
> further guess that you're replicating one brick at home and the other
> at the office.
> 
> Yes, this is generally an unusable use case for to latency and
> connectivity reasons. Your 2Gb transfer was very likely a self heal
> due to a connectivity problem from one of your clients. When your
> home client performed a lookup() of the files, it caught the
> discrepancy and fixed it. The latency is multiplied due to the very
> nature of clustering and your latent connection.
> 
> For a more useful answer, I'd suggest describing your needs and
> asking for help. There is tons of experienced storage professionals
> here that are happy to share their knowledge and advice.
> 
> On March 27, 2019 7:23:35 AM PDT, richard lucassen
> <mailinglists at lucassen.org> wrote:
> >Hello list,
> >
> >glusterfs 5.4-1 on Debian Buster (both servers and clients)
> >
> >I'm quite new to GFS and it's an old problem I know. When running a
> >simple "ls -alR" on a local directory containing 50MB and 3468 files
> >it takes:
> >
> >real    0m0.567s
> >user    0m0.084s
> >sys     0m0.168s
> >
> >Same thing for a copy of that dir on GFS takes more than 5 seconds:
> >
> >real    0m5.557s
> >user    0m0.128s
> >sys     0m0.208s
> >
> >Ok. But from my workstation at home, an "ls -alR" of that directory
> >takes more than half an hour and the upload is more than 2GB (no
> >typo: TWO Gigabytes). To keep it simple, the ls of a few directories:
> >
> >$ time ls
> >all  xabc-db  xabc-dc1  xabc-gluster  xabc-mail  xabc-otp  xabc-smtp
> >
> >real    0m5.766s
> >user    0m0.001s
> >sys     0m0.003s
> >
> >it receives 56kB and sends 2.3 MB for a simple ls.
> >
> >This is weird isn't it? Why this huge upload?
> >
> >Changing these options mentioned here doesn't make any difference:
> >
> >https://lists.gluster.org/pipermail/gluster-users/2016-January/024865.html
> >
> >Anyone a hint? Or should I drop GFS? This is unusable IMHO.
> >
> >Richard.
> >
> >-- 
> >richard lucassen
> >http://contact.xaq.nl/
> >_______________________________________________
> >Gluster-users mailing list
> >Gluster-users at gluster.org
> >https://lists.gluster.org/mailman/listinfo/gluster-users
> 


-- 
richard lucassen
http://contact.xaq.nl/