[Gluster-users] Slow performance from simple tar -x && rm -r benchmark

[John Lauro]

Bryan can you reproduce this test and see what times you get?  Just extracting a tar file of the linux kernel...  I had the same problem when I looked into gluster a few months ago.

I would be interested if you can get acceptable performance for this type of operation.

>From what I could tell it is normal for such slow performance when creating lots and lots of small files.  That is why gluster isn't recommended for homedirectories, etc...  Reading lots of small files I think is ok, and updating large files is ok, but creating lots of small files is terrible.

To show that large files are fine, you can try the following expirment.  Create a large file (ie: 20gb file with dd) and make a loopback device out it and then mount the loopback device, you can then get decent performance out of that loopback running on gluster, but obviously only one machine would be able to mount that loopback device at a time...  That test shows it might work ok for hosting virtual machines.


----- Original Message -----
From: "Bryan Whitehead" <driver at megahappy.net>
To: "Chris Webb" <chris at arachsys.com>
Cc: gluster-users at gluster.org
Sent: Monday, March 19, 2012 5:00:02 PM
Subject: Re: [Gluster-users] Slow performance from simple tar -x && rm -r	benchmark

I have a number of labs I test my glusterfs installs on. From
Infinband 40G w/switch and also some cheap $800 boxes on a gig
network.

None of them exhibit the poor performance i'm seeing in your post - so
I'm just throwing out the differences I'm seeing with your config vs
mine.

Another option you might want to try is increasing the max number of threads:

gluster volume set <name> performance.io-thread-count 64


On Mon, Mar 19, 2012 at 2:34 AM, Chris Webb <chris at arachsys.com> wrote:
> Bryan Whitehead <driver at megahappy.net> writes:
>
>> I didn't see any sync's after the tar/rm commands...
>
> By default, ext4 flushes both metadata and data every five seconds, so a
> post-benchmark sync tends to make little difference on a reasonable large test,
> but for completeness:
>
>  # time bash -c 'tar xfz ~/linux-3.3-rc7.tgz; sync; rm -rf linux-3.3-rc7; sync'
>  real    0m23.826s
>  user    0m20.681s
>  sys     0m2.392s
>
> vs
>
>  # time bash -c 'tar xfz ~/linux-3.3-rc7.tgz; sync; rm -rf linux-3.3-rc7; sync'
>
>  real    4m24.067s
>  user    0m24.692s
>  sys     0m7.588s
>
> showing very similar timings and the same effect.
>
>> try using xfs instead of ext4.
>
> I'll build the xfs tooling, add kernel support, and give this a go, but I'm
> surprised you think changing the underlying filesystem would eliminate the big
> gap between native and gluster performance. I could imagine it improving both
> somewhat, but if anything, I'd expect a higher performance filesystem to
> amplify the differences. Do you think that glusterfs does something that's
> particularly expensive on ext4, much more expensive than the operations proxied
> through it?
>
> Best wishes,
>
> Chris.
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users