[Gluster-devel] performance improvements

[Daniel van Ham Colchete]

On 10/23/07, Vincent Régnard <vregnard at tbs-internet.com> wrote:
>
> Hi all,
>
> The afr synchronisation using "find -mtime -1 -type f -exec head -c1
> trick" takes approximately 30 minutes for a 20GB filesystem with 300.000
> files. Which seems too long to be acceptable for us. I'd like to tune
> some parameters to increase performance.
>
> Vincent.
>
Vicent,

so, in 1800 seconds you lookup(), open(), read() and close() 300.000 times?
That's 6ms for each file, and that's really good. Are you sure you
interconnect has a 5ms round-trip time? I would bet it is less.

IMO, usually you shouldn't measure GlusterFS performance with things
happening serially. GlusterFS is really good when things are happening in
parallel. I prefer to measure a network filesystem performance not on how
much time it takes to do one operation, but on how many operations it can do
in at the same time. If you had 300 threads trying to read all those files
it would be a lot faster. Usually that's the way real utilization happens,
if you have a webserver and a mail server using your storage, you will have
lots of web requests and e-mail sessions reading and writing at the same
time.

Best regards,
Daniel