[Gluster-devel] Performance issue

[Anand Avati]

> FYI, the 1.4 branch is much more efficient handling metadata operations
> (switched from ASCII protocol to binary), so it is significantly faster in
> situations involving large numbers of small files.  It's not in an official
> release yet, unfortunately, but it does work nicely.
>
> Perhaps the GlusterFS developers will consider doing an intermediate
> release with this improvement or make it an option in 1.3 (probably too big
> a change, though, and would be difficult reworking the code to make it
> optional), as the first official 1.4 release is probably still a ways
> away...
>

Porting the binary protocol and nbio (which came hand-in-hand into 1.4) will
be pretty disruptive in the 1.3 series. We are working hard on getting 1.4
out soon.


>
> PS The developers are also working towards eliminating the namespace
> requirement for unify/stripe; this could be a significant improvement, as
> well.
>

Currently only unify needs namespace. stripe does not. The new branch will
have unify as is (with the namespace etc) but a new translator which will be
a functional replacement to unify will be introduced. This new translator
(hash) internally builds a distributed hash table over the storage servers
improving the FS performance significantly. The DHT is similar in concept to
many of the popular DHT algos, and hashes the filename and stores the file
on the hashed node. So functionally, it is similar to unify (directory
structure is replicated, file exists on only one node) but the internals are
quite different, without a namespace too.

avati