[Gluster-devel] Performance issue

[Brent A Nelson]

On Wed, 16 Jul 2008, Ben Mok wrote:

> Hello ,
>
>
>
> I did the test to generate 100,000 files on glusterfs, each file is 4k, it
> need to take 150m27s . I did the same test in local disk and samba share,
> just 1m14s and 4m35s. Can you tell me how to improve the performance , since
> I want to use glusterfs to handle a large amount of files.
>
> Thank you very much !!
>
>
>
> Ben

FYI, the 1.4 branch is much more efficient handling metadata operations 
(switched from ASCII protocol to binary), so it is significantly faster in 
situations involving large numbers of small files.  It's not in an 
official release yet, unfortunately, but it does work nicely.

Perhaps the GlusterFS developers will consider doing an intermediate 
release with this improvement or make it an option in 1.3 (probably too 
big a change, though, and would be difficult reworking the code to make it 
optional), as the first official 1.4 release is probably still a ways 
away...

Thanks,

Brent

PS The developers are also working towards eliminating the namespace 
requirement for unify/stripe; this could be a significant improvement, as 
well.