[Gluster-users] To GlusterFS or not...

[Jeff Darcy]

> The biggest issue that we are having, is that we are talking about
> -billions- of small (max 5MB) files. Seek times are killing us
> completely from what we can make out. (OS, HW/RAID has been tweaked to
> kingdom come and back).

This is probably the key point.  It's unlikely that seek times are going
to get better with GlusterFS, unless it's because the new servers have
more memory and disks, but if that's the case then you might as well
just deploy more memory and disks in your existing scheme.  On top of
that, using any distributed file system is likely to mean more network
round trips, to maintain consistency.  There would be a benefit from
letting GlusterFS handle the distribution (and redistribution) of files
automatically instead of having to do your own sharding, but that's not
the same as a performance benefit.

> I’m not yet too clued up on all the GlusterFS naming, but essentially
> if we do go the GlusterFS route, we would like to use non replicated
> storage bricks on all the front-end, as well as back-end servers in
> order to maximize storage.

That's fine, so long as you recognize that recovering from a failed
server becomes more of a manual process, but it's probably a moot point
in light of the seek-time issue mentioned above.  As much as I hate to
discourage people from using GlusterFS, it's even worse to have them be
disappointed, or for other users with other needs to be so as we spend
time trying to fix the unfixable.