[Gluster-devel] Size of GlusterFS installations? Hints? Alternatives?
Amar S. Tumballi
amar at zresearch.com
Tue May 27 23:16:27 UTC 2008
Hi Steffen,
Replies inline.
On Mon, May 26, 2008 at 8:01 AM, Steffen Grunewald <
steffen.grunewald at aei.mpg.de> wrote:
> I'm looking into GlusterFS to store data on a cluster (amd64, Debian Etch)
> in a distributed way with redundancy (kind of RAID-1, two nodes mirroring
> each other). Each node has a dedicated harddisk which can be used for data
> storage, and I'm still free to decide which FS to choose.
Sure, we advice you to feel comfortable before choosing anything.
>
> For easier recovery I'd prefer not to split files across nodes (typical
> size ~ a few MB).
>
Yes, we recommend that for filesizes below few hundred MBs.
>
> Would GlusterFS be suited for such a task?
Yes.
>
> Would it scale to a couple of hundreds of disk "pairs"?
Yes.
>
> Which underlying filesystem should I choose?
you have lot of options. ext3, reiserfs, xfs, ZFS (if you use Solaris, on
BSD extended attribute support for ZFS is not yet there, hence you may not
get RAID-1 functionality of GlusterFS)
>
>
> Since I'm using my own "homegrown" kernel, which modules would I have to
> build - is it mandatory to use the patched version of fuse?
No, its not mandatory to use GlusterFS patched fuse. Its just recommended
because, it has higher I/O performance. But yes, if you want to use
GlusterFS, you need FUSE module.
If you have Infiniband setup, few IB modules (mainly ib_uverbs) are required
to get ''ib-verbs'' tranport to work. GlusterFS has native support for
Infiniband userspace verbs (ie, RDMA support) protocol, using which you can
gain a lot in performance, for both i/o intensive and low latency
applications.
> Any suggestions for the stack of translators (and their order therein)?
> How to best organize redundancy? (since I don't have it in hardware, and
> I can get hold of "missing" files *if* I know their names, this should
> be not too hard?)
>
Unify (afr1 (disk1a, disk2a.. diskNa), afr2 (disk 1b, disk2b...diskNb)...
afrN(...)) [1]
Hope the above description is understood as a tree graph.
Yes, your file resides on the hardware (ie, on backend filesystem) as is.
So, you will get the file back, even if you take the disk out.
> Any alternatives? (as fas as I know, e.g. Lustre doesn't have redundancy
> features...)
>
No idea.
Regards,
Amar
[1] disks can be mapped to remote servers too.
--
Amar Tumballi
Gluster/GlusterFS Hacker
[bulde on #gluster/irc.gnu.org]
http://www.zresearch.com - Commoditizing Super Storage!
More information about the Gluster-devel
mailing list