[Gluster-users] Configuration Advice

[Brian Candler]

On Thu, May 31, 2012 at 11:30:28AM -0700, Mike Seda wrote:
> Hi All,
> Does the following GlusterFS config seem solid for a small (24
> compute-node) HPC cluster with a mixed I/O pattern?
> - 4 storage-nodes (distribute-only) with 3 bricks
> - Each storage-node will be HP DL360 G8 (32 GB RAM, 16 core)
> - Each storage-node will have 1 HP D2600 shelf with 12 x 2 TB drives
> (1 RAID 10 given to OS as 1 LVM PV)

Comments and things to consider:

* I'd say CPU and RAM are massive overkill for a storage node. IMO 4 core
and 8GB would be fine, unless your working data set is so small it would
fit into 32GB of cache.
* RAID 10 for the OS? Do you mean dedicating *four* disks for the OS? Also
seems overkill. Two disks of RAID1 would be more than enough. Or if you
are using LVM, a small LV for the OS (plus a small boot partition).
The DL360 appears to have 8 x 2.5" drive slots, are they part of the data
array or just for OS?
* What form of RAID are you planning to use for the data? Hardware raid
controller or software md raid?  RAID10, RAID6, ..?  For a mixed I/O pattern
which contains more than a tiny amount of writes, don't even consider RAID5
or RAID6.
* Why three bricks per node? Are you planning to split the 12 x 2 TB drives
into three 4-drive arrays?
* What sort of network are you linking this to the compute nodes with?
GigE, 10gigE, Infiniband, something else?

> If so, do you think I get away with only 2 storage-nodes?

That's entirely dependent on your application. Since you are doing
distribution, not replication, maybe you could get away with one storage
node? What if you had 3TB drives instead of 2TB drives?