[Gluster-users] Configuration Advice

[Mike Seda]

On 06/01/2012 12:53 AM, Brian Candler wrote:
> On Thu, May 31, 2012 at 11:30:28AM -0700, Mike Seda wrote:
>> Hi All,
>> Does the following GlusterFS config seem solid for a small (24
>> compute-node) HPC cluster with a mixed I/O pattern?
>> - 4 storage-nodes (distribute-only) with 3 bricks
>> - Each storage-node will be HP DL360 G8 (32 GB RAM, 16 core)
>> - Each storage-node will have 1 HP D2600 shelf with 12 x 2 TB drives
>> (1 RAID 10 given to OS as 1 LVM PV)
> Comments and things to consider:
>
> * I'd say CPU and RAM are massive overkill for a storage node.

Right. I figured. I got my original numbers from here though:
http://download.gluster.com/pub/gluster/RHSSA/3.2/Documentation/UG/html/sect-User_Guide-gssa_prepare-chec_min_req.html

> IMO 4 core
> and 8GB would be fine, unless your working data set is so small it would
> fit into 32GB of cache.
> * RAID 10 for the OS? Do you mean dedicating *four* disks for the OS?

Nope. I mean one RAID 10 for the entire D2600 shelf. Apparently, you can 
have a RAID 10 (1+0 or whatever) that spans more than 4 drives. Never 
done it, but I've heard that it's possible.

> Also
> seems overkill. Two disks of RAID1 would be more than enough.

That's my plan.

> Or if you
> are using LVM, a small LV for the OS (plus a small boot partition).

Yep. That's the plan.

> The DL360 appears to have 8 x 2.5" drive slots, are they part of the data
> array or just for OS?

I will only have 2 drives there just for that RAID 1.

> * What form of RAID are you planning to use for the data? Hardware raid
> controller or software md raid?  RAID10, RAID6, ..?  For a mixed I/O pattern
> which contains more than a tiny amount of writes, don't even consider RAID5
> or RAID6.

Hardware RAID 10.

> * Why three bricks per node? Are you planning to split the 12 x 2 TB drives
> into three 4-drive arrays?

Well I was going to split up the large RAID 10 LUN with LVM. Each brick 
would map to it's own dedicated LV on each node.

> * What sort of network are you linking this to the compute nodes with?
> GigE, 10gigE, Infiniband, something else?

1GbE unfortunately.

>> If so, do you think I get away with only 2 storage-nodes?
> That's entirely dependent on your application. Since you are doing
> distribution, not replication, maybe you could get away with one storage
> node? What if you had 3TB drives instead of 2TB drives?

My hands are tied there. 2 TB is what we have.

Thanks for the response.