[Gluster-users] best practices? 500+ win/mac computers, gluster, samba, new SAN, new hardware

[Brian Candler]

On Wed, Feb 01, 2012 at 12:21:17PM -0600, D. Dante Lorenso wrote:
> >Gluster does give you the option of a "distributed replicated" volume, so
> >you can get both the "RAID 0" and "RAID 1" functionality.
> 
> If you have 8 drives connected to a single machine, how do you
> introduce those drives to Gluster?  I was thinking I'd combine them
> into a single volume using RAID 0 and mount that volume on a box and
> turn it into a brick.  Otherwise you have to add 8 separate bricks,
> right?  That's not better is it?

I'm in the process of building a pair of test systems (in my case 12 disks
per server), and haven't quite got to building the Gluster layer, but yes 8
separate filesystems and 8 separate bricks per server is what I'm suggesting
you consider.

Then you create a distributed replicated volume using 16 bricks across 2
servers, added in the correct order so that they pair up and down
(serverA:brick1 serverB:brick1 serverA:brick2 serverB:brick2 etc) - or
across 4 servers or however many you're building.

The advantage is that if you lose one disk, 7/8 of the data is still usable
on both disks, and 1/8 is still available on one disk.  If you lose a second
disk, there is a 1 in 15 chance that it's the mirror of the other failed
one, but a 14 in 15 chance that you won't lose any data.  Furthermore,
replacing the failed disk will only have to synchronise (heal) one disk
worth of data.

Now, if you decide to make RAID0 sets instead, then losing one disk will
destroy the whole filesystem.  If you lose any disk in the second server you
will have lost everything.  And when you replace the one failed disk, you
will need to make a new filesystem across the whole RAID0 array and resync
all 8 disks worth of data.

I think it only makes sense to build an array brick if you are using RAID1
or higher.  RAID1 or RAID10 is fast but presumably you don't want to store 4
copies of your data, 2 on each server.  The write performance of RAID5 and
RAID6 is terrible.  An expensive RAID card with battery-backed write-through
cache will make it slightly less terrible, but still terrible.

Regards,

Brian.