[Gluster-users] Question about Volume Type when bricks are on SAN

[Mike Hanby]

Thanks Craig, that's good information.

From: Craig Carl [mailto:craig at gluster.com]
Sent: Tuesday, October 26, 2010 9:53 PM
To: Mike Hanby
Cc: gluster-users at gluster.org
Subject: Re: [Gluster-users] Question about Volume Type when bricks are on SAN

Mike -
   Gluster does not use LVM, but we do support using it under Gluster. There are only three use cases for Gluster and LVM that make sense.

1. You want to snapshot the filesystem (logical volume).
2. Your RAID controller won't create multiple LUN's and you have to export a very large filesystem. You can use LVM to carve that filesystem into multiple, smaller logical volumes for faster fscks and (see 3).
3. You have a Gluster Storage server with more disk space that others. Gluster will perform better if every brick in the cluster is the same size. If you are using a brick size of 2TB and you add a server with 10TB to the cluster you should, for best performance, create 5 2TB logical volumes with LVM then add all of those bricks to a volume;

gluster> volume add-brick avolume server1:/dev/vg1/lv1
gluster> volume add-brick avolume server1:/dev/vg1/lv2
gluster> volume add-brick avolume server1:/dev/vg1/lv3
gluster> volume add-brick avolume server1:/dev/vg1/lv4
gluster> volume add-brick avolume server1:/dev/vg1/lv5
and of course
gluster> volume rebalance avolume start

Remember, your smallest brick size MUST be larger that your largest file. Once your Gluster volumes are created Gluster will hide all of the bricks under a single namespace.


Thanks,
Craig

-->
Craig Carl
Gluster, Inc.
Cell - (408) 829-9953 (California, USA)
Gtalk - craig.carl at gmail.com


________________________________
From: "Mike Hanby" <mhanby at uab.edu>
To: gluster-users at gluster.org
Sent: Monday, October 25, 2010 3:45:29 PM
Subject: Re: [Gluster-users] Question about Volume Type when bricks        are        on        SAN

So based on the suggestions from both you and Daniel many smaller LUNs should be better for performance and maintenance.

Question regarding LVM, do you use LVM to place all of the luns into a single volume group and ultimately a single logical volume / brick? Or does each LUN become it's own volume group and logical volume and brick?

Based on what you suggested regarding fsck on smaller file systems I suspect the latter, where you'd have many smaller bricks per server rather than a single large brick comprised of many luns in a single logical volume.

Thanks,

Mike

-----Original Message-----
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of Craig Carl
Sent: Sunday, October 24, 2010 2:15 AM
To: Daniel Mons
Cc: gluster-users at gluster.org
Subject: Re: [Gluster-users] Question about Volume Type when bricks are on SAN

Another good reason to keep your LUN's small is the possibility of having to fsck the filesystem. Even on really fast disk it can take days to fsck a 16TB LUN. Gluster hides the multiple LUNs in a volume so you never have to deal with them, other than a bit more SAN management and more typing to setup a volume the first time there is no downside.

Thanks,

Craig

-->
Craig Carl
Senior Systems Engineer
Gluster


From: "Daniel Mons" <daemons at kanuka.com.au>
To: gluster-users at gluster.org
Sent: Saturday, October 23, 2010 5:38:12 AM
Subject: Re: [Gluster-users] Question about Volume Type when bricks are on SAN

On Sat, Oct 23, 2010 at 5:22 AM, Mike Hanby <mhanby at uab.edu> wrote:
> Each Gluster server has been allocated a single 9TB LUN (if it makes any sense to do so, I could have the SAN admin provide many smaller LUNs per server and use LVM to combine into single brick)

Generally speaking you should LVM over many smaller LUNs. With
multiport FC HBAs being the standard, you will get better performance
as separate requests hit separate LUNs. Multiple LUNs mean multiple
SCSI queues, which can reduce I/O latency substantially.

We try to cap our LUNs at around 100GB. It does make for a lot of
LUNs per server, but it's worth if for the performance gain. Talk to
your storage admin and ask them what their recommendation is based on
your vendor and their best practices.

-Dan
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users