[Gluster-users] Question about Volume Type when bricks are on SAN

[Craig Carl]

Mike - 
Gluster does not use LVM, but we do support using it under Gluster. There are only three use cases for Gluster and LVM that make sense. 

1. You want to snapshot the filesystem (logical volume). 
2. Your RAID controller won't create multiple LUN's and you have to export a very large filesystem. You can use LVM to carve that filesystem into multiple, smaller logical volumes for faster fscks and (see 3). 
3. You have a Gluster Storage server with more disk space that others. Gluster will perform better if every brick in the cluster is the same size. If you are using a brick size of 2TB and you add a server with 10TB to the cluster you should, for best performance, create 5 2TB logical volumes with LVM then add all of those bricks to a volume; 

gluster> volume add-brick avolume server1:/dev/vg1/lv1 
gluster> volume add-brick avolume server1:/dev/vg1/lv2 
gluster> volume add-brick avolume server1:/dev/vg1/lv3 
gluster> volume add-brick avolume server1:/dev/vg1/lv4 
gluster> volume add-brick avolume server1:/dev/vg1/lv5 
and of course 
gluster> volume rebalance avolume start 

Remember, your smallest brick size MUST be larger that your largest file. Once your Gluster volumes are created Gluster will hide all of the bricks under a single namespace. 




Thanks, 
Craig 

--> 
Craig Carl 



Gluster, Inc. 
Cell - (408) 829-9953 (California, USA) 
Gtalk - craig.carl at gmail.com 


From: "Mike Hanby" <mhanby at uab.edu> 
To: gluster-users at gluster.org 
Sent: Monday, October 25, 2010 3:45:29 PM 
Subject: Re: [Gluster-users] Question about Volume Type when bricks are on SAN 

So based on the suggestions from both you and Daniel many smaller LUNs should be better for performance and maintenance. 

Question regarding LVM, do you use LVM to place all of the luns into a single volume group and ultimately a single logical volume / brick? Or does each LUN become it's own volume group and logical volume and brick? 

Based on what you suggested regarding fsck on smaller file systems I suspect the latter, where you'd have many smaller bricks per server rather than a single large brick comprised of many luns in a single logical volume. 

Thanks, 

Mike 

-----Original Message----- 
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of Craig Carl 
Sent: Sunday, October 24, 2010 2:15 AM 
To: Daniel Mons 
Cc: gluster-users at gluster.org 
Subject: Re: [Gluster-users] Question about Volume Type when bricks are on SAN 

Another good reason to keep your LUN's small is the possibility of having to fsck the filesystem. Even on really fast disk it can take days to fsck a 16TB LUN. Gluster hides the multiple LUNs in a volume so you never have to deal with them, other than a bit more SAN management and more typing to setup a volume the first time there is no downside. 

Thanks, 

Craig 

--> 
Craig Carl 
Senior Systems Engineer 
Gluster 


From: "Daniel Mons" <daemons at kanuka.com.au> 
To: gluster-users at gluster.org 
Sent: Saturday, October 23, 2010 5:38:12 AM 
Subject: Re: [Gluster-users] Question about Volume Type when bricks are on SAN 

On Sat, Oct 23, 2010 at 5:22 AM, Mike Hanby <mhanby at uab.edu> wrote: 
> Each Gluster server has been allocated a single 9TB LUN (if it makes any sense to do so, I could have the SAN admin provide many smaller LUNs per server and use LVM to combine into single brick) 

Generally speaking you should LVM over many smaller LUNs. With 
multiport FC HBAs being the standard, you will get better performance 
as separate requests hit separate LUNs. Multiple LUNs mean multiple 
SCSI queues, which can reduce I/O latency substantially. 

We try to cap our LUNs at around 100GB. It does make for a lot of 
LUNs per server, but it's worth if for the performance gain. Talk to 
your storage admin and ask them what their recommendation is based on 
your vendor and their best practices. 

-Dan 
_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users 
_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users