[Gluster-users] Gluster pool help

[Ted Miller]

On 9/9/2014 7:12 PM, Michael Bushey wrote:
> I'm currently using GlusterFS 3.5.2 on a pair of production servers to 
> share uploaded files and it's been reliable and working well with just the 
> two servers. I've done some local tests of trying to add and remove servers 
> and the results have not been good. I'm starting to think I have the wrong 
> idea of what GlusterFS does and I'm trying to stick the square peg in the 
> round hole. Is there some config to say "number of replicas = number of 
> servers"?
>
> My need is to have a copy of the data local on each server. If I have 5 
> servers, I want five copies of the data. Basically each server should have 
> it's own copy like it's a local ext4/zfs file-system, except it will 
> immediately sync files added or removed on the other servers. I'm not sure 
> the what the gluster definition of brick is, but I believe I want the 
> number of bricks to equal the number of servers (at least for this 
> particular file-system).
>
> I've played around a bit and lost my data every time I've tried to add or 
> remove a node. There is plenty of documentation, but it all says the same 
> thing and doesn't answer the basics.
>
> From `gluster volume info`: Number of Bricks: 1 x 2 = 2
>
> I'm assuming the middle 2 is the number of bricks. I have no clue why we're 
> multiplying by 1 to get itself.
> http://gluster.org/community/documentation/index.php/Gluster_3.2:_Displaying_Volume_Information 
> - does not show this magic multiplication. The page for 3.3 does not exist.
>
> We're cloud based and we want to be able to add and remove servers on the 
> fly. If for example I want to scale from 5 to 7 servers, it looks like I 
> need to create a new storage pool with 7 replicas. Would it be the most 
> logical to name my brick with the number of replicas? For example server1 
> to server5 already have /var/gluster/files5. I could then create 
> /var/gluster/files7 on server1 to server7, create the pool with 7 replicas, 
> then pick a machine like server1 to copy the data from /var/gluster/files5 
> to /var/gluster/files7, then destroy /var/gluster/files5. This seems 
> extremely convoluted but it does not appear possible to expand a storage 
> pool and expand the number of replicants with it.
>
> Thanks in advance for help and your time to read this.
> Michael
I have done this a few times, but it is not real recent, so I will try to 
explain in a way that is both correct and understandable.

1. As to the question about the "`gluster volume info`: Number of Bricks: 1 x 
2 = 2"
The first number is the number of bricks that the data is distributed 
across.  If that number is 3, each of three bricks has 1/3 of the data.  This 
is the "Distribute" mode of operating gluster.  From what you describe, you 
are not interested in that. You are interested in the "Replicate" mode.  The 
numbering was adopted because gluster makes it possible to use various 
combinations of Distribute and Replicate at the same time, and this numbering 
describes the arrangement very succinctly.

2. As to how to you add a brick to your volume, so you can add a server to 
your system.  You use the 'gluster volume add-brick' command.  You are 
telling gluster: You are currently a two-brick system and I want to make you 
a 3-brick system (each brick being on a different server) by adding 
'server3'.  Typing 'gluster volume add-brick help' returns

gluster volume add-brick <VOLNAME> [<stripe|replica> <COUNT>] <NEW-BRICK> ... [force]

Assuming that the brick is ready and freshly formatted with xfs, the command 
ends up looking something like:

gluster volume add-brick upload replica 3 server3:/bricks/upload/upload

This will create the gluster structure on the brick, and will start the 
process of replicating the gluster data onto the brick.  The gluster mount 
should be usable at that point, pulling the data from other bricks if it has 
not been replicated onto the local brick yet.

If this server has been in service before, it is probably best to reformat 
the brick before putting it into service.  Otherwise gluster will see 
remnants of the past use on the brick, and not want to use it, assuming there 
may be valuable information already stored there.

CAUTION: Unless they have fixed this issue, the replication process can bring 
a server to its knees until it is fully replicated.

3. To remove a server from your system, the process is basically reversed.  
If you want to go from 5 servers to 4, you issue the command:

gluster volume remove-brick upload replica 4 server5:/bricks/upload/upload

This tells gluster to drop the brick on server5 from the volume, and 
reassures gluster that you know that you will now have a replicated volume 
with 4 replicas on it.

Hope this helps.

Ted Miller
Elkhart, IN

P.S. I am not reading the list regularly, so if you need more info, best to 
copy me when sending to the list.  Just saw your request and said "I've done 
that, I can share some information."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140910/39a2af10/attachment.html>