[Gluster-users] Question on replicated volumes with bricks on the same server

[Antonio Messina]

On Thu, Feb 13, 2014 at 12:54 PM, Vijay Bellur <vbellur at redhat.com> wrote:
> On 02/12/2014 11:10 PM, Antonio Messina wrote:
>>
>> On Wed, Feb 12, 2014 at 11:21 AM, Vijay Bellur <vbellur at redhat.com> wrote:
>>>
>>> On 02/11/2014 03:21 PM, Antonio Messina wrote:
>>>>
>>>> So, my guess is that if I create a "replica N" replicated+distributed
>>>> volumes using the bricks:
>>>>
>>>>     gluster-1:/srv/gluster
>>>>     ...
>>>>     gluster-[N*M]:/srv/gluster
>>>>
>>>> gluster internally creates a distributed volumes made of the following
>>>> replicated "volumes":
>>>>
>>>>     replicated volume 1: gluster-[1..N]:/srv/gluster
>>>>     replicated volume 2: gluster-[N+1..2N]:/srv/gluster
>>>>     ...
>>>>     replicated volume M: gluster-[N*(M-1)+1..N*M]:/srv/gluster
>>>>
>>>> Is that correct or there is a more complex algorithm involved?
>>>>
>>>
>>> The interpretation is correct. The way replica sets are chosen is related
>>> to
>>> the order in which bricks are defined at the time of volume creation.
>>
>>
>> Thank you for your clarification.
>>
>> However, I am also trying to play with the vol files. It's not clear to me
>> yet
>> *when* and *how* these files are read, so I don't know if I am doing
>> something very stupid... however, here is my question:
>>
>> I create a distributed+replicated volume with 4 bricks with:
>>
>>      gluster volume create default replica 2
>> gluster-data00{1,2,3,4}:/srv/gluster/brick force
>>
>> mount the filesystem, created multiple files on it and I have verified
>> that the replicas are stored as we discussed before.
>>
>> The `default-fuse.vol` file looks like:
>>
>>      [...]
>>      volume default-replicate-0
>>          type cluster/replicate
>>          subvolumes default-client-0 default-client-1
>>      end-volume
>>
>>      volume default-replicate-1
>>          type cluster/replicate
>>          subvolumes default-client-2 default-client-3
>>      end-volume
>>
>>      volume default-dht
>>          type cluster/distribute
>>          subvolumes default-replicate-0 default-replicate-1
>>      end-volume
>>      [...]
>>
>> Then I have stopped the volume and modified the vol file, in order to
>> swap two servers (default-client-3 and default-client-1):
>

Thank you Vijay for your explanations!

> Modifying a volume file manually is not a recommended step and the mechanism
> employed here is quite disruptive to GlusterFS and your data :).

I was just trying to understand how GlusterFS work, playing with a few
VMs, so I don't care if I lose data. But it's good to know that this
is *NOT* the way to do things :)

> A lot of GlusterFS translators store state information in the form of
> extended attributes. Some of the keys used in these extended attributes are
> a function of the subvolume names and with this manual swap, it can cause
> self-heals to happen in a random fashion. Re-using a brick in a different
> form is not normally recommended and a fair degree of state cleanup has to
> be done before it can be used.

Understood.

>> Am I missing some basic step?
>
>
> If you are testing functionality by altering the volume file manually, it
> would be advisable to change something more simpler - like altering a
> translator option to see the change being operational rather than changing
> the volume topology manually. volume topology modifications are recommended
> to be performed from the gluster CLI.

as I said, I am just playing around in order to understand how does it
work (and not work). We are investigating the possibility to use
gluster for a mid-size installation, we need to do our homework :)

.a.

-- 
antonio.s.messina at gmail.com
antonio.messina at uzh.ch                     +41 (0)44 635 42 22
GC3: Grid Computing Competence Center      http://www.gc3.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland