[Gluster-devel] expanding tiered volumes

[Vijay Bellur]

On 02/24/2015 02:40 AM, Dan Lambright wrote:
>
>
> ----- Original Message -----
>> From: "Vijay Bellur" <vbellur at redhat.com>
>> To: "Dan Lambright" <dlambrig at redhat.com>, "Gluster Devel" <gluster-devel at gluster.org>
>> Sent: Monday, February 23, 2015 5:26:58 AM
>> Subject: Re: [Gluster-devel] expanding tiered volumes
>>
>> On 02/16/2015 08:20 PM, Dan Lambright wrote:
>>
>>> One of the hard parts has been integrating this new volume type with
>>> existing features, such as adding and removing bricks.
>>> Allowing the user to add and remove bricks to both the hot and cold tiers
>>> is a challenge on multiple levels.
>>> The CLI commands would have to specify which tier to add/remove bricks to.
>>
>> Could you please detail the CLI being considered for defining a hot/cold
>> tier? I think that will be useful in understanding the problem better.
>>
>>> New phenomena could occur such as rebalancing operations happening
>>> simultanously on both the hot and cold tiers.
>>> In general rebalancing at the same time cache promotion/demotion is
>>> happening is a tricky problem, it requires some tweaks to DHT which are
>>> forthcoming.
>>> To manage the complexity and get this feature done "on time" (per Gluster's
>>> release schedule) we would like to only allow adding and removing bricks
>>> to the cold
>>> tier for the first version. I would like to see if there are any comments
>>> from the upstream community.
>>
>> We can have some flexibility if the core feature lands by end of this
>> week. Are there any thoughts on how either tier can be expanded or
>> shrunk? If it is not very intrusive and seems appropriate to be pulled
>> in as part of a bug fix, we can look at this beyond feature freeze and
>> before code freeze.
>
> Attaching a tier is akin to adding bricks:
>
> volume attach-tier <VOLNAME> [<replica> <COUNT>] <NEW-BRICK> ... [force]
>
> Adding bricks to the "cold tier" would be the same CLI as with normal volumes.
>
> volume add-brick <VOLNAME> [<stripe|replica> <COUNT>] <NEW-BRICK> ... [force]
>
> Adding bricks to the "hot tier" could be specified using a modified CLI, e.g. add the keyword "attached-tier".
>
> volume add-brick <VOLNAME> [<stripe|replica> <COUNT>] <NEW-BRICK> ... [force] [attached-tier]
>
> Technically, the ability to expand both the hot and cold tiers is feasible given enough development time.  An additional change to DHT is expected from the DHT group to allow rebalancing while promotion and demotion are happening. That fix is the first step.
>

Thanks for this, Dan!

I think we can add the ability to expand the hot tier after feature 
freeze. If we can create cli stubs and call this out as a known issue by 
feature freeze, we can add the capability to expand either tier soon 
after feature freeze. Providing the ability to expand both tiers is 
something that our users would like to see IMO.

-Vijay