[Gluster-devel] sub-directory geo-replication, snapshot features

[Kaushal M]

On Wed, Mar 9, 2016 at 6:24 PM, Shyam <srangana at redhat.com> wrote:
> On 03/09/2016 02:25 AM, Pranith Kumar Karampuri wrote:
>>
>>
>>
>> On 03/09/2016 10:40 AM, Kaushal M wrote:
>>>
>>> On Tue, Mar 8, 2016 at 11:58 PM, Atin Mukherjee <amukherj at redhat.com>
>>> wrote:
>>>>
>>>>
>>>> On 03/08/2016 07:32 PM, Pranith Kumar Karampuri wrote:
>>>>>
>>>>> hi,
>>>>>           Late last week I sent a solution for how to achieve
>>>>> subdirectory-mount support with access-controls
>>>>>
>>>>> (http://www.gluster.org/pipermail/gluster-devel/2016-March/048537.html).
>>>>>
>>>>> What follows here is a short description of how other features of
>>>>> gluster volumes are implemented for sub-directories.
>>>>>
>>>>> Please note that the sub-directories are not allowed to be accessed by
>>>>> normal mounts i.e. top-level volume mounts. All access to the
>>>>> sub-directories goes only through sub-directory mounts.
>>>>
>>>> Is this acceptable? If I have a,b,c sub directories in the volume and if
>>>> I mount the same volume in /mnt then do you mean to say I won't be able
>>>> to access /mnt/a or /mnt/b and I can only access them using sub
>>>> directory mounts? Or you are talking about some specific case here?
>>>>>
>>>>> 1) Geo-replication:
>>>>> The direction in which we are going is to allow geo-replicating just
>>>>> some sub-directories and not all of the volume based on options. When
>>>>> these options are set, server xlators populate extra information in the
>>>>> frames/xdata to write changelog for the fops coming from their
>>>>> sub-directory mounts. changelog xlator on seeing this will only
>>>>> geo-replicate the files/directories that are in the changelog. Thus
>>>>> only
>>>>> the sub-directories are geo-replicated. There is also a suggestion from
>>>>> Vijay and Aravinda to have separate domains for operations inside
>>>>> sub-directories for changelogs.
>>>>>
>>>>> 2) Sub-directory snapshots using lvm
>>>>> Every time a sub-directory needs to be created, Our idea is that the
>>>>> admin needs to execute subvolume creation command which creates a mount
>>>>> to an empty snapshot at the given sub-directory name. All these
>>>>> directories can be modified in parallel and we can take individual
>>>>> snapshots of each of the directories. We will be providing a detailed
>>>>> list commands to do the same once they are fleshed out. At the moment
>>>>> these are the directions we are going to increase granularity from
>>>>> volume to subdirectory for the main features.
>>>
>>> We use hardlinks to the `.glusterfs` directory on bricks. So wouldn't
>>> having multiple filesystems inside a brick break the brick?
>>
>>
>> You are right. I think we will have to do full separation, where it will
>> be more like multiple tenants :-/.
>
>
> Ah! I had a half baked response to this yesterday, asking if subdirs were
> their own FS instances and LVM instances etc. looks like that was the case.
>
> Anyway, apart form the .glusterfs this idea merits some thinking, like a
> virtual volume that chains up separate mounts on the local FS, consider it a
> virtual namespace that links subvolumes together. So that we do not need to
> create separate volumes and have that heavy weight process in place, at the
> same time provide snapshots as well.

If we are thinking about subdirs as virtual volumes and want to avoid
a process per volume, why not just consider brick-multiplexing and
proper full fledged volumes.
We already have multiplexing on the agenda for 4.0, and it would seem
easier to just leverage the facilities provided by that to get similar
behavior, instead of attempting to add-in sub-directory support for
all sort of operations/features.

One of the benefits of supporting sub directory mounts is that it
would make it easier to create and manage multiple shares for many
tenants. Trying to re-implement all volume operations for
sub-directories only makes it harder for us and makes the solution
really complex.

>
>
>>
>>>
>>> Also, I'd prefer if sub-directory mounts and sub-directory snapshots
>>> remained separate, and not tied with each other. This mail gives the
>>> feeling that they will be tied together.
>>
>>
>> With the above point, I don't think it will be.
>>
>>>
>>>>> Pranith
>>>>> _______________________________________________
>>>>> Gluster-devel mailing list
>>>>> Gluster-devel at gluster.org
>>>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>>
>>>> _______________________________________________
>>>> Gluster-devel mailing list
>>>> Gluster-devel at gluster.org
>>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel