[Gluster-devel] Glusterd2 - Some anticipated changes to glusterfs source

[Kaushal M]

On Thu, Aug 3, 2017 at 2:14 AM, Niels de Vos <ndevos at redhat.com> wrote:
> On Wed, Aug 02, 2017 at 05:03:35PM +0530, Prashanth Pai wrote:
>> Hi all,
>>
>> The ongoing work on glusterd2 necessitates following non-breaking and
>> non-exhaustive list of changes to glusterfs source code:
>>
>> Port management
>> - Remove hard-coding of glusterd's port as 24007 in clients and elsewhere.
>>   Glusterd2 can be configured to listen to clients on any port (still
>> defaults to
>>   24007 though)
>> - Let the bricks and daemons choose any available port and if needed report
>>   the port used to glusterd during the "sign in" process. Prasanna has a
>> patch
>>   to do this.
>> - Glusterd <--> brick (or any other local daemon) communication should
>>   always happen over Unix Domain Socket. Currently glusterd and brick
>>   process communicates over UDS and also port 24007. This will allow us
>>   to set better authentication and rules for port 24007 as it shall only be
>> used
>>   by clients.
>
> I prefer this last point to be configurable. At least for debugging we
> should be able to capture network traces and display the communication
> in Wireshark. Defaulting to UNIX Domain Sockets is fine though.

This is the communication between GD2 and bricks, of which there is
not a lot happening, and not much to capture.
But I agree, it will be nice to have this configurable.

>
>
>> Changes to xlator options
>> - Xlator authors do not have to modify glusterd2 code to expose new xlator
>>   options. IOW, glusterd2 will not contain the "glusterd_volopt_map" table.
>>   Most of its fields will be moved to the xlator itself. Glusterd2 can load
>>   xlator's shared object and read it's volume_options table. This also means
>>   xlators have to adhere to some naming conventions for options.
>> - Add following additional fields (names are indicative) to volume_option_t:
>>     - Tag: This is to enable users to list only options having a certain
>> tag.
>>              IOW, it allows us to filter "volume set help" like output.
>>              Example of tags: debug, perf, network etc.
>>     - Opversion: The minimum (or a range) op-version required by the xlator.
>>     - Configurable: A bool to indicate whether this option is
>> user-configurable.
>>                           This may also be clubbed with DOC/NO_DOC
>> functionality.
>
> This is something I have been thinking about to do as well. libgfapi
> users would like to list all the valid options before mounting (and
> receiving the .vol file) is done. Similar to how many mount options are
> set over FUSE, the options should be available through libgfapi.
> Hardcoding the options is just wrong, inspecting the available xlators
> (.so files) seems to make more sense. Each option would have to describe
> if it can be client-side so that we can apply some resonable filters by
> default.
>

Looks like we'd missed this. All the fields available in the vol opt
map will move to xlator option tables, including the client flag.

> A GitHub Issue with this feature request is at
> https://github.com/gluster/glusterfs/issues/263. I appreciate additional
> comments and ideas about it :-)
>

We need to open an issue for our requested changes are well, which
will be a superset of this request. We'll make sure to mention this
feature request in it.
Or we could use a single issue as a tracker for all the xlator option
changes, in which case I'd prefer we update the existing issue.

>
>> - Xlators like AFR, changelog require non-static information such as brick
>> path
>>   to be present in it's options in the volfile. Currently, xlator authors
>> have
>>   to modify glusterd code to get it.
>>   This can rather be indicated by the xlator itself using
>> templates/placehoders.
>>   For example, "changelog-dir" can be set in xlator's option as as
>>   <<brick-path>>/.glusterfs/changelogs and then glusterd2 will ensure to
>> replace
>>   <<brick-path>> with actual path during volfile generation.
>
> I suggest to stick with whatever is a common syntax for other
> configuration files that uses placeholders. Maybe just {variable} or
> $VARIABLE, the <<variable>> looks a bit awkward.

The exact syntax for these variables hasn't been decided yet. But I'm
leaning towards '{{ variable }}' used in the Go template package,
which is what we'll mostly end up using to implement this
functionality.

>
>
>> We'd like to hear your thoughts, suggestions and comments to these proposed
>> changes.
>
> Thanks for sharing!
> Niels
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel