[Gluster-devel] Glusterd2 - Some anticipated changes to glusterfs source

[Niels de Vos]

On Wed, Aug 02, 2017 at 05:03:35PM +0530, Prashanth Pai wrote:
> Hi all,
> 
> The ongoing work on glusterd2 necessitates following non-breaking and
> non-exhaustive list of changes to glusterfs source code:
> 
> Port management
> - Remove hard-coding of glusterd's port as 24007 in clients and elsewhere.
>   Glusterd2 can be configured to listen to clients on any port (still
> defaults to
>   24007 though)
> - Let the bricks and daemons choose any available port and if needed report
>   the port used to glusterd during the "sign in" process. Prasanna has a
> patch
>   to do this.
> - Glusterd <--> brick (or any other local daemon) communication should
>   always happen over Unix Domain Socket. Currently glusterd and brick
>   process communicates over UDS and also port 24007. This will allow us
>   to set better authentication and rules for port 24007 as it shall only be
> used
>   by clients.

I prefer this last point to be configurable. At least for debugging we
should be able to capture network traces and display the communication
in Wireshark. Defaulting to UNIX Domain Sockets is fine though.


> Changes to xlator options
> - Xlator authors do not have to modify glusterd2 code to expose new xlator
>   options. IOW, glusterd2 will not contain the "glusterd_volopt_map" table.
>   Most of its fields will be moved to the xlator itself. Glusterd2 can load
>   xlator's shared object and read it's volume_options table. This also means
>   xlators have to adhere to some naming conventions for options.
> - Add following additional fields (names are indicative) to volume_option_t:
>     - Tag: This is to enable users to list only options having a certain
> tag.
>              IOW, it allows us to filter "volume set help" like output.
>              Example of tags: debug, perf, network etc.
>     - Opversion: The minimum (or a range) op-version required by the xlator.
>     - Configurable: A bool to indicate whether this option is
> user-configurable.
>                           This may also be clubbed with DOC/NO_DOC
> functionality.

This is something I have been thinking about to do as well. libgfapi
users would like to list all the valid options before mounting (and
receiving the .vol file) is done. Similar to how many mount options are
set over FUSE, the options should be available through libgfapi.
Hardcoding the options is just wrong, inspecting the available xlators
(.so files) seems to make more sense. Each option would have to describe
if it can be client-side so that we can apply some resonable filters by
default.

A GitHub Issue with this feature request is at
https://github.com/gluster/glusterfs/issues/263. I appreciate additional
comments and ideas about it :-)


> - Xlators like AFR, changelog require non-static information such as brick
> path
>   to be present in it's options in the volfile. Currently, xlator authors
> have
>   to modify glusterd code to get it.
>   This can rather be indicated by the xlator itself using
> templates/placehoders.
>   For example, "changelog-dir" can be set in xlator's option as as
>   <<brick-path>>/.glusterfs/changelogs and then glusterd2 will ensure to
> replace
>   <<brick-path>> with actual path during volfile generation.

I suggest to stick with whatever is a common syntax for other
configuration files that uses placeholders. Maybe just {variable} or
$VARIABLE, the <<variable>> looks a bit awkward.


> We'd like to hear your thoughts, suggestions and comments to these proposed
> changes.

Thanks for sharing!
Niels