[Gluster-devel] Glusterd2 - Some anticipated changes to glusterfs source

Thu Aug 3 08:42:30 UTC 2017

On Thu, Aug 3, 2017 at 12:56 PM, Kaushal M <kshlmster at gmail.com> wrote:

> On Thu, Aug 3, 2017 at 2:14 AM, Niels de Vos <ndevos at redhat.com> wrote:
> > On Wed, Aug 02, 2017 at 05:03:35PM +0530, Prashanth Pai wrote:
> >> Hi all,
> >>
> >> The ongoing work on glusterd2 necessitates following non-breaking and
> >> non-exhaustive list of changes to glusterfs source code:
> >>
> >> Port management
> >> - Remove hard-coding of glusterd's port as 24007 in clients and
> elsewhere.
> >>   Glusterd2 can be configured to listen to clients on any port (still
> >> defaults to
> >>   24007 though)
> >> - Let the bricks and daemons choose any available port and if needed
> report
> >>   the port used to glusterd during the "sign in" process. Prasanna has a
> >> patch
> >>   to do this.
> >> - Glusterd <--> brick (or any other local daemon) communication should
> >>   always happen over Unix Domain Socket. Currently glusterd and brick
> >>   process communicates over UDS and also port 24007. This will allow us
> >>   to set better authentication and rules for port 24007 as it shall
> only be
> >> used
> >>   by clients.
> >
> > I prefer this last point to be configurable. At least for debugging we
> > should be able to capture network traces and display the communication
> > in Wireshark. Defaulting to UNIX Domain Sockets is fine though.
>
> This is the communication between GD2 and bricks, of which there is
> not a lot happening, and not much to capture.
> But I agree, it will be nice to have this configurable.
>
>
Could glusterd start attempting port binding at 24007 and progress on to
higher port numbers until successful and register the bound port number
with rpcbind ? This way the setup will be auto-configurable and admins need
not scratch their heads to decide upon one port number. Gluster clients
could always talk to rpcbind on the nodes to get glusterd service port
whenever a reconnect is required.

> >
> >
> >> Changes to xlator options
> >> - Xlator authors do not have to modify glusterd2 code to expose new
> xlator
> >>   options. IOW, glusterd2 will not contain the "glusterd_volopt_map"
> table.
> >>   Most of its fields will be moved to the xlator itself. Glusterd2 can
> load
> >>   xlator's shared object and read it's volume_options table. This also
> means
> >>   xlators have to adhere to some naming conventions for options.
> >> - Add following additional fields (names are indicative) to
> volume_option_t:
> >>     - Tag: This is to enable users to list only options having a certain
> >> tag.
> >>              IOW, it allows us to filter "volume set help" like output.
> >>              Example of tags: debug, perf, network etc.
> >>     - Opversion: The minimum (or a range) op-version required by the
> xlator.
> >>     - Configurable: A bool to indicate whether this option is
> >> user-configurable.
> >>                           This may also be clubbed with DOC/NO_DOC
> >> functionality.
> >
> > This is something I have been thinking about to do as well. libgfapi
> > users would like to list all the valid options before mounting (and
> > receiving the .vol file) is done. Similar to how many mount options are
> > set over FUSE, the options should be available through libgfapi.
> > Hardcoding the options is just wrong, inspecting the available xlators
> > (.so files) seems to make more sense. Each option would have to describe
> > if it can be client-side so that we can apply some resonable filters by
> > default.
> >
>
> Looks like we'd missed this. All the fields available in the vol opt
> map will move to xlator option tables, including the client flag.
>
> > A GitHub Issue with this feature request is at
> > https://github.com/gluster/glusterfs/issues/263. I appreciate additional
> > comments and ideas about it :-)
> >
>
> We need to open an issue for our requested changes are well, which
> will be a superset of this request. We'll make sure to mention this
> feature request in it.
> Or we could use a single issue as a tracker for all the xlator option
> changes, in which case I'd prefer we update the existing issue.
>
> >
> >> - Xlators like AFR, changelog require non-static information such as
> brick
> >> path
> >>   to be present in it's options in the volfile. Currently, xlator
> authors
> >> have
> >>   to modify glusterd code to get it.
> >>   This can rather be indicated by the xlator itself using
> >> templates/placehoders.
> >>   For example, "changelog-dir" can be set in xlator's option as as
> >>   <<brick-path>>/.glusterfs/changelogs and then glusterd2 will ensure
> to
> >> replace
> >>   <<brick-path>> with actual path during volfile generation.
> >
> > I suggest to stick with whatever is a common syntax for other
> > configuration files that uses placeholders. Maybe just {variable} or
> > $VARIABLE, the <<variable>> looks a bit awkward.
>
> The exact syntax for these variables hasn't been decided yet. But I'm
> leaning towards '{{ variable }}' used in the Go template package,
> which is what we'll mostly end up using to implement this
> functionality.
>
> >
> >
> >> We'd like to hear your thoughts, suggestions and comments to these
> proposed
> >> changes.
> >
> > Thanks for sharing!
> > Niels
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > http://lists.gluster.org/mailman/listinfo/gluster-devel
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>

-- 
Milind
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20170803/b6e87ae1/attachment.html>