<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Aug 3, 2017 at 12:56 PM, Kaushal M <span dir="ltr"><<a href="mailto:kshlmster@gmail.com" target="_blank">kshlmster@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Thu, Aug 3, 2017 at 2:14 AM, Niels de Vos <<a href="mailto:ndevos@redhat.com">ndevos@redhat.com</a>> wrote:<br>
</span><span class="">> On Wed, Aug 02, 2017 at 05:03:35PM +0530, Prashanth Pai wrote:<br>
>> Hi all,<br>
>><br>
>> The ongoing work on glusterd2 necessitates following non-breaking and<br>
>> non-exhaustive list of changes to glusterfs source code:<br>
>><br>
>> Port management<br>
>> - Remove hard-coding of glusterd's port as 24007 in clients and elsewhere.<br>
>> Glusterd2 can be configured to listen to clients on any port (still<br>
>> defaults to<br>
>> 24007 though)<br>
>> - Let the bricks and daemons choose any available port and if needed report<br>
>> the port used to glusterd during the "sign in" process. Prasanna has a<br>
>> patch<br>
>> to do this.<br>
>> - Glusterd <--> brick (or any other local daemon) communication should<br>
>> always happen over Unix Domain Socket. Currently glusterd and brick<br>
>> process communicates over UDS and also port 24007. This will allow us<br>
>> to set better authentication and rules for port 24007 as it shall only be<br>
>> used<br>
>> by clients.<br>
><br>
> I prefer this last point to be configurable. At least for debugging we<br>
> should be able to capture network traces and display the communication<br>
> in Wireshark. Defaulting to UNIX Domain Sockets is fine though.<br>
<br>
</span>This is the communication between GD2 and bricks, of which there is<br>
not a lot happening, and not much to capture.<br>
But I agree, it will be nice to have this configurable.<br>
<span class=""><br></span></blockquote><div><br></div><div>Could glusterd start attempting port binding at 24007 and progress on to higher port numbers until successful and register the bound port number with rpcbind ? This way the setup will be auto-configurable and admins need not scratch their heads to decide upon one port number. Gluster clients could always talk to rpcbind on the nodes to get glusterd service port whenever a reconnect is required.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
><br>
><br>
>> Changes to xlator options<br>
>> - Xlator authors do not have to modify glusterd2 code to expose new xlator<br>
>> options. IOW, glusterd2 will not contain the "glusterd_volopt_map" table.<br>
>> Most of its fields will be moved to the xlator itself. Glusterd2 can load<br>
>> xlator's shared object and read it's volume_options table. This also means<br>
>> xlators have to adhere to some naming conventions for options.<br>
>> - Add following additional fields (names are indicative) to volume_option_t:<br>
>> - Tag: This is to enable users to list only options having a certain<br>
>> tag.<br>
>> IOW, it allows us to filter "volume set help" like output.<br>
>> Example of tags: debug, perf, network etc.<br>
>> - Opversion: The minimum (or a range) op-version required by the xlator.<br>
>> - Configurable: A bool to indicate whether this option is<br>
>> user-configurable.<br>
>> This may also be clubbed with DOC/NO_DOC<br>
>> functionality.<br>
><br>
> This is something I have been thinking about to do as well. libgfapi<br>
> users would like to list all the valid options before mounting (and<br>
> receiving the .vol file) is done. Similar to how many mount options are<br>
> set over FUSE, the options should be available through libgfapi.<br>
> Hardcoding the options is just wrong, inspecting the available xlators<br>
> (.so files) seems to make more sense. Each option would have to describe<br>
> if it can be client-side so that we can apply some resonable filters by<br>
> default.<br>
><br>
<br>
</span>Looks like we'd missed this. All the fields available in the vol opt<br>
map will move to xlator option tables, including the client flag.<br>
<span class=""><br>
> A GitHub Issue with this feature request is at<br>
> <a href="https://github.com/gluster/glusterfs/issues/263" rel="noreferrer" target="_blank">https://github.com/gluster/<wbr>glusterfs/issues/263</a>. I appreciate additional<br>
> comments and ideas about it :-)<br>
><br>
<br>
</span>We need to open an issue for our requested changes are well, which<br>
will be a superset of this request. We'll make sure to mention this<br>
feature request in it.<br>
Or we could use a single issue as a tracker for all the xlator option<br>
changes, in which case I'd prefer we update the existing issue.<br>
<span class=""><br>
><br>
>> - Xlators like AFR, changelog require non-static information such as brick<br>
>> path<br>
>> to be present in it's options in the volfile. Currently, xlator authors<br>
>> have<br>
>> to modify glusterd code to get it.<br>
>> This can rather be indicated by the xlator itself using<br>
>> templates/placehoders.<br>
>> For example, "changelog-dir" can be set in xlator's option as as<br>
>> <<brick-path>>/.glusterfs/<wbr>changelogs and then glusterd2 will ensure to<br>
>> replace<br>
>> <<brick-path>> with actual path during volfile generation.<br>
><br>
> I suggest to stick with whatever is a common syntax for other<br>
> configuration files that uses placeholders. Maybe just {variable} or<br>
> $VARIABLE, the <<variable>> looks a bit awkward.<br>
<br>
</span>The exact syntax for these variables hasn't been decided yet. But I'm<br>
leaning towards '{{ variable }}' used in the Go template package,<br>
which is what we'll mostly end up using to implement this<br>
functionality.<br>
<div class="HOEnZb"><div class="h5"><br>
><br>
><br>
>> We'd like to hear your thoughts, suggestions and comments to these proposed<br>
>> changes.<br>
><br>
> Thanks for sharing!<br>
> Niels<br>
> ______________________________<wbr>_________________<br>
> Gluster-devel mailing list<br>
> <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
> <a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-devel</a><br>
______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-devel</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Milind<br><br></div></div></div></div>
</div></div>