[Gluster-users] Glusterfs fuse uses system ports to mount remote bricks

[Canh Ngo]

Thank you for all your suggestions!

I choose to reserve some system ports for our services as it has less work
to do comparing to change Gluster cluster config: need to restart Gluster
node one by one and remount volumes of all clients.

Kind regards,
Canh Ngo.

On Wed, Jun 13, 2018 at 8:13 PM, Raghavendra Talur <rtalur at redhat.com>
wrote:

> Try
>
> gluster volume set VOLNAME client.bind-insecure on
>
> and remount clients. If servers refuse connection, you might also have to
> set server.allow-insecure to on.
>
>
> On Wed, Jun 13, 2018 at 9:41 AM, Milind Changire <mchangir at redhat.com>
> wrote:
>
>> On Wed, Jun 13, 2018 at 6:12 PM, Canh Ngo <canhnt at gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> We run a storage cluster using GlusterFS v3.10.12 on CentOS7. Clients
>>> (CentOS) are using glusterfs 3.8.4.
>>>
>>> We notice when clients mounts bricks of a volume, sometimes glusterfs
>>> uses system ports (i.e. in port range 0-1024) to connect to remote
>>> glusterfsd port. e.g:
>>>
>>> Server:
>>> tcp        0      0 10.165.210.169:49161    10.165.210.51:850
>>> ESTABLISHED 32760/glusterfsd
>>>
>>> Client:
>>> tcp        0      0 10.165.210.51:850       10.165.210.169:49161
>>> ESTABLISHED 25483/glusterfs
>>>
>>> Thus, sometimes glusterfs occupies our system ports (e.g. 995, 179, 443,
>>> etc), that causes other services cannot start.
>>>
>>> Is is a bug or an expected behavior? I expect glusterfs should use IANA
>>> private ports rather than system ports. Do you know if we can configure
>>> glusterfs to use a specific port range?
>>>
>>> ​Thanks,
>>> Canh Ngo.
>>>
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>
>>
>>
>>
>> Here's some info on the kernel (sysctl) tunables that you could tweak:
>> /proc/sys/net/ipv4/ip_local_port_range
>> /proc/sys/net/ipv4/ip_local_reserved_ports
>>
>> Here's what networking/ip-sysctl.txt from the kernel documentation
>> directory says:
>> ip_local_port_range - 2 INTEGERS
>>         Defines the local port range that is used by TCP and UDP to
>>         choose the local port. The first number is the first, the
>>         second the last local port number.
>>         If possible, it is better these numbers have different parity.
>>         (one even and one odd values)
>>         The default values are 32768 and 60999 respectively.
>>
>> ip_local_reserved_ports - list of comma separated ranges
>>         Specify the ports which are reserved for known third-party
>>         applications. These ports will not be used by automatic port
>>         assignments (e.g. when calling connect() or bind() with port
>>         number 0). Explicit port allocation behavior is unchanged.
>>
>>         The format used for both input and output is a comma separated
>>         list of ranges (e.g. "1,2-4,10-10" for ports 1, 2, 3, 4 and
>>         10). Writing to the file will clear all previously reserved
>>         ports and update the current list with the one given in the
>>         input.
>>
>>         Note that ip_local_port_range and ip_local_reserved_ports
>>         settings are independent and both are considered by the kernel
>>         when determining which ports are available for automatic port
>>         assignments.
>>
>>         You can reserve ports which are not in the current
>>         ip_local_port_range, e.g.:
>>
>>         $ cat /proc/sys/net/ipv4/ip_local_port_range
>>         32000   60999
>>         $ cat /proc/sys/net/ipv4/ip_local_reserved_ports
>>         8080,9148
>>
>>         although this is redundant. However such a setting is useful
>>         if later the port range is changed to a value that will
>>         include the reserved ports.
>>
>>         Default: Empty
>>
>> You could check the values of these files on your system and configure
>> them accordingly. Gluster specifically looks at
>> /proc/sys/net/ipv4/ip_local_reserved_ports to avoid assigning values
>> from the reserved port range. Alternatively, you could configure the system
>> via /etc/sysctl.conf to persist the settings across reboots:
>>
>> net.ipv4.ip_local_reserved_ports
>> net.ipv4.ip_local_port_range
>>
>> Hope this helps.
>>
>> --
>> Milind
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180615/283d3137/attachment.html>