[Gluster-users] Gluster communication via TLS client problem

Mon Feb 12 19:40:31 UTC 2024

Am 31.01.24 um 18:11 schrieb Aravinda:
> Sure let us know if it works with re setup.
> 
> 
> 
>>   One reason I don't want to use docker is, I need to install it on VMs in
> 
>> the future. If everything is working, I will put the hole setup into
> 
>> ansible. But first step first. And the first step is, geting TLS running.
> 
> 
> 
> Kadalu Binnacle also supports ssh. Just change the command_mode to ssh, remove/replace the docker commands.
> 
> 
> 
> Refer below doc for Binnacle SSH config options.
> 
> 
> 
> https://github.com/kadalu/binnacle?tab=readme-ov-file#run-a-command-using-ssh
> 
> 
> 
> Blog post: https://aravindavk.in/blog/gluster-volume-setup-binnacle/
> 
> 
> 
> --
> 
> Thanks and Regards
> 
> Aravinda
> 
> Kadalu Technologies
> 
> 
> 
> 
> 
> 
> 
> 
> ---- On Wed, 31 Jan 2024 22:01:24 +0530 Stefan Kania <stefan at kania-online.de> wrote ---
> 
> 
> 
> Hi Aravinda,
>   
> im not so into Docker :-( So I just looked at your commands and I saw
> that you did exacly the same I did. I even removed all TLS configuration
> and all certificates and then copied your commands (as far as it was
> possible) to create the certificates and compared it with my commands.
> Everything is exacly the same. But my setup is not working :-(. At this
> point I think it's not a Gluster problem but a problem of my Debian
> installation and configuration. So I will start from scratch and do it
> all again.
> One reason I don't want to use docker is, I need to install it on VMs in
> the future. If everything is working, I will put the hole setup into
> ansible. But first step first. And the first step is, geting TLS running.
>   
> Stefan
>   
> Am 31.01.24 um 09:22 schrieb Aravinda:
>> Hi Stefan,
>>
>>
>>
>> I reproduced this in our lab and it is working without any issues.
>>
>>
>>
>> Lab setup: Debian 12 and Gluster version 10.5
>>
>> Three servers and one client: c01.gluster, c02.gluster, c03.gluster and cluster-client.gluster
>>
>>
>>
>> I used RSA key length as 4096 instead of 2048 and used the below volume option
>>
>>
>>
>> gluster volume set gv1 ssl.cipher-list 'HIGH:!SSLv2'
>>
>>
>>
>> I used Kadalu Binnacle (https://github.com/kadalu/binnacle) to setup container based three nodes cluster. The details and the test file are available in the below Github repository.
>>
>>
>>
>> https://github.com/aravindavk/gluster-tests?tab=readme-ov-file#gluster-tls-tests
>>
>>
>>
>> -- 
>> Aravinda
>>
>> Kadalu Technologies
>>
>>
>>
>>
>>
>>
>>
>>
>> ---- On Mon, 29 Jan 2024 22:10:50 +0530 Stefan Kania <mailto:stefan at kania-online.de> wrote ---
>>
>>
>>
>> Hi Strahil, hi Aravinda
>>
>> Am 28.01.24 um 23:03 schrieb Strahil Nikolov:
>>> You didn't specify correctly the IP in the SANS but I'm not sure if that's the root cause.
>>> In the SANs section Specify all hosts + their IPs: IP.1=1.2.3.4IP.2=2.3.4.5DNS.1=c01.glusterDNS.2=c02.gluster
>>
>> That's what I did now:
>>
>> I took the commands from the article you recommended and added all the
>> alternative names and IPs into the certificate:
>> -------------
>> openssl req -new -x509 -key /etc/ssl/glusterfs.key -subj "/CN=`hostname
>> -f`" -addext "subjectAltName =
>> IP:192.168.57.41,IP:192.168.57.42,IP:192.168.57.43,IP:192.168.57.51,DNS:c01.gluster,DNS:c02.gluster,DNS:c03.gluster,DNS:cluster-client.gluster"
>> -out /etc/ssl/glusterfs.pem
>> -------------
>> Stille getting on the server:
>> -------------
>> [2024-01-29 16:32:08.877499 +0000] I
>> [socket.c:4288:ssl_setup_connection_params] 0-socket.management: SSL
>> support for MGMT is ENABLED IO path is ENABLED certificate depth is 1
>> for peer 192.168.57.51:49151
>> [2024-01-29 16:32:08.881842 +0000] E [socket.c:224:ssl_dump_error_stack]
>> 0-socket.management:   error:0A00010B:SSL routines::wrong version number
>>
>> -------------
>>
>> And on the client:
>> -------------
>> [2024-01-29 16:32:08.865731 +0000] I [MSGID: 100030]
>> [glusterfsd.c:2767:main] 0-/usr/sbin/glusterfs: Started running version
>> [{arg=/usr/sbin/glusterfs}, {version=10.5},
>> {cmdlinestr=/usr/sbin/glusterfs --process-name fuse
>> --volfile-server=c02.gluster --volfile-id=/gv1 /mnt}]
>> [2024-01-29 16:32:08.870129 +0000] I [glusterfsd.c:2447:daemonize]
>> 0-glusterfs: Pid of current running process is 664
>> [2024-01-29 16:32:08.880528 +0000] I [MSGID: 101190]
>> [event-epoll.c:667:event_dispatch_epoll_worker] 0-epoll: Started thread
>> with index [{index=1}]
>> [2024-01-29 16:32:08.880935 +0000] I [MSGID: 101190]
>> [event-epoll.c:667:event_dispatch_epoll_worker] 0-epoll: Started thread
>> with index [{index=0}]
>> [2024-01-29 16:32:08.885755 +0000] I
>> [glusterfsd-mgmt.c:2681:mgmt_rpc_notify] 0-glusterfsd-mgmt: disconnected
>> from remote-host: c02.gluster
>> [2024-01-29 16:32:08.885879 +0000] I
>> [glusterfsd-mgmt.c:2720:mgmt_rpc_notify] 0-glusterfsd-mgmt: Exhausted
>> all volfile servers
>> [2024-01-29 16:32:08.887116 +0000] W
>> [glusterfsd.c:1458:cleanup_and_exit]
>> (-->/lib/x86_64-linux-gnu/libgfrpc.so.0(+0xfa35) [0x7fd18d185a35]
>> -->/usr/sbin/glusterfs(+0x14769) [0x55d4f8d5d769]
>> -->/usr/sbin/glusterfs(cleanup_and_exit+0x57) [0x55d4f8d54447] ) 0-:
>> received signum (1), shutting down
>> [2024-01-29 16:32:08.887209 +0000] I [fuse-bridge.c:7065:fini] 0-fuse:
>> Unmounting '/mnt'.
>> [2024-01-29 16:32:08.889719 +0000] I [fuse-bridge.c:7069:fini] 0-fuse:
>> Closing fuse connection to '/mnt'.
>> [2024-01-29 16:32:08.889909 +0000] W
>> [glusterfsd.c:1458:cleanup_and_exit]
>> (-->/lib/x86_64-linux-gnu/libc.so.6(+0x89044) [0x7fd18d00a044]
>> -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xc5) [0x55d4f8d5be05]
>> -->/usr/sbin/glusterfs(cleanup_and_exit+0x57) [0x55d4f8d54447] ) 0-:
>> received signum (15), shutting down
>> -------------
>>
>> executing the connect command on the client:
>> --------------
>> openssl s_client -showcerts -connect c02.gluster:24007
>> --------------
>>
>> shows on the sever:
>> --------------
>> [2024-01-29 16:37:08.747123 +0000] I
>> [socket.c:4288:ssl_setup_connection_params] 0-socket.management: SSL
>> support for MGMT is ENABLED IO path is ENABLED certificate depth is 1
>> for peer 192.168.57.51:58060
>> [2024-01-29 16:37:08.767715 +0000] E
>> [socket.c:426:ssl_setup_connection_postfix] 0-socket.management: SSL
>> connect error (client: 192.168.57.51:58060) (server: 192.168.57.42:24007)
>> --------------
>>
>> So still the same, no changes :-(
>>
>> Stefan
>   

After restart from the beginning including setting up Debian 12 now 
everything is working. Thank you for your help.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3477 bytes
Desc: Kryptografische S/MIME-Signatur
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20240212/c3a61682/attachment.p7s>