[Gluster-devel] Seeing non-priv port + auth issue in the gluster brick log

Deepak C Shetty deepakcs at linux.vnet.ibm.com
Sat May 11 07:13:20 UTC 2013


Hi All,
    I am trying to mount a gluster volume from inside a VM (being used 
as a VDSM host) and seeing the below issue.
 From the VM, I am able to ping, telnet to the gluster host (no 
networking issues present)

*Client side*
============

[root at vdsm_tsm_int glusterfs]# mount -t glusterfs 9.121.60.166:dpkvol /mnt
Mount failed. Please check the log file for more details.

(Using IP or hostname in the mount cmdline doesn't change anything)

mnt.log
-------

[2013-05-11 06:38:54.199518] I [glusterfsd.c:1878:main] 
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 
3.4.0beta1 (/usr/sbin/glusterfs --volfile-id=dpkvol 
--volfile-server=9.121.60.166 /mnt)
[2013-05-11 06:38:54.203474] I [socket.c:3480:socket_init] 0-glusterfs: 
SSL support is NOT enabled
[2013-05-11 06:38:54.203652] I [socket.c:3495:socket_init] 0-glusterfs: 
using system polling thread
[2013-05-11 06:38:54.206592] W [common-utils.c:2330:gf_ports_reserved] 
0-glusterfs-socket:  is not a valid port identifier
[2013-05-11 06:38:55.253221] I [socket.c:3480:socket_init] 
0-dpkvol-client-0: SSL support is NOT enabled
[2013-05-11 06:38:55.253268] I [socket.c:3495:socket_init] 
0-dpkvol-client-0: using system polling thread
[2013-05-11 06:38:55.253300] I [client.c:2154:notify] 0-dpkvol-client-0: 
parent translators are ready, attempting connect on transport
[2013-05-11 06:38:55.255834] W [common-utils.c:2330:gf_ports_reserved] 
0-glusterfs-socket:  is not a valid port identifier
Given volfile:
+------------------------------------------------------------------------------+
   1: volume dpkvol-client-0
   2:     type protocol/client
   3:     option transport-type tcp
   4:     option remote-subvolume /home/dpkshetty/brick
   5:     option remote-host llmvm03
   6: end-volume
   7:
   8: volume dpkvol-dht
   9:     type cluster/distribute
  10:     subvolumes dpkvol-client-0
  11: end-volume
  12:
  13: volume dpkvol-write-behind
  14:     type performance/write-behind
  15:     subvolumes dpkvol-dht
  16: end-volume
  17:
  18: volume dpkvol-read-ahead
  19:     type performance/read-ahead
  20:     subvolumes dpkvol-write-behind
  21: end-volume
  22:
  23: volume dpkvol-io-cache
  24:     type performance/io-cache
  25:     subvolumes dpkvol-read-ahead
  26: end-volume
  27:
  28: volume dpkvol-quick-read
  29:     type performance/quick-read
  30:     subvolumes dpkvol-io-cache
  31: end-volume
  32:
  33: volume dpkvol-open-behind
  34:     type performance/open-behind
  35:     subvolumes dpkvol-quick-read
  36: end-volume
  37:
  38: volume dpkvol-md-cache
  39:     type performance/md-cache
  40:     subvolumes dpkvol-open-behind
  41: end-volume
  42:
  43: volume dpkvol
  44:     type debug/io-stats
  45:     option count-fop-hits off
  46:     option latency-measurement off
  47:     subvolumes dpkvol-md-cache
  48: end-volume

+------------------------------------------------------------------------------+
[2013-05-11 06:38:57.007678] I [rpc-clnt.c:1670:rpc_clnt_reconfig] 
0-dpkvol-client-0: changing port to 49152 (from 0)
[2013-05-11 06:38:57.007973] W [socket.c:514:__socket_rwv] 
0-dpkvol-client-0: readv failed (No data available)
[2013-05-11 06:38:57.020391] W [common-utils.c:2330:gf_ports_reserved] 
0-glusterfs-socket:  is not a valid port identifier
[2013-05-11 06:38:58.375306] I 
[client-handshake.c:1658:select_server_supported_programs] 
0-dpkvol-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2013-05-11 06:38:59.077357] W 
[client-handshake.c:1365:client_setvolume_cbk] 0-dpkvol-client-0: failed 
to set the volume (Permission denied)
[2013-05-11 06:38:59.077535] W 
[client-handshake.c:1391:client_setvolume_cbk] 0-dpkvol-client-0: failed 
to get 'process-uuid' from reply dict
[2013-05-11 06:38:59.077571] E 
[client-handshake.c:1397:client_setvolume_cbk] 0-dpkvol-client-0: 
SETVOLUME on remote-host failed: Authentication failed
[2013-05-11 06:38:59.077606] I 
[client-handshake.c:1482:client_setvolume_cbk] 0-dpkvol-client-0: 
sending AUTH_FAILED event
[2013-05-11 06:38:59.077647] E [fuse-bridge.c:4788:notify] 0-fuse: 
Server authenication failed. Shutting down.
[2013-05-11 06:38:59.077680] I [fuse-bridge.c:5212:fini] 0-fuse: 
Unmounting '/mnt'.
[2013-05-11 06:38:59.082462] W [glusterfsd.c:970:cleanup_and_exit] 
(-->/usr/lib64/libc.so.6(clone+0x6d) [0x3cbd0f199d] 
(-->/usr/lib64/libpthread.so.0() [0x3cbd407d14] 
(-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xc5) [0x406be5]))) 0-: 
received signum (15), shutting down

version
---------

[root at vdsm_tsm_int glusterfs]# gluster --version
glusterfs 3.4.0beta1 built on May 10 2013 17:55:27
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU 
General Public License.



*Server side* (gluster host)
=============

brick log
----------

[2013-05-11 06:40:19.912512] E [addr.c:152:gf_auth] 0-auth/addr: client 
is bound to port 1070 which is not privileged
[2013-05-11 06:40:19.912610] E [authenticate.c:246:gf_authenticate] 
0-auth: no authentication module is interested in accepting 
remote-client (null)
[2013-05-11 06:40:19.912639] E [server-handshake.c:587:server_setvolume] 
0-dpkvol-server: Cannot authenticate client from 
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0 3.4.0beta1
[2013-05-11 06:40:20.611853] I [server.c:771:server_rpc_notify] 
0-dpkvol-server: disconnecting connectionfrom 
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0
[2013-05-11 06:40:20.611908] I 
[server-helpers.c:735:server_connection_put] 0-dpkvol-server: Shutting 
down connection 
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0
[2013-05-11 06:40:20.611937] I 
[server-helpers.c:623:server_connection_destroy] 0-dpkvol-server: 
destroyed connection of 
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0

glusterd.vol
------------

volume management
     type mgmt/glusterd
     option working-directory /var/lib/glusterd
     option transport-type socket,rdma
     option transport.socket.keepalive-time 10
     option transport.socket.keepalive-interval 2
     option transport.socket.read-fail-log off
     option rpc-auth-allow-insecure on
end-volume

brick vol
----------

(snip..)
volume dpkvol-server
     type protocol/server
     option rpc-auth-allow-insecure on
     option auth.addr./home/dpkshetty/brick.allow null
     option auth.login.91e851d7-2f55-4946-abc2-dd18eeba4a93.password 
6c740e73-ff98-4991-b08c-1f1e9c0f9ee9
     option auth.login./home/dpkshetty/brick.allow 
91e851d7-2f55-4946-abc2-dd18eeba4a93
     option transport-type tcp
     subvolumes /home/dpkshetty/brick
end-volume

(removing auth.login lines also doesnt have any effect)

volume info
-----------

gluster volume info

Volume Name: dpkvol
Type: Distribute
Volume ID: 71fb0238-6661-4c06-ba5c-7f36c399330c
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: llmvm03:/home/dpkshetty/brick
Options Reconfigured:
server.allow-insecure: on

version
-------

gluster --version
glusterfs 3.4.0alpha2 built on Apr 10 2013 16:21:16
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU 
General Public License.



*Questions/Observations*
========================

1) Inspite of having server.allow-insecure: on (in volume option) and 
option rpc-auth-allow-insecure on (in glusterd volfile)... why do i 
still see the non-priv port error in brick logs ?
(I am running the mount cmdline as root inside VM, but its possible QEMU 
(which hosts the VM) is translating the n/w port to non-priv, but still 
auth allow insecure should have taken care of this)

2) For the same gluster host, if i try to connect from my laptop 
(instead of VM), mount works and i don't see any errors in the brick log

3) gluster --remote-host=<server ip/host> volume info works from inside 
VM (which means rpc-auth-allow-insecure on (in glusterd volfile) is 
working for this case), but not working for the mount case

4) The auth issue (from my obs) is kicking in only when non-priv port is 
being detected on the server side.. but that should not be the case as 
insecure options are set

5) Could the version mismatch between the client server be any reason 
here ?

thanx,
deepak







More information about the Gluster-devel mailing list