[Gluster-users] Fwd: Seeing non-priv port + auth issue in the gluster brick log

Deepak C Shetty deepakcs at linux.vnet.ibm.com
Mon May 13 04:12:33 UTC 2013

Fwd.ing to Gluster users, in the hope that many more people can see this 
and hopefully can provide any clues


-------- Original Message --------
Subject: 	[Gluster-devel] Seeing non-priv port + auth issue in the 
gluster brick log
Date: 	Sat, 11 May 2013 12:43:20 +0530
From: 	Deepak C Shetty <deepakcs at linux.vnet.ibm.com>
Organization: 	IBM India Pvt. Ltd.
To: 	Gluster Devel <gluster-devel at nongnu.org>

Hi All,
    I am trying to mount a gluster volume from inside a VM (being used
as a VDSM host) and seeing the below issue.
 From the VM, I am able to ping, telnet to the gluster host (no
networking issues present)

*Client side*

[root at vdsm_tsm_int glusterfs]# mount -t glusterfs /mnt
Mount failed. Please check the log file for more details.

(Using IP or hostname in the mount cmdline doesn't change anything)


[2013-05-11 06:38:54.199518] I [glusterfsd.c:1878:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version
3.4.0beta1 (/usr/sbin/glusterfs --volfile-id=dpkvol
--volfile-server= /mnt)
[2013-05-11 06:38:54.203474] I [socket.c:3480:socket_init] 0-glusterfs:
SSL support is NOT enabled
[2013-05-11 06:38:54.203652] I [socket.c:3495:socket_init] 0-glusterfs:
using system polling thread
[2013-05-11 06:38:54.206592] W [common-utils.c:2330:gf_ports_reserved]
0-glusterfs-socket:  is not a valid port identifier
[2013-05-11 06:38:55.253221] I [socket.c:3480:socket_init]
0-dpkvol-client-0: SSL support is NOT enabled
[2013-05-11 06:38:55.253268] I [socket.c:3495:socket_init]
0-dpkvol-client-0: using system polling thread
[2013-05-11 06:38:55.253300] I [client.c:2154:notify] 0-dpkvol-client-0:
parent translators are ready, attempting connect on transport
[2013-05-11 06:38:55.255834] W [common-utils.c:2330:gf_ports_reserved]
0-glusterfs-socket:  is not a valid port identifier
Given volfile:
   1: volume dpkvol-client-0
   2:     type protocol/client
   3:     option transport-type tcp
   4:     option remote-subvolume /home/dpkshetty/brick
   5:     option remote-host llmvm03
   6: end-volume
   8: volume dpkvol-dht
   9:     type cluster/distribute
  10:     subvolumes dpkvol-client-0
  11: end-volume
  13: volume dpkvol-write-behind
  14:     type performance/write-behind
  15:     subvolumes dpkvol-dht
  16: end-volume
  18: volume dpkvol-read-ahead
  19:     type performance/read-ahead
  20:     subvolumes dpkvol-write-behind
  21: end-volume
  23: volume dpkvol-io-cache
  24:     type performance/io-cache
  25:     subvolumes dpkvol-read-ahead
  26: end-volume
  28: volume dpkvol-quick-read
  29:     type performance/quick-read
  30:     subvolumes dpkvol-io-cache
  31: end-volume
  33: volume dpkvol-open-behind
  34:     type performance/open-behind
  35:     subvolumes dpkvol-quick-read
  36: end-volume
  38: volume dpkvol-md-cache
  39:     type performance/md-cache
  40:     subvolumes dpkvol-open-behind
  41: end-volume
  43: volume dpkvol
  44:     type debug/io-stats
  45:     option count-fop-hits off
  46:     option latency-measurement off
  47:     subvolumes dpkvol-md-cache
  48: end-volume

[2013-05-11 06:38:57.007678] I [rpc-clnt.c:1670:rpc_clnt_reconfig]
0-dpkvol-client-0: changing port to 49152 (from 0)
[2013-05-11 06:38:57.007973] W [socket.c:514:__socket_rwv]
0-dpkvol-client-0: readv failed (No data available)
[2013-05-11 06:38:57.020391] W [common-utils.c:2330:gf_ports_reserved]
0-glusterfs-socket:  is not a valid port identifier
[2013-05-11 06:38:58.375306] I
0-dpkvol-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2013-05-11 06:38:59.077357] W
[client-handshake.c:1365:client_setvolume_cbk] 0-dpkvol-client-0: failed
to set the volume (Permission denied)
[2013-05-11 06:38:59.077535] W
[client-handshake.c:1391:client_setvolume_cbk] 0-dpkvol-client-0: failed
to get 'process-uuid' from reply dict
[2013-05-11 06:38:59.077571] E
[client-handshake.c:1397:client_setvolume_cbk] 0-dpkvol-client-0:
SETVOLUME on remote-host failed: Authentication failed
[2013-05-11 06:38:59.077606] I
[client-handshake.c:1482:client_setvolume_cbk] 0-dpkvol-client-0:
sending AUTH_FAILED event
[2013-05-11 06:38:59.077647] E [fuse-bridge.c:4788:notify] 0-fuse:
Server authenication failed. Shutting down.
[2013-05-11 06:38:59.077680] I [fuse-bridge.c:5212:fini] 0-fuse:
Unmounting '/mnt'.
[2013-05-11 06:38:59.082462] W [glusterfsd.c:970:cleanup_and_exit]
(-->/usr/lib64/libc.so.6(clone+0x6d) [0x3cbd0f199d]
(-->/usr/lib64/libpthread.so.0() [0x3cbd407d14]
(-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xc5) [0x406be5]))) 0-:
received signum (15), shutting down


[root at vdsm_tsm_int glusterfs]# gluster --version
glusterfs 3.4.0beta1 built on May 10 2013 17:55:27
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
You may redistribute copies of GlusterFS under the terms of the GNU
General Public License.

*Server side* (gluster host)

brick log

[2013-05-11 06:40:19.912512] E [addr.c:152:gf_auth] 0-auth/addr: client
is bound to port 1070 which is not privileged
[2013-05-11 06:40:19.912610] E [authenticate.c:246:gf_authenticate]
0-auth: no authentication module is interested in accepting
remote-client (null)
[2013-05-11 06:40:19.912639] E [server-handshake.c:587:server_setvolume]
0-dpkvol-server: Cannot authenticate client from
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0 3.4.0beta1
[2013-05-11 06:40:20.611853] I [server.c:771:server_rpc_notify]
0-dpkvol-server: disconnecting connectionfrom
[2013-05-11 06:40:20.611908] I
[server-helpers.c:735:server_connection_put] 0-dpkvol-server: Shutting
down connection
[2013-05-11 06:40:20.611937] I
[server-helpers.c:623:server_connection_destroy] 0-dpkvol-server:
destroyed connection of


volume management
     type mgmt/glusterd
     option working-directory /var/lib/glusterd
     option transport-type socket,rdma
     option transport.socket.keepalive-time 10
     option transport.socket.keepalive-interval 2
     option transport.socket.read-fail-log off
     option rpc-auth-allow-insecure on

brick vol

volume dpkvol-server
     type protocol/server
     option rpc-auth-allow-insecure on
     option auth.addr./home/dpkshetty/brick.allow null
     option auth.login.91e851d7-2f55-4946-abc2-dd18eeba4a93.password
     option auth.login./home/dpkshetty/brick.allow
     option transport-type tcp
     subvolumes /home/dpkshetty/brick

(removing auth.login lines also doesnt have any effect)

volume info

gluster volume info

Volume Name: dpkvol
Type: Distribute
Volume ID: 71fb0238-6661-4c06-ba5c-7f36c399330c
Status: Started
Number of Bricks: 1
Transport-type: tcp
Brick1: llmvm03:/home/dpkshetty/brick
Options Reconfigured:
server.allow-insecure: on


gluster --version
glusterfs 3.4.0alpha2 built on Apr 10 2013 16:21:16
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
You may redistribute copies of GlusterFS under the terms of the GNU
General Public License.


1) Inspite of having server.allow-insecure: on (in volume option) and
option rpc-auth-allow-insecure on (in glusterd volfile)... why do i
still see the non-priv port error in brick logs ?
(I am running the mount cmdline as root inside VM, but its possible QEMU
(which hosts the VM) is translating the n/w port to non-priv, but still
auth allow insecure should have taken care of this)

2) For the same gluster host, if i try to connect from my laptop
(instead of VM), mount works and i don't see any errors in the brick log

3) gluster --remote-host=<server ip/host> volume info works from inside
VM (which means rpc-auth-allow-insecure on (in glusterd volfile) is
working for this case), but not working for the mount case

4) The auth issue (from my obs) is kicking in only when non-priv port is
being detected on the server side.. but that should not be the case as
insecure options are set

5) Could the version mismatch between the client server be any reason
here ?


Gluster-devel mailing list
Gluster-devel at nongnu.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130513/79d5eeed/attachment.html>

More information about the Gluster-users mailing list