[Gluster-devel] Connection refused error over Infiniband...

[Sriharsha Setty]

Hi  all,

I have a setup where  I want to mount a glusterfs volume on 6 nodes (network
filesystem style)  using Infiniband.  Here is my server and client
configuration:


 cat /etc/glusterfs-server.vol
----------------------------------------
### File: /etc/glusterfs-server.vol - GlusterFS Server Volume Specification

### Export volume "brick" with the contents of "/home/export" directory.
volume brick
  type storage/posix                   # POSIX FS translator
  option directory /glusterfs      # Export this directory
end-volume

### Add network serving capability to above brick.
volume server
  type protocol/server
#  option transport-type tcp/server     # For TCP/IP transport
  option transport-type ib-sdp/server  # For Infiniband transport
  option bind-address 172.22.0.5     # Default is to listen on all
interfaces
  option listen-port 6996              # Default is 6996
  option client-volume-filename /etc/glusterfs/glusterfs-client.vol
  subvolumes brick
# NOTE: Access to any volume through protocol/server is denied by
# default. You need to explicxitly grant access through # "auth"
# option.
  option auth.ip.brick.allow 172.22.* # Allow access to "brick" volume
end-volume




cat  /etc/glusterfs-client.vol
----------------------------------
### File: /etc/glusterfs-client.vol - GlusterFS Client Volume Specification

### Add client feature and attach to remote subvolume
volume client
  type protocol/client
 #option transport-type tcp/client     # for TCP/IP transport
  option transport-type ib-sdp/client  # for Infiniband transport
  option remote-host 172.22.0.5      # IP address of the remote brick
  option remote-port 6996              # default server port is 6996
  option remote-subvolume brick        # name of the remote volume
end-volume



This is what I get when I do:
glusterfs  --log-file=/dev/stdout --log-level=DEBUG -f /etc/glusterfs-
client.vol /mnt
-------------------------------
[ib-sdp-client.c:70:ib_sdp_connect] client: socket fd = 4
2007-07-17 01:12:04 D [ib-sdp-client.c:88:ib_sdp_connect] client: finalized
on port `1023'
2007-07-17 01:12:04 D [ib-sdp-client.c:141:ib_sdp_connect] client: connect
on 4 in progress (non-blocking)
2007-07-17 01:12:04 D [ib-sdp-client.c:179:ib_sdp_connect] client:
connection on 4 still in progress - try later
2007-07-17 01:12:05 W [client-protocol.c:4071:client_protocol_reconnect]
client: attempting reconnect
2007-07-17 01:12:05 E [ib-sdp-client.c:171:ib_sdp_connect] client:
non-blocking connect() returned: 111 (Connection refused)
2007-07-17 01:12:07 W [client-protocol.c:4071:client_protocol_reconnect]
client: attempting reconnect
2007-07-17 01:12:07 D [ib-sdp-client.c:70:ib_sdp_connect] client: socket fd
= 4
2007-07-17 01:12:07 D [ib-sdp-client.c:88:ib_sdp_connect] client: finalized
on port `1023'
2007-07-17 01:12:07 D [ib-sdp-client.c:141:ib_sdp_connect] client: connect
on 4 in progress (non-blocking)
2007-07-17 01:12:07 D [ib-sdp-client.c:179:ib_sdp_connect] client:
connection on 4 still in progress - try later
2007-07-17 01:12:10 W [client-protocol.c:4071:client_protocol_reconnect]
client: attempting reconnect
2007-07-17 01:12:10 E [ib-sdp-client.c:171:ib_sdp_connect] client:
non-blocking connect() returned: 111 (Connection refused)
2007-07-17 01:12:15 W [client-protocol.c:4071:client_protocol_reconnect]
client: attempting reconnect


Looks to me that the server runs fine when I start it.  Any thoughts will be
appreciated.
Lemme know if I missed out any detail that you need to know.

TIA,
/harsha