[Gluster-users] Setting up 3.1

[Marcus Bointon]

Thanks for responding Amar.

On 13 Jan 2011, at 06:49, Amar Tumballi wrote:

> The default transport-type is 'tcp' in 3.1.x version, hence you don't need to specify any transport type for tcp.

That's good to know - where is that documented?

> What was the earlier error in glusterd's log file ? 'glusterd.vol' should not be edited by anyone, hence its not documented anywhere. Also, we changed the default port from 6996 to 24007 in 3.1.x version, so obviously there is nothing on 6996 after you upgraded.

This was the initial (default) config:

Given volfile:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option working-directory /etc/glusterd
  4:     option transport-type socket,rdma
  5:     option transport.socket.keepalive-time 10
  6:     option transport.socket.keepalive-interval 2
  7: end-volume
  8: 

and it reported errors like this:

[2011-01-07 21:03:06.219877] I [glusterfsd.c:672:cleanup_and_exit] glusterfsd: shutting down
[2011-01-07 21:03:08.286880] I [glusterd.c:275:init] management: Using /etc/glusterd as working directory
[2011-01-07 21:03:08.289496] E [socket.c:322:__socket_server_bind] socket.management: binding to  failed: Address already in use
[2011-01-07 21:03:08.289544] E [socket.c:325:__socket_server_bind] socket.management: Port is already in use
[2011-01-07 21:03:08.289652] I [glusterd.c:87:glusterd_uuid_init] glusterd: retrieved UUID: 8563bfd6-4ce9-4aa3-bef5-a20597dda496
[2011-01-07 21:03:08.292233] W [rpc-transport.c:849:rpc_transport_load] rpc-transport: missing 'option transport-type'. defaulting to "socket"
[2011-01-07 21:03:08.293277] I [glusterd-handler.c:2600:glusterd_friend_add] glusterd: connect returned 0
[2011-01-07 21:03:08.296098] W [rpc-transport.c:849:rpc_transport_load] rpc-transport: missing 'option transport-type'. defaulting to "socket"

I changed transport-type to 'tcp' and all the errors went away.

Now I look, it is indeed working on port 24007. I changed the transport-type back to the original, restarted it and found that it still works - but why is it now using a transport-type that's not listed? In searching the docs again, I can still only find documentation on transport-type for gluster versions older than 3.0.

> Is there some documentation or howto for getting gluster 3.1.1 working over tcp? I don't think my situation is exactly rare!
> 
> All the docs present are updated for version 3.1.x, and are kept up to date. Please refer to http://www.gluster.com/community/documentation/index.php/Gluster_3.0_to_3.1_Upgrade_Guide and let us know what exactly was the problem in upgrading.

The main problem I found with that page is that it didn't actually say much that was very useful - it's a leaf node (not a root) of a whole tree of docs you then need discover and read in order to do the upgrade - for example the need for a server pool is not mentioned at all when it's a vital new concept. I'm installing on Ubuntu, but I have now found the firewall info on the redhat install page (and nowhere else according to a search). I don't think firewall config belongs on OS-specific pages. You could show how to open ports on each specific OS, but the general requirements should documented independently and findably - searching for 'firewall port' doesn't find it. Overall I found the older (obsolete) user guide much more usable, but obviously that's out of date.

A nicer upgrade path would be one that reads a volfile (maybe just the volgen line) and emits a series of gluster client commands to build the same structure.

I shall persevere...

Marcus