[Gluster-users] 3.1 Can't mount glusterfs mountpoint - Address already in use

Mon Jan 31 20:24:50 UTC 2011

I have a problem with Gluster 3.1.1 (debian) that I didn't have a few
weeks ago. I can run glusterd on my 6 servers, I can see all 6 of them
from the main one, but the logs keep complaining about:

==> /var/log/glusterfs/nfs.log <==
[2011-01-31 14:46:49.157527] E
[client-handshake.c:1067:client_query_portmap_cbk]
bhl-volume-client-98: failed to get the port number for remote
subvolume
[2011-01-31 14:46:53.158708] E
[client-handshake.c:1067:client_query_portmap_cbk]
bhl-volume-client-98: failed to get the port number for remote
subvolume
^[	[2011-01-31 14:46:57.159754] E
[client-handshake.c:1067:client_query_portmap_cbk]
bhl-volume-client-98: failed to get the port number for remote
subvolume
[2011-01-31 14:47:01.160804] E
[client-handshake.c:1067:client_query_portmap_cbk]
bhl-volume-client-98: failed to get the port number for remote
subvolume

Then, if I try to mount the glusterfs I get the following errors in
glusterfs.log

==> /var/log/glusterfs/mnt-glusterfs.log <==
[2011-01-31 14:46:48.720968] I [glusterd.c:275:init] management: Using
/etc/glusterd as working directory
[2011-01-31 14:46:48.721514] E [socket.c:322:__socket_server_bind]
socket.management: binding to  failed: Address already in use
[2011-01-31 14:46:48.721535] E [socket.c:325:__socket_server_bind]
socket.management: Port is already in use
[2011-01-31 14:46:48.721561] E [glusterd.c:348:init] management:
creation of listener failed
[2011-01-31 14:46:48.721577] E [xlator.c:909:xlator_init] management:
Initialization of volume 'management' failed, review your volfile
again
[2011-01-31 14:46:48.721595] E [graph.c:331:glusterfs_graph_init]
management: initializing translator failed
[2011-01-31 14:46:48.721635] I [fuse-bridge.c:3616:fini] fuse:
Unmounting '/mnt/glusterfs'.
[2011-01-31 14:46:48.823199] I [glusterfsd.c:672:cleanup_and_exit]
glusterfsd: shutting down

But I don't see how it's already in use; I've made sure everything was
stopped/killed on the main server, but when I restart everything fresh
the above happens. So, are these two related, or how can I debug the
mounting issue? incidentally, if I do a volume info everything seems
to be mounted and ok:

# gluster
gluster> volume info bhl-volume

Volume Name: bhl-volume
Type: Distributed-Replicate
Status: Started
Number of Bricks: 72 x 2 = 144
Transport-type: tcp
Bricks:
Brick1: clustr-01:/mnt/data01
Brick2: clustr-02:/mnt/data01
Brick3: clustr-03:/mnt/data01
Brick4: clustr-04:/mnt/data01
Brick5: clustr-05:/mnt/data01
Brick6: clustr-06:/mnt/data01
Brick7: clustr-01:/mnt/data02
Brick8: clustr-02:/mnt/data02
Brick9: clustr-03:/mnt/data02
Brick10: clustr-04:/mnt/data02
Brick11: clustr-05:/mnt/data02
Brick12: clustr-06:/mnt/data02
Brick13: clustr-01:/mnt/data03
Brick14: clustr-02:/mnt/data03
Brick15: clustr-03:/mnt/data03
Brick16: clustr-04:/mnt/data03
Brick17: clustr-05:/mnt/data03
Brick18: clustr-06:/mnt/data03
Brick19: clustr-01:/mnt/data04
Brick20: clustr-02:/mnt/data04
Brick21: clustr-03:/mnt/data04
Brick22: clustr-04:/mnt/data04
Brick23: clustr-05:/mnt/data04
Brick24: clustr-06:/mnt/data04
Brick25: clustr-01:/mnt/data05
Brick26: clustr-02:/mnt/data05
Brick27: clustr-03:/mnt/data05
Brick28: clustr-04:/mnt/data05
Brick29: clustr-05:/mnt/data05
Brick30: clustr-06:/mnt/data05
Brick31: clustr-01:/mnt/data06
Brick32: clustr-02:/mnt/data06
Brick33: clustr-03:/mnt/data06
Brick34: clustr-04:/mnt/data06
Brick35: clustr-05:/mnt/data06
Brick36: clustr-06:/mnt/data06
Brick37: clustr-01:/mnt/data07
Brick38: clustr-02:/mnt/data07
Brick39: clustr-03:/mnt/data07
Brick40: clustr-04:/mnt/data07
Brick41: clustr-05:/mnt/data07
Brick42: clustr-06:/mnt/data07
Brick43: clustr-01:/mnt/data08
Brick44: clustr-02:/mnt/data08
Brick45: clustr-03:/mnt/data08
Brick46: clustr-04:/mnt/data08
Brick47: clustr-05:/mnt/data08
Brick48: clustr-06:/mnt/data08
Brick49: clustr-01:/mnt/data09
Brick50: clustr-02:/mnt/data09
Brick51: clustr-03:/mnt/data09
Brick52: clustr-04:/mnt/data09
Brick53: clustr-05:/mnt/data09
Brick54: clustr-06:/mnt/data09
Brick55: clustr-01:/mnt/data10
Brick56: clustr-02:/mnt/data10
Brick57: clustr-03:/mnt/data10
Brick58: clustr-04:/mnt/data10
Brick59: clustr-05:/mnt/data10
Brick60: clustr-06:/mnt/data10
Brick61: clustr-01:/mnt/data11
Brick62: clustr-02:/mnt/data11
Brick63: clustr-03:/mnt/data11
Brick64: clustr-04:/mnt/data11
Brick65: clustr-05:/mnt/data11
Brick66: clustr-06:/mnt/data11
Brick67: clustr-01:/mnt/data12
Brick68: clustr-02:/mnt/data12
Brick69: clustr-03:/mnt/data12
Brick70: clustr-04:/mnt/data12
Brick71: clustr-05:/mnt/data12
Brick72: clustr-06:/mnt/data12
Brick73: clustr-01:/mnt/data13
Brick74: clustr-02:/mnt/data13
Brick75: clustr-03:/mnt/data13
Brick76: clustr-04:/mnt/data13
Brick77: clustr-05:/mnt/data13
Brick78: clustr-06:/mnt/data13
Brick79: clustr-01:/mnt/data14
Brick80: clustr-02:/mnt/data14
Brick81: clustr-03:/mnt/data14
Brick82: clustr-04:/mnt/data14
Brick83: clustr-05:/mnt/data14
Brick84: clustr-06:/mnt/data14
Brick85: clustr-01:/mnt/data15
Brick86: clustr-02:/mnt/data15
Brick87: clustr-03:/mnt/data15
Brick88: clustr-04:/mnt/data15
Brick89: clustr-05:/mnt/data15
Brick90: clustr-06:/mnt/data15
Brick91: clustr-01:/mnt/data16
Brick92: clustr-02:/mnt/data16
Brick93: clustr-03:/mnt/data16
Brick94: clustr-04:/mnt/data16
Brick95: clustr-05:/mnt/data16
Brick96: clustr-06:/mnt/data16
Brick97: clustr-01:/mnt/data17
Brick98: clustr-02:/mnt/data17
Brick99: clustr-03:/mnt/data17
Brick100: clustr-04:/mnt/data17
Brick101: clustr-05:/mnt/data17
Brick102: clustr-06:/mnt/data17
Brick103: clustr-01:/mnt/data18
Brick104: clustr-02:/mnt/data18
Brick105: clustr-03:/mnt/data18
Brick106: clustr-04:/mnt/data18
Brick107: clustr-05:/mnt/data18
Brick108: clustr-06:/mnt/data18
Brick109: clustr-01:/mnt/data19
Brick110: clustr-02:/mnt/data19
Brick111: clustr-03:/mnt/data19
Brick112: clustr-04:/mnt/data19
Brick113: clustr-05:/mnt/data19
Brick114: clustr-06:/mnt/data19
Brick115: clustr-01:/mnt/data20
Brick116: clustr-02:/mnt/data20
Brick117: clustr-03:/mnt/data20
Brick118: clustr-04:/mnt/data20
Brick119: clustr-05:/mnt/data20
Brick120: clustr-06:/mnt/data20
Brick121: clustr-01:/mnt/data21
Brick122: clustr-02:/mnt/data21
Brick123: clustr-03:/mnt/data21
Brick124: clustr-04:/mnt/data21
Brick125: clustr-05:/mnt/data21
Brick126: clustr-06:/mnt/data21
Brick127: clustr-01:/mnt/data22
Brick128: clustr-02:/mnt/data22
Brick129: clustr-03:/mnt/data22
Brick130: clustr-04:/mnt/data22
Brick131: clustr-05:/mnt/data22
Brick132: clustr-06:/mnt/data22
Brick133: clustr-01:/mnt/data23
Brick134: clustr-02:/mnt/data23
Brick135: clustr-03:/mnt/data23
Brick136: clustr-04:/mnt/data23
Brick137: clustr-05:/mnt/data23
Brick138: clustr-06:/mnt/data23
Brick139: clustr-01:/mnt/data24
Brick140: clustr-02:/mnt/data24
Brick141: clustr-03:/mnt/data24
Brick142: clustr-04:/mnt/data24
Brick143: clustr-05:/mnt/data24
Brick144: clustr-06:/mnt/data24
gluster>

-- 
http://philcryer.com