[Gluster-users] Fedora 17 GlusterFS 3.3.0 problmes
Nathan Stratton
nathan at robotics.net
Fri Jun 22 20:49:50 UTC 2012
On Fri, 22 Jun 2012, Harry Mangalam wrote:
> Were you automounting your glusterfs?
>
> This is extremely similar to what I described previously:
>
> <http://comments.gmane.org/gmane.comp.file-systems.gluster.user/9241>
>
> In my case the servers were running SL6.2; clients were running
> CentOS 5.7 and were automounting the glusterfs with IPoIB. When we
> switched to hard mounts, the problem seems to have gone away, but I'd
> be interested in seeing this resolved. The "Client lk-version numbers
> are not same, reopening the fds" strings in the logs were also
> identical, even when I compiled new client versions that matched the
> server version.
>
> If this does not have a trivial answer, I'll be happy to file an
> official bug report. One report could be a twit :). second one could
> be a bug.
I was automounting, however I get the same problem when I manually mount
with:
mount -t glusterfs localhost:/share /share
><>
Nathan Stratton
nathan at robotics.net
http://www.robotics.net
> hjm
>
> On Fri, Jun 22, 2012 at 7:49 AM, Nathan Stratton <nathan at robotics.net> wrote:
>>
>> When I do a NFS mount and do a ls I get:
>>
>> [root at ovirt share]# ls
>> ls: reading directory .: Too many levels of symbolic links
>>
>> [root at ovirt share]# ls -fl
>> ls: reading directory .: Too many levels of symbolic links
>> total 3636
>> drwxr-xr-x 3 root root 16384 Jun 21 19:34 .
>> dr-xr-xr-x. 21 root root 4096 Jun 21 19:29 ..
>> drwxr-xr-x 3 root root 16384 Jun 21 19:34 .
>> dr-xr-xr-x. 21 root root 4096 Jun 21 19:29 ..
>> drwxr-xr-x 3 root root 16384 Jun 21 19:34 .
>> dr-xr-xr-x. 21 root root 4096 Jun 21 19:29 ..
>> drwxr-xr-x 3 root root 16384 Jun 21 19:34 .
>> dr-xr-xr-x. 21 root root 4096 Jun 21 19:29 ..
>> drwxr-xr-x 3 root root 16384 Jun 21 19:34 .
>>
>> {and so on}
>>
>> When I try a fuse mount I get:
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-0: Server and
>> Client lk-version numbers are not same, reopening the fds
>>
>> I have tried the fedora 16 RPMs and also built new fedora 17 RPMs.
>>
>> Full Log:
>>
>> [2012-06-21 19:24:35.633510] I [glusterfsd.c:1666:main]
>> 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0
>> [2012-06-21 19:24:35.646832] I [io-cache.c:1549:check_cache_size_ok]
>> 0-share-quick-read: Max cache size is 16825450496
>> [2012-06-21 19:24:35.646916] I [io-cache.c:1549:check_cache_size_ok]
>> 0-share-io-cache: Max cache size is 16825450496
>> [2012-06-21 19:24:35.660807] I [client.c:2142:notify] 0-share-client-0:
>> parent translators are ready, attempting connect on transport
>> [2012-06-21 19:24:35.664026] I [client.c:2142:notify] 0-share-client-1:
>> parent translators are ready, attempting connect on transport
>> [2012-06-21 19:24:35.666801] I [client.c:2142:notify] 0-share-client-2:
>> parent translators are ready, attempting connect on transport
>> [2012-06-21 19:24:35.669385] I [client.c:2142:notify] 0-share-client-3:
>> parent translators are ready, attempting connect on transport
>> [2012-06-21 19:24:35.671951] I [client.c:2142:notify] 0-share-client-4:
>> parent translators are ready, attempting connect on transport
>> [2012-06-21 19:24:35.674514] I [client.c:2142:notify] 0-share-client-5:
>> parent translators are ready, attempting connect on transport
>> [2012-06-21 19:24:35.677093] I [client.c:2142:notify] 0-share-client-6:
>> parent translators are ready, attempting connect on transport
>> [2012-06-21 19:24:35.679652] I [client.c:2142:notify] 0-share-client-7:
>> parent translators are ready, attempting connect on transport
>> Given volfile:
>> +------------------------------------------------------------------------------+
>> 1: volume share-client-0
>> 2: type protocol/client
>> 3: option remote-host virt01.casabi.net
>> 4: option remote-subvolume /export
>> 5: option transport-type tcp
>> 6: option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
>> 7: option password 9641cac2-de93-4621-b94b-9320ada20fd7
>> 8: end-volume
>> 9:
>> 10: volume share-client-1
>> 11: type protocol/client
>> 12: option remote-host virt02.casabi.net
>> 13: option remote-subvolume /export
>> 14: option transport-type tcp
>> 15: option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
>> 16: option password 9641cac2-de93-4621-b94b-9320ada20fd7
>> 17: end-volume
>> 18:
>> 19: volume share-client-2
>> 20: type protocol/client
>> 21: option remote-host virt03.casabi.net
>> 22: option remote-subvolume /export
>> 23: option transport-type tcp
>> 24: option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
>> 25: option password 9641cac2-de93-4621-b94b-9320ada20fd7
>> 26: end-volume
>> 27:
>> 28: volume share-client-3
>> 29: type protocol/client
>> 30: option remote-host virt04.casabi.net
>> 31: option remote-subvolume /export
>> 32: option transport-type tcp
>> 33: option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
>> 34: option password 9641cac2-de93-4621-b94b-9320ada20fd7
>> 35: end-volume
>> 36:
>> 37: volume share-client-4
>> 38: type protocol/client
>> 39: option remote-host virt05.casabi.net
>> 40: option remote-subvolume /export
>> 41: option transport-type tcp
>> 42: option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
>> 43: option password 9641cac2-de93-4621-b94b-9320ada20fd7
>> 44: end-volume
>> 45:
>> 46: volume share-client-5
>> 47: type protocol/client
>> 48: option remote-host virt06.casabi.net
>> 49: option remote-subvolume /export
>> 50: option transport-type tcp
>> 51: option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
>> 52: option password 9641cac2-de93-4621-b94b-9320ada20fd7
>> 53: end-volume
>> 54:
>> 55: volume share-client-6
>> 56: type protocol/client
>> 57: option remote-host virt07.casabi.net
>> 58: option remote-subvolume /export
>> 59: option transport-type tcp
>> 60: option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
>> 61: option password 9641cac2-de93-4621-b94b-9320ada20fd7
>> 62: end-volume
>> 63:
>> 64: volume share-client-7
>> 65: type protocol/client
>> 66: option remote-host virt08.casabi.net
>> 67: option remote-subvolume /export
>> 68: option transport-type tcp
>> 69: option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
>> 70: option password 9641cac2-de93-4621-b94b-9320ada20fd7
>> 71: end-volume
>> 72:
>> 73: volume share-replicate-0
>> 74: type cluster/replicate
>> 75: subvolumes share-client-0 share-client-1
>> 76: end-volume
>> 77:
>> 78: volume share-replicate-1
>> 79: type cluster/replicate
>> 80: subvolumes share-client-2 share-client-3
>> 81: end-volume
>> 82:
>> 83: volume share-replicate-2
>> 84: type cluster/replicate
>> 85: subvolumes share-client-4 share-client-5
>> 86: end-volume
>> 87:
>> 88: volume share-replicate-3
>> 89: type cluster/replicate
>> 90: subvolumes share-client-6 share-client-7
>> 91: end-volume
>> 92:
>> 93: volume share-dht
>> 94: type cluster/distribute
>> 95: subvolumes share-replicate-0 share-replicate-1 share-replicate-2
>> share-replicate-3
>> 96: end-volume
>> 97:
>> 98: volume share-write-behind
>> 99: type performance/write-behind
>> 100: subvolumes share-dht
>> 101: end-volume
>> 102:
>> 103: volume share-read-ahead
>> 104: type performance/read-ahead
>> 105: subvolumes share-write-behind
>> 106: end-volume
>> 107:
>> 108: volume share-io-cache
>> 109: type performance/io-cache
>> 110: subvolumes share-read-ahead
>> 111: end-volume
>> 112:
>> 113: volume share-quick-read
>> 114: type performance/quick-read
>> 115: subvolumes share-io-cache
>> 116: end-volume
>> 117:
>> 118: volume share-md-cache
>> 119: type performance/md-cache
>> 120: subvolumes share-quick-read
>> 121: end-volume
>> 122:
>> 123: volume share
>> 124: type debug/io-stats
>> 125: option latency-measurement off
>> 126: option count-fop-hits off
>> 127: subvolumes share-md-cache
>> 128: end-volume
>>
>> +------------------------------------------------------------------------------+
>> [2012-06-21 19:24:35.682819] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
>> 0-share-client-0: changing port to 24009 (from 0)
>> [2012-06-21 19:24:35.682973] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
>> 0-share-client-1: changing port to 24010 (from 0)
>> [2012-06-21 19:24:35.683014] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
>> 0-share-client-2: changing port to 24010 (from 0)
>> [2012-06-21 19:24:35.683039] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
>> 0-share-client-3: changing port to 24010 (from 0)
>> [2012-06-21 19:24:35.683121] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
>> 0-share-client-5: changing port to 24010 (from 0)
>> [2012-06-21 19:24:35.683162] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
>> 0-share-client-4: changing port to 24010 (from 0)
>> [2012-06-21 19:24:35.683206] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
>> 0-share-client-7: changing port to 24010 (from 0)
>> [2012-06-21 19:24:35.683246] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
>> 0-share-client-6: changing port to 24010 (from 0)
>> [2012-06-21 19:24:39.644198] I
>> [client-handshake.c:1636:select_server_supported_programs] 0-share-client-0:
>> Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
>> [2012-06-21 19:24:39.644493] I
>> [client-handshake.c:1433:client_setvolume_cbk] 0-share-client-0: Connected
>> to 10.59.0.11:24009, attached to remote volume '/export'.
>> [2012-06-21 19:24:39.644512] I
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-0: Server and
>> Client lk-version numbers are not same, reopening the fds
>> [2012-06-21 19:24:39.644579] I [afr-common.c:3627:afr_notify]
>> 0-share-replicate-0: Subvolume 'share-client-0' came back up; going online.
>> [2012-06-21 19:24:39.644621] I
>> [client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-0: Server
>> lk version = 1
>> [2012-06-21 19:24:39.646923] I
>> [client-handshake.c:1636:select_server_supported_programs] 0-share-client-1:
>> Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
>> [2012-06-21 19:24:39.647207] I
>> [client-handshake.c:1433:client_setvolume_cbk] 0-share-client-1: Connected
>> to 10.59.0.12:24010, attached to remote volume '/export'.
>> [2012-06-21 19:24:39.647227] I
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-1: Server and
>> Client lk-version numbers are not same, reopening the fds
>> [2012-06-21 19:24:39.647371] I
>> [client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-1: Server
>> lk version = 1
>> [2012-06-21 19:24:39.649587] I
>> [client-handshake.c:1636:select_server_supported_programs] 0-share-client-2:
>> Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
>> [2012-06-21 19:24:39.649806] I
>> [client-handshake.c:1433:client_setvolume_cbk] 0-share-client-2: Connected
>> to 10.59.0.13:24010, attached to remote volume '/export'.
>> [2012-06-21 19:24:39.649826] I
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-2: Server and
>> Client lk-version numbers are not same, reopening the fds
>> [2012-06-21 19:24:39.649865] I [afr-common.c:3627:afr_notify]
>> 0-share-replicate-1: Subvolume 'share-client-2' came back up; going online.
>> [2012-06-21 19:24:39.649935] I
>> [client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-2: Server
>> lk version = 1
>> [2012-06-21 19:24:39.652192] I
>> [client-handshake.c:1636:select_server_supported_programs] 0-share-client-3:
>> Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
>> [2012-06-21 19:24:39.652428] I
>> [client-handshake.c:1433:client_setvolume_cbk] 0-share-client-3: Connected
>> to 10.59.0.14:24010, attached to remote volume '/export'.
>> [2012-06-21 19:24:39.652448] I
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-3: Server and
>> Client lk-version numbers are not same, reopening the fds
>> [2012-06-21 19:24:39.652566] I
>> [client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-3: Server
>> lk version = 1
>> [2012-06-21 19:24:39.654706] I
>> [client-handshake.c:1636:select_server_supported_programs] 0-share-client-5:
>> Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
>> [2012-06-21 19:24:39.654910] I
>> [client-handshake.c:1433:client_setvolume_cbk] 0-share-client-5: Connected
>> to 10.59.0.16:24010, attached to remote volume '/export'.
>> [2012-06-21 19:24:39.654926] I
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-5: Server and
>> Client lk-version numbers are not same, reopening the fds
>> [2012-06-21 19:24:39.654961] I [afr-common.c:3627:afr_notify]
>> 0-share-replicate-2: Subvolume 'share-client-5' came back up; going online.
>> [2012-06-21 19:24:39.655035] I
>> [client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-5: Server
>> lk version = 1
>> [2012-06-21 19:24:39.657289] I
>> [client-handshake.c:1636:select_server_supported_programs] 0-share-client-4:
>> Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
>> [2012-06-21 19:24:39.657511] I
>> [client-handshake.c:1433:client_setvolume_cbk] 0-share-client-4: Connected
>> to 10.59.0.15:24010, attached to remote volume '/export'.
>> [2012-06-21 19:24:39.657526] I
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-4: Server and
>> Client lk-version numbers are not same, reopening the fds
>> [2012-06-21 19:24:39.657646] I
>> [client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-4: Server
>> lk version = 1
>> [2012-06-21 19:24:39.659884] I
>> [client-handshake.c:1636:select_server_supported_programs] 0-share-client-7:
>> Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
>> [2012-06-21 19:24:39.660075] I
>> [client-handshake.c:1433:client_setvolume_cbk] 0-share-client-7: Connected
>> to 10.59.0.18:24010, attached to remote volume '/export'.
>> [2012-06-21 19:24:39.660091] I
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-7: Server and
>> Client lk-version numbers are not same, reopening the fds
>> [2012-06-21 19:24:39.660127] I [afr-common.c:3627:afr_notify]
>> 0-share-replicate-3: Subvolume 'share-client-7' came back up; going online.
>> [2012-06-21 19:24:39.660202] I
>> [client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-7: Server
>> lk version = 1
>> [2012-06-21 19:24:39.662579] I
>> [client-handshake.c:1636:select_server_supported_programs] 0-share-client-6:
>> Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
>> [2012-06-21 19:24:39.662908] I
>> [client-handshake.c:1433:client_setvolume_cbk] 0-share-client-6: Connected
>> to 10.59.0.17:24010, attached to remote volume '/export'.
>> [2012-06-21 19:24:39.662931] I
>> [client-handshake.c:1445:client_setvolume_cbk] 0-share-client-6: Server and
>> Client lk-version numbers are not same, reopening the fds
>> [2012-06-21 19:24:39.668028] I [fuse-bridge.c:4193:fuse_graph_setup] 0-fuse:
>> switched to graph 0
>> [2012-06-21 19:24:39.668234] I
>> [client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-6: Server
>> lk version = 1
>> [2012-06-21 19:24:39.668287] I [fuse-bridge.c:3376:fuse_init]
>> 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel
>> 7.18
>> [2012-06-21 19:24:39.668739] I
>> [afr-common.c:1964:afr_set_root_inode_on_first_lookup] 0-share-replicate-0:
>> added root inode
>> [2012-06-21 19:24:39.669232] I
>> [afr-common.c:1964:afr_set_root_inode_on_first_lookup] 0-share-replicate-1:
>> added root inode
>> [2012-06-21 19:24:39.669321] I
>> [afr-common.c:1964:afr_set_root_inode_on_first_lookup] 0-share-replicate-2:
>> added root inode
>> [2012-06-21 19:24:39.669379] I
>> [afr-common.c:1964:afr_set_root_inode_on_first_lookup] 0-share-replicate-3:
>> added root inode
>>
>>
>>
>>> <>
>>
>> Nathan Stratton
>> nathan at robotics.net
>> http://www.robotics.net
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
More information about the Gluster-users
mailing list