[Gluster-users] Files and directories not accessible

Joe Julian joe at julianfamily.org
Thu Sep 27 00:06:45 UTC 2012 

I had something that produced similar errors once, turned out that the 
brick needed an fsck.

On 09/26/2012 02:58 PM, Gerald Brandt wrote:
> These are the errors in my nfs.log file:
>
> [2012-09-26 16:55:59.018675] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-gl-raid6-client-0: changing port to 24009 (from 0)
> [2012-09-26 16:55:59.019002] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-gl-raid1-client-0: changing port to 24010 (from 0)
> [2012-09-26 16:56:00.768936] E [nfs3.c:812:nfs3_getattr] 0-nfs-nfsv3: Volume is disabled: gl-raid6
> [2012-09-26 16:56:00.769041] W [rpcsvc.c:523:rpcsvc_handle_rpc_call] 0-rpcsvc: failed to queue error reply
> [2012-09-26 16:56:02.688516] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 16
> [2012-09-26 16:56:02.688646] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported
> [2012-09-26 16:56:02.834420] I [client-handshake.c:1636:select_server_supported_programs] 0-gl-raid6-client-0: Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
> [2012-09-26 16:56:02.835091] I [client-handshake.c:1433:client_setvolume_cbk] 0-gl-raid6-client-0: Connected to 192.168.101.100:24009, attached to remote volume '/export/gluster/RAID6'.
> [2012-09-26 16:56:02.835182] I [client-handshake.c:1445:client_setvolume_cbk] 0-gl-raid6-client-0: Server and Client lk-version numbers are not same, reopening the fds
> [2012-09-26 16:56:02.835858] I [client-handshake.c:453:client_set_lk_version_cbk] 0-gl-raid6-client-0: Server lk version = 1
> [2012-09-26 16:56:02.836157] I [client-handshake.c:1636:select_server_supported_programs] 0-gl-raid1-client-0: Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
> [2012-09-26 16:56:02.836714] I [client-handshake.c:1433:client_setvolume_cbk] 0-gl-raid1-client-0: Connected to 192.168.101.100:24010, attached to remote volume '/export/gluster/RAID1'.
> [2012-09-26 16:56:02.836765] I [client-handshake.c:1445:client_setvolume_cbk] 0-gl-raid1-client-0: Server and Client lk-version numbers are not same, reopening the fds
> [2012-09-26 16:56:02.837113] I [client-handshake.c:453:client_set_lk_version_cbk] 0-gl-raid1-client-0: Server lk version = 1
> [2012-09-26 16:56:54.895081] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : 816f71e8-b7aa-4288-a4eb-daa35fdd84fc
> [2012-09-26 16:56:54.895219] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: b23d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
> [2012-09-26 16:56:54.897229] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : a82addf6-18f3-4bd5-abff-1c14c7b03df3
> [2012-09-26 16:56:54.897301] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: b53d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
> [2012-09-26 16:56:54.901589] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : bf0204a1-114e-481a-90e1-ae3ea763552d
> [2012-09-26 16:56:54.901654] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: b83d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
> [2012-09-26 16:56:54.904033] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : 076093c7-a6dd-48c1-be95-8436693e50fd
> [2012-09-26 16:56:54.904100] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: bb3d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
> [2012-09-26 16:56:54.905984] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : d8c5df74-3e48-4e3a-b7cb-d14517238808
> [2012-09-26 16:56:54.906044] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: be3d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
> [2012-09-26 16:56:54.908171] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : 6a174fb8-b420-4893-b762-de4b52d44a8c
> [2012-09-26 16:56:54.908234] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: c13d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
> [2012-09-26 16:56:54.910128] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : 0cf654e6-7811-433f-8332-808d57025dee
> [2012-09-26 16:56:54.910188] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: c43d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
> [2012-09-26 16:56:54.912192] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : a83b3838-6881-48a1-be77-9cf0d72e1d22
> [2012-09-26 16:56:54.912302] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: c73d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
> [2012-09-26 16:56:54.914221] E [nfs3.c:753:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.101.254:833) gl-raid6 : 646a34f7-e713-4c5b-b34b-d530729f674b
> [2012-09-26 16:56:54.914281] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: ca3d379c, GETATTR: NFS: 2(No such file or directory), POSIX: 14(Bad address)
>
>
>
> ----- Original Message -----
> From: "Gerald Brandt" <gbr at majentis.com>
> To: "gluster-users" <gluster-users at gluster.org>
> Sent: Tuesday, September 25, 2012 6:29:46 PM
> Subject: [Gluster-users] Files and directories not accessible
>
> Hi,
>
> I had a running/working GlusterFS system, but the boot drive died.  I re-installed and setup gluster to export the same data (using NFS).  This is what I get when I mount it on a remote system:
>
> #ls -l
> ls: cannot access dbbackups: No such file or directory
> ls: cannot access fanart: No such file or directory
> ls: cannot access games: No such file or directory
> ls: cannot access movies: No such file or directory
> ls: cannot access music: No such file or directory
> ls: cannot access music-old: No such file or directory
> ls: cannot access pictures: No such file or directory
> ls: cannot access posters: No such file or directory
> ls: cannot access scan0001.tif: No such file or directory
> ls: cannot access scan0002.tif: No such file or directory
> ls: cannot access tv1: No such file or directory
> total 0
> d????????? ? ? ? ?            ? dbbackups
> d????????? ? ? ? ?            ? fanart
> d????????? ? ? ? ?            ? games
> d????????? ? ? ? ?            ? movies
> d????????? ? ? ? ?            ? music
> d????????? ? ? ? ?            ? music-old
> d????????? ? ? ? ?            ? pictures
> d????????? ? ? ? ?            ? posters
> -????????? ? ? ? ?            ? scan0001.tif
> -????????? ? ? ? ?            ? scan0002.tif
> d????????? ? ? ? ?            ? tv1
>
>
> Any ideas what's happening?  I have a few gluster installs and I've never seen this before.
>
> Ubuntu 12.04
> XFS brick
> GlusterFS 3.3
>
> Volume Name: gl-raid6
> Type: Distribute
> Volume ID: 027c1eea-3b1c-4a51-8097-3db9425b45f0
> Status: Started
> Number of Bricks: 1
> Transport-type: tcp
> Bricks:
> Brick1: 192.168.101.100:/export/gluster/RAID6
> Options Reconfigured:
> performance.read-ahead: on
> performance.io-cache: on
> performance.write-behind-window-size: 4mb
> performance.io-thread-count: 64
> performance.cache-size: 134217728
> nfs.trusted-sync: on
> nfs.port: 2049
>
> Interesting is I set it up as a Replica, and then removed the second brick, and it went to distribute.
>
> Gerald
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

More information about the Gluster-users mailing list