[Gluster-users] FW: fix-layout stalls with xattr errors
shylesh
shylesh at gluster.com
Fri Dec 30 07:50:07 UTC 2011
Hi Dan,
I created two bricks both have ext4 file system.
The issue seems to be in fs features that you have disabled.
Formatted the *brick1* with ext4:
root at SERVER1 mnt]# dumpe2fs /dev/sda| grep 'Filesystem features'
dumpe2fs 1.41.12 (17-May-2010)
Filesystem features: has_journal ext_attr resize_inode dir_index
filetype needs_recovery extent flex_bg sparse_super large_file huge_file
uninit_bg dir_nlink extra_isize
Formatted *brick 2* with ext4:
[root at SERVER2 ~]# dumpe2fs /dev/sda| grep 'Filesystem features'
dumpe2fs 1.41.12 (17-May-2010)
Filesystem features: has_journal ext_attr resize_inode dir_index
filetype extent flex_bg sparse_super large_file
As you said i have disabled some of the features from *brick2*.
I created a distribute volume with these two bricks. created some files
on the mount point and tried setting xattr for these files.
I got error messages
=======================================================================================
[2011-12-30 01:57:22.551634] I
[client3_1-fops.c:818:client3_1_setxattr_cbk] 1-test-client-1: remote
operation failed: Operation not supported
[2011-12-30 01:57:22.551658] W [fuse-bridge.c:850:fuse_err_cbk]
0-glusterfs-fuse: 201305: SETXATTR() /92 => -1 (Operation not supported)
[2011-12-30 01:57:22.556490] I
[client3_1-fops.c:818:client3_1_setxattr_cbk] 1-test-client-1: remote
operation failed: Operation not supported
[2011-12-30 01:57:22.556520] W [fuse-bridge.c:850:fuse_err_cbk]
0-glusterfs-fuse: 201311: SETXATTR() /95 => -1 (Operation not supported)
[2011-12-30 01:57:22.564089] I
[client3_1-fops.c:818:client3_1_setxattr_cbk] 1-test-client-1: remote
operation failed: Operation not supported
[2011-12-30 01:57:22.564114] W [fuse-bridge.c:850:fuse_err_cbk]
0-glusterfs-fuse: 201321: SETXATTR() /100 => -1 (Operation not supported)
========================================================================================
where as i created another volume with only *brick1* and everything
went smoothly.
so i suspect problem is not with rebalance but with ext4 features that
are disabled on *brick2*.
Please let me know if i am missing anything that can be tried.
Thanks,
Shylesh
> ------------------------------------------------------------------------
> *From:* gluster-users-bounces at gluster.org
> [gluster-users-bounces at gluster.org] on behalf of Dan Bretherton
> [d.a.bretherton at reading.ac.uk]
> *Sent:* Thursday, December 29, 2011 6:05 AM
> *To:* gluster-users
> *Subject:* [Gluster-users] fix-layout stalls with xattr errors
>
> Hello All-
> I am having problems with rebalance ... fix-layout in version 3.2.5.
> I extended a volume with add-brick but the fix-layout stalls after a
> small number of layout fixes and does not make any more progress. I
> have tried the operation twice on different servers with the same
> result. The following errors are found in the fuse mount log file on
> the server carrying out the operation.
>
> [2011-12-28 21:38:14.840013] I
> [afr-common.c:1038:afr_launch_self_heal] 0-nemo2-replicate-4:
> background data self-heal triggered. path:
> /users/hzu/DATA/ERAINT/ORCA025/2010/snow_ERAINT_2010.nc
> [2011-12-28 21:38:15.93079] E
> [client3_1-fops.c:1498:client3_1_fxattrop_cbk] 0-nemo2-client-8:
> remote operation failed: Operation not supported
> [2011-12-28 21:38:15.93141] E
> [client3_1-fops.c:1498:client3_1_fxattrop_cbk] 0-nemo2-client-9:
> remote operation failed: Operation not supported
> [2011-12-28 21:38:15.93385] I
> [client3_1-fops.c:1187:client3_1_fstat_cbk] 0-nemo2-client-8:
> remote operation failed: Operation not supported
> [2011-12-28 21:38:15.93521] I
> [client3_1-fops.c:1187:client3_1_fstat_cbk] 0-nemo2-client-9:
> remote operation failed: Operation not supported
>
>
> The file in the error message is a link, and it is not broken as seen
> from the volume mount point or the bricks.
>
> There are some worrying error messages in the brick log files for
> nemo2-client-8 and nemo2-client-9. Here are some exerpts from the
> nemo2-client-8 log, which is similar to the 0-nemo2-client-9 log.
>
> [2011-12-28 21:23:05.827877] W [posix.c:3928:do_xattrop]
> 0-nemo2-posix: Extended attributes not supported by filesystem
> [2011-12-28 21:23:05.827932] I
> [server3_1-fops.c:1705:server_fxattrop_cbk] 0-nemo2-server: 8438:
> FXATTROP 0 (-2111276040) ==> -1 (Operation not support
> ed)
> [2011-12-28 21:23:05.828848] E [posix.c:4200:posix_fstat]
> 0-nemo2-posix: fstat failed on fd=0x2aaaac703804: Operation not
> supported
> [2011-12-28 21:23:05.828879] I
> [server3_1-fops.c:1113:server_fstat_cbk] 0-nemo2-server: 8439:
> FSTAT 0 (-2111276040) ==> -1 (Operation not supported)
> [2011-12-28 21:29:29.871213] W
> [socket.c:1494:__socket_proto_state_machine] 0-tcp.nemo2-server:
> reading from socket failed. Error (Transport endpoint i
> s not connected), peer (192.171.166.81:1003)
> [2011-12-28 21:29:29.871305] I
> [server-helpers.c:360:do_lock_table_cleanup] 0-nemo2-server:
> inodelk released on /users/hzu/DATA/ERAINT/ORCA025/2010/sno
> w_ERAINT_2010.nc
> [2011-12-28 21:29:29.871345] I
> [server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
> /users/hzu/DATA/ERAINT/ORCA025/2010/snow_ERAINT_2010.
> nc
>
> [2011-12-28 21:34:36.190023] I
> [server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on /
> [2011-12-28 21:34:36.190055] I
> [server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
> /users
> [2011-12-28 21:34:36.190086] I
> [server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
> /users/hzu
> [2011-12-28 21:34:36.190102] I
> [server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
> /users/hzu/DATA
> [2011-12-28 21:34:36.190135] I
> [server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
> /users/hzu/DATA/ERAINT
> [2011-12-28 21:34:36.190154] I
> [server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
> /users/hzu/DATA/ERAINT/ORCA025
> [2011-12-28 21:34:36.190171] I
> [server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
> /users/hzu/DATA/ERAINT/ORCA025/2009
>
> [2011-12-28 21:38:15.92433] I
> [server3_1-fops.c:1705:server_fxattrop_cbk] 0-nemo2-server: 12228:
> FXATTROP 7 (-2111276040) ==> -1 (Operation not supported)
> [2011-12-28 21:38:15.92743] E [posix.c:4200:posix_fstat]
> 0-nemo2-posix: fstat failed on fd=0x2aaaac703804: Operation not
> supported
> [2011-12-28 21:38:15.92775] I
> [server3_1-fops.c:1113:server_fstat_cbk] 0-nemo2-server: 12229:
> FSTAT 7 (-2111276040) ==> -1 (Operation not supported)
>
>
> The backend filesystems are ext4 and the are mounted with options
> "acl,user_xattr". I tested extended attribute support (as suggested
> here:
> http://gluster.org/pipermail/gluster-users/2010-December/006257.html)
> and could not find any problems, so I don't understand the "Extended
> attributes not supported by filesystem" error. The only unusual thing
> about the filesystems is the reduced number of filesystem features
> enabled compared to other bricks. These are the ext4 features enabled.
>
> has_journal ext_attr resize_inode dir_index filetype needs_recovery
> sparse_super large_file
>
> All the other bricks in the volume have these features plus extent,
> flex_bg, huge_file, uninit_bg, dir_nlink and extra_isize. I don't
> know if any of these missing ext4 features are part of the problem.
> Does anybody know what's going on here?
>
> Regards
> Dan.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20111230/974cbf96/attachment.html>
More information about the Gluster-users
mailing list