[Gluster-devel] gluster rebalance progress

[Emmanuel Dreyfus]

Emmanuel Dreyfus <manu at netbsd.org> wrote:

> Theses two alternative can also be done in storage/posix xlator, rather
> that in cluster/dht, so that .attribute become just absent from any
> client view. In fact I think this is better to do it in storage/posix
> than in cluster/dht, since the same problem may happen later with other
> xlators. 

storage/posix seems the right place, since there is already some code to hide
the replica trash here. I made the patch attached at the end of this e-mail
but got no real progress. I suspect I will have to delete all extended
attributes and start over. Here are the logs:

 I [afr-common.c:836:afr_fresh_lookup_cbk] 0-gfs1-replicate-0: added root
inode
 I [afr-common.c:836:afr_fresh_lookup_cbk] 0-gfs1-replicate-1: added root
inode
 W [afr-common.c:634:afr_lookup_self_heal_check] 0-gfs1-replicate-0: /test:
gfid different on subvolume
 W [afr-common.c:634:afr_lookup_self_heal_check] 0-gfs1-replicate-0: /test:
gfid different on subvolume
 W [dht-common.c:177:dht_lookup_dir_cbk] 0-gfs1-dht: /test: gfid different on
gfs1-replicate-0
 I [client3_1-fops.c:411:client3_1_stat_cbk] 0-gfs1-client-2: remote operation
failed: No such file or directory
 I [client3_1-fops.c:411:client3_1_stat_cbk] 0-gfs1-client-3: remote operation
failed: No such file or directory
 I [client3_1-fops.c:2132:client3_1_opendir_cbk] 0-gfs1-client-2: remote
operation failed: No such file or directory
 I [client3_1-fops.c:2132:client3_1_opendir_cbk] 0-gfs1-client-0: remote
operation failed: No such file or directory
 I [client3_1-fops.c:2132:client3_1_opendir_cbk] 0-gfs1-client-3: remote
operation failed: No such file or directory
 W [client3_1-fops.c:5041:client3_1_readdir] 0-gfs1-client-0: (13045760):
failed to get fd ctx. EBADFD
 W [client3_1-fops.c:5106:client3_1_readdir] 0-gfs1-client-0: failed to send
the fop: Bad file descriptor
 I [afr-dir-read.c:120:afr_examine_dir_readdir_cbk] 0-gfs1-replicate-0: /test:
failed to do opendir on gfs1-client-0
 I [afr-dir-read.c:174:afr_examine_dir_readdir_cbk] 0-gfs1-replicate-0:  entry
self-heal triggered. path: /test, reason: checksums of directory differ,
forced merge option set
 I [client3_1-fops.c:1303:client3_1_entrylk_cbk] 0-gfs1-client-0: remote
operation failed: No such file or directory
 I [afr-self-heal-entry.c:2265:afr_sh_post_nonblocking_entry_cbk]
0-gfs1-replicate-0: Non Blocking entrylks failed.
 W [afr-common.c:110:afr_set_split_brain]  0-gfs1-replicate-0: invalid
argument: inode
 I [client3_1-fops.c:411:client3_1_stat_cbk] 0-gfs1-client-2: remote operation
failed: No such file or directory
 I [client3_1-fops.c:411:client3_1_stat_cbk] 0-gfs1-client-3: remote operation
failed: No such file or directory
 W [client3_1-fops.c:5158:client3_1_readdirp] 0-gfs1-client-0: (13045760):
failed to get fd ctx. EBADFD
 W [client3_1-fops.c:5222:client3_1_readdirp] 0-gfs1-client-0: failed to send
the fop: Bad file descriptor
 W [client3_1-fops.c:5158:client3_1_readdirp] 0-gfs1-client-2: (13045760):
failed to get fd ctx. EBADFD
 W [client3_1-fops.c:5222:client3_1_readdirp] 0-gfs1-client-2: failed to send
the fop: Bad file descriptor

--- xlators/storage/posix/src/posix.c.orig 
+++ xlators/storage/posix/src/posix.c 
@@ -4094,8 +4094,21 @@
 
                 if ((!strcmp(real_path, base_path))
                     && (!strcmp(entry->d_name, GF_REPLICATE_TRASH_DIR)))
                         continue;
+#ifdef __NetBSD__
+               /*
+                * NetBSD with UFS1 backend uses backing files for
+                * extended attributes. They can be found in a
+                * .attribute file located at the root of the filesystem
+                * We hide it to glusterfs clients, since chaos will occur 
+                * when the cluster/dht xlator decides to distribute
+                * exended attribute backing file accross storage servers.
+                */ 
+                if ((!strcmp(real_path, base_path))
+                    && (!strcmp(entry->d_name, ".attribute")))
+                        continue;
+#endif /* __NetBSD__ */
 
                 if ((!strcmp (real_path, base_path))
                     && (!strncmp (GF_HIDDEN_PATH, entry->d_name,
                                   strlen(GF_HIDDEN_PATH)))) {

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu at netbsd.org