[Bugs] [Bug 1219848] New: Directories are missing on the mount point after attaching tier to distribute replicate volume.

Fri May 8 13:14:59 UTC 2015

https://bugzilla.redhat.com/show_bug.cgi?id=1219848

            Bug ID: 1219848
           Summary: Directories are missing on the mount point after
                    attaching tier to distribute replicate volume.
           Product: GlusterFS
           Version: 3.7.0
         Component: tiering
          Keywords: Triaged
          Severity: urgent
          Assignee: bugs at gluster.org
          Reporter: rkavunga at redhat.com
        QA Contact: bugs at gluster.org
                CC: annair at redhat.com, bugs at gluster.org,
                    dlambrig at redhat.com, nchilaka at redhat.com,
                    trao at redhat.com
        Depends On: 1214222
            Blocks: 1186580 (qe_tracker_everglades), 1199352
                    (glusterfs-3.7.0), 1214666

+++ This bug was initially created as a clone of Bug #1214222 +++

Description of problem:
Directories are missing on the mount point after attaching tier to distribute
replicate volume.

Version-Release number of selected component (if applicable):

[root at rhsqa14-vm1 ~]# rpm -qa | grep gluster
glusterfs-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-devel-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-geo-replication-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-resource-agents-3.7dev-0.952.gita7f1d08.el6.noarch
glusterfs-debuginfo-3.7dev-0.952.gita7f1d08.el6.x86_64
glusterfs-libs-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-api-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-fuse-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-extra-xlators-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-regression-tests-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-rdma-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-cli-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-server-3.7dev-0.994.gitf522001.el6.x86_64
glusterfs-api-devel-3.7dev-0.994.gitf522001.el6.x86_64
[root at rhsqa14-vm1 ~]# 

[root at rhsqa14-vm1 ~]# glusterfs --version
glusterfs 3.7dev built on Apr 13 2015 07:14:26
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2013 Red Hat, Inc. <http://www.redhat.com/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
[root at rhsqa14-vm1 ~]# 

How reproducible:

easy

Steps to Reproduce:
1. create a distrep volume
2. fuse mount the volume and create few directories with files.
3. ls -la on mount and keep the record of output.
4. Attach a tier to the volume and execute ls -la on mount point.
5. directories will be missing.

Actual results:

Volume Name: testing
Type: Distributed-Replicate
Volume ID: 42ac4aff-461e-4001-b1c0-f4d42e04452f
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: 10.70.46.233:/rhs/brick1/T4
Brick2: 10.70.46.236:/rhs/brick1/T4
Brick3: 10.70.46.233:/rhs/brick2/T4
Brick4: 10.70.46.236:/rhs/brick2/T4

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_rhsqa14vm5-lv_root
                       18G  3.2G   14G  20% /
tmpfs                 3.8G     0  3.8G   0% /dev/shm
/dev/vda1             477M   33M  419M   8% /boot
10.70.46.233:/testing
                      100G  244M  100G   1% /mnt
10.70.46.233:/mix     199G  330M  199G   1% /mnt1
10.70.46.233:/everglades
                       20G  5.2M   20G   1% /mnt2
10.70.46.233:/Tim      20G  3.3M   20G   1% /mnt3
[root at rhsqa14-vm5 mnt2]# 
[root at rhsqa14-vm5 mnt2]# 
[root at rhsqa14-vm5 mnt2]# 
[root at rhsqa14-vm5 mnt2]# cd /mnt
[root at rhsqa14-vm5 mnt]# ls -la
total 4
drwxr-xr-x.  5 root root  110 Apr 17 02:54 .
dr-xr-xr-x. 28 root root 4096 Apr 22 01:50 ..
drwxr-xr-x.  6 root root  140 Apr 16 06:14 linux-4.0
drwxr-xr-x.  3 root root   48 Apr 16 02:52 .trashcan
[root at rhsqa14-vm5 mnt]# 

[root at rhsqa14-vm1 ~]# gluster v attach-tier testing replica 2
10.70.46.233:/rhs/brick3/mko 10.70.46.236:/rhs/brick3/mko
volume add-brick: success
[root at rhsqa14-vm1 ~]# gluster v info testing

Volume Name: testing
Type: Tier
Volume ID: 42ac4aff-461e-4001-b1c0-f4d42e04452f
Status: Started
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.46.236:/rhs/brick3/mko
Brick2: 10.70.46.233:/rhs/brick3/mko
Brick3: 10.70.46.233:/rhs/brick1/T4
Brick4: 10.70.46.236:/rhs/brick1/T4
Brick5: 10.70.46.233:/rhs/brick2/T4
Brick6: 10.70.46.236:/rhs/brick2/T4
[root at rhsqa14-vm1 ~]# 

[root at rhsqa14-vm5 mnt]# 
[root at rhsqa14-vm5 mnt]# ls -la
total 4
drwxr-xr-x.  5 root root  149 Apr 22  2015 .
dr-xr-xr-x. 28 root root 4096 Apr 22 01:50 ..
drwxr-xr-x.  3 root root   48 Apr 16 02:52 .trashcan
[root at rhsqa14-vm5 mnt]# 

Expected results:
Irrespective of the tiers data must be presented to user.

Additional info:

--- Additional comment from Dan Lambright on 2015-04-22 16:06:20 EDT ---

The problem here, is you did not start the migration daemon.

gluster v rebalance t tier start

This does the "fix layout" to make all directories on all bricks.

You should not have to worry about that. It should be done automatically when
you attach a tier.

We will write a fix for that.

--- Additional comment from Anand Avati on 2015-04-24 06:43:07 EDT ---

REVIEW: http://review.gluster.org/10363 (tiering: Send both attach-tier and
tier-start together) posted (#1) for review on master by mohammed rafi  kc
(rkavunga at redhat.com)

--- Additional comment from Anand Avati on 2015-04-24 06:44:54 EDT ---

REVIEW: http://review.gluster.org/10363 (tiering: Send both attach-tier and
tier-start together) posted (#2) for review on master by mohammed rafi  kc
(rkavunga at redhat.com)

--- Additional comment from Anand Avati on 2015-04-28 02:44:17 EDT ---

REVIEW: http://review.gluster.org/10363 (tiering: Send both attach-tier and
tier-start together) posted (#3) for review on master by mohammed rafi  kc
(rkavunga at redhat.com)

--- Additional comment from Joseph Elwin Fernandes on 2015-05-01 02:57:28 EDT
---

--- Additional comment from Anand Avati on 2015-05-04 07:49:28 EDT ---

REVIEW: http://review.gluster.org/10363 (tiering: Send both attach-tier and
tier-start together) posted (#4) for review on master by mohammed rafi  kc
(rkavunga at redhat.com)

--- Additional comment from Anand Avati on 2015-05-04 11:30:28 EDT ---

REVIEW: http://review.gluster.org/10363 (tiering: Send both attach-tier and
tier-start together) posted (#5) for review on master by mohammed rafi  kc
(rkavunga at redhat.com)

--- Additional comment from Anand Avati on 2015-05-05 01:17:28 EDT ---

REVIEW: http://review.gluster.org/10363 (tiering: Send both attach-tier and
tier-start together) posted (#6) for review on master by mohammed rafi  kc
(rkavunga at redhat.com)

Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1186580
[Bug 1186580] QE tracker bug for Everglades
https://bugzilla.redhat.com/show_bug.cgi?id=1199352
[Bug 1199352] GlusterFS 3.7.0 tracker
https://bugzilla.redhat.com/show_bug.cgi?id=1214222
[Bug 1214222] Directories are missing on the mount point after attaching
tier to distribute replicate volume.
https://bugzilla.redhat.com/show_bug.cgi?id=1214666
[Bug 1214666] Data Tiering:command prompt hangs when fetching quota list of
a tiered volume
-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.