[Bugs] [Bug 1217445] New: data tiering: tiering core functionality Data heating/cooling not working on a tiered volume

Thu Apr 30 11:45:58 UTC 2015

https://bugzilla.redhat.com/show_bug.cgi?id=1217445

            Bug ID: 1217445
           Summary: data tiering: tiering core functionality Data
                    heating/cooling not working on a tiered volume
           Product: GlusterFS
           Version: mainline
         Component: tiering
          Severity: urgent
          Assignee: bugs at gluster.org
          Reporter: nchilaka at redhat.com
        QA Contact: bugs at gluster.org
                CC: bugs at gluster.org

Description of problem:
======================
Seems like data is not getting heated or cooled after setting volume options on
tiered volume.
i have tested this with both fresh tiered volume and an existing vol converted
to tier.

I have set volume options as below
Options Reconfigured:
cluster.tier-demote-frequency: 10
features.ctr-enabled: 1

A new file gets created in hot tier as expected but even after 10min(even
though demote is in secs) the file still remains in hot tier-doesnt get
demoted. I have not accessed file in any manner in that 10min

Version-Release number of selected component (if applicable):
============================================================
[root at yarrow ~]# gluster --version
glusterfs 3.7.0alpha0 built on Apr 28 2015 01:37:11
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General
Public License.
[root at yarrow ~]# rpm -qa|grep gluster
glusterfs-fuse-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-libs-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-cli-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-server-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-api-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64

Steps to Reproduce:
===================
1.create a tiered vol
2.set options like features.ctr-enabled: to on and 
cluster.tier-demote-frequency and cluster.tier-promote-frequency
3.see if data is getting heated/cooled after the set times

Actual results:
===============
data not getting heated or cold

Expected results:
=================
data should get heated or cold and move b/w tiers 

[root at yarrow ~]# gluster v create tiered replica 2 yarrow:/brick_100G_10/tiered
moonshine:/brick_100G_10/tiered  yarrow:/brick_100G_9/tiered
moonshine:/brick_100G_9/tiered
volume create: tiered: success: please start the volume to access data
You have new mail in /var/spool/mail/root
[root at yarrow ~]# gluster v attach-tier moonshine:/ssdbricks_50G_1/tiered
yarrow:/ssdbricks_50G_1/tiered
volume add-brick: failed: Volume moonshine:/ssdbricks_50G_1/tiered does not
exist
[root at yarrow ~]# gluster v attach-tier tiered moonshine:/ssdbricks_50G_1/tiered
yarrow:/ssdbricks_50G_1/tiered
volume add-brick: success
[root at yarrow ~]# gluster v info tiered

Volume Name: tiered
Type: Tier
Volume ID: 5e2977ba-c102-44ad-9b8c-bc95f988df5c
Status: Created
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: yarrow:/ssdbricks_50G_1/tiered
Brick2: moonshine:/ssdbricks_50G_1/tiered
Brick3: yarrow:/brick_100G_10/tiered
Brick4: moonshine:/brick_100G_10/tiered
Brick5: yarrow:/brick_100G_9/tiered
Brick6: moonshine:/brick_100G_9/tiered
[root at yarrow ~]# gluster v info tiered

Volume Name: tiered
Type: Tier
Volume ID: 5e2977ba-c102-44ad-9b8c-bc95f988df5c
Status: Created
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: yarrow:/ssdbricks_50G_1/tiered
Brick2: moonshine:/ssdbricks_50G_1/tiered
Brick3: yarrow:/brick_100G_10/tiered
Brick4: moonshine:/brick_100G_10/tiered
Brick5: yarrow:/brick_100G_9/tiered
Brick6: moonshine:/brick_100G_9/tiered
[root at yarrow ~]# gluster v set tiered features.ctr-enabled sadsa
volume set: failed: option ctr-enabled sadsa: 'sadsa' is not a valid boolean
value
[root at yarrow ~]# gluster v set tiered features.ctr-enabled 1
volume set: success
[root at yarrow ~]# gluster v info tiered

Volume Name: tiered
Type: Tier
Volume ID: 5e2977ba-c102-44ad-9b8c-bc95f988df5c
Status: Created
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: yarrow:/ssdbricks_50G_1/tiered
Brick2: moonshine:/ssdbricks_50G_1/tiered
Brick3: yarrow:/brick_100G_10/tiered
Brick4: moonshine:/brick_100G_10/tiered
Brick5: yarrow:/brick_100G_9/tiered
Brick6: moonshine:/brick_100G_9/tiered
Options Reconfigured:
features.ctr-enabled: 1
[root at yarrow ~]# gluster v set tiered tier-demote-frequency 10
volume set: success
[root at yarrow ~]# gluster v info tiered

Volume Name: tiered
Type: Tier
Volume ID: 5e2977ba-c102-44ad-9b8c-bc95f988df5c
Status: Created
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: yarrow:/ssdbricks_50G_1/tiered
Brick2: moonshine:/ssdbricks_50G_1/tiered
Brick3: yarrow:/brick_100G_10/tiered
Brick4: moonshine:/brick_100G_10/tiered
Brick5: yarrow:/brick_100G_9/tiered
Brick6: moonshine:/brick_100G_9/tiered
Options Reconfigured:
cluster.tier-demote-frequency: 10
features.ctr-enabled: 1
[root at yarrow ~]# gluster v start tiered
volume start: tiered: success
[root at yarrow ~]# gluster v info tiered

Volume Name: tiered
Type: Tier
Volume ID: 5e2977ba-c102-44ad-9b8c-bc95f988df5c
Status: Started
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: yarrow:/ssdbricks_50G_1/tiered
Brick2: moonshine:/ssdbricks_50G_1/tiered
Brick3: yarrow:/brick_100G_10/tiered
Brick4: moonshine:/brick_100G_10/tiered
Brick5: yarrow:/brick_100G_9/tiered
Brick6: moonshine:/brick_100G_9/tiered
Options Reconfigured:
cluster.tier-demote-frequency: 10
features.ctr-enabled: 1
You have new mail in /var/spool/mail/root
[root at yarrow ~]# ls /*br*/tiered
/brick_100G_10/tiered:

/brick_100G_9/tiered:

/ssdbricks_50G_1/tiered:
good
[root at yarrow ~]# ls -l ls /*br*/tiered
ls: cannot access ls: No such file or directory
/brick_100G_10/tiered:
total 0

/brick_100G_9/tiered:
total 0

/ssdbricks_50G_1/tiered:
total 8
-rw-r--r--. 2 root root 15 Apr 30 17:05 good
[root at yarrow ~]# date
Thu Apr 30 17:07:00 IST 2015
[root at yarrow ~]# echo "select * from gf_file_tb; select * from gf_flink_tb;" |
sqlite3 /ssdbricks_50G_1/tiered/.glusterfs/tiered.db
11b2cd42-350a-46d1-b141-f37322b225e7|1430393750|277120|0|0|0|0|0|0|1|1
11b2cd42-350a-46d1-b141-f37322b225e7|00000000-0000-0000-0000-000000000001|good|/good|0|0
[root at yarrow ~]# dae
bash: dae: command not found...
[root at yarrow ~]# date
Thu Apr 30 17:07:55 IST 2015
[root at yarrow ~]# ls -l ls /*br*/tiered
ls: cannot access ls: No such file or directory
/brick_100G_10/tiered:
total 0

/brick_100G_9/tiered:
total 0

/ssdbricks_50G_1/tiered:
total 8
-rw-r--r--. 2 root root 15 Apr 30 17:05 good
You have new mail in /var/spool/mail/root
[root at yarrow ~]# echo "select * from gf_file_tb; select * from gf_flink_tb;" |
sqlite3 /ssdbricks_50G_1/tiered/.glusterfs/tiered.db
11b2cd42-350a-46d1-b141-f37322b225e7|1430393750|277120|0|0|0|0|0|0|1|1
11b2cd42-350a-46d1-b141-f37322b225e7|00000000-0000-0000-0000-000000000001|good|/good|0|0
[root at yarrow ~]# gluster --version
glusterfs 3.7.0alpha0 built on Apr 28 2015 01:37:11
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General
Public License.
[root at yarrow ~]# rpm -qa|grep gluster
glusterfs-fuse-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-libs-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-cli-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-server-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
glusterfs-api-3.7.0alpha0-0.17.gited96153.el7.centos.x86_64
[root at yarrow ~]# gluster v info

Volume Name: newvol
Type: Tier
Volume ID: afa217fd-d255-4343-9ee7-b932d2c2bd47
Status: Started
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: moonshine:/ssdbricks_50G_1/newvoll
Brick2: yarrow:/ssdbricks_50G_1/newvoll
Brick3: moonshine:/brick_100G_10/newvol
Brick4: yarrow:/brick_100G_10/newvol
Brick5: moonshine:/brick_100G_9/newvol
Brick6: yarrow:/brick_100G_9/newvol
Options Reconfigured:
cluster.tier-demote-frequency: 1
features.ctr-enabled: on

Volume Name: tiered
Type: Tier
Volume ID: 5e2977ba-c102-44ad-9b8c-bc95f988df5c
Status: Started
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: yarrow:/ssdbricks_50G_1/tiered
Brick2: moonshine:/ssdbricks_50G_1/tiered
Brick3: yarrow:/brick_100G_10/tiered
Brick4: moonshine:/brick_100G_10/tiered
Brick5: yarrow:/brick_100G_9/tiered
Brick6: moonshine:/brick_100G_9/tiered
Options Reconfigured:
cluster.tier-demote-frequency: 10
features.ctr-enabled: 1
[root at yarrow ~]# echo "select * from gf_file_tb; select * from gf_flink_tb;" |
sqlite3 /ssdbricks_50G_1/tiered/.glusterfs/tiered.db

[root at moonshine newvol]# ls /*br*/tiered
/brick_100G_10/tiered:

/brick_100G_9/tiered:

/ssdbricks_50G_1/tiered:
You have new mail in /var/spool/mail/root
[root at moonshine newvol]# echo "select * from gf_file_tb; select * from
gf_flink_tb;" | sqlite3 /ssdbricks_50G_1/tiered/.glusterfs/tiered.db
[root at moonshine newvol]# 

Additional info:

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.