[Bugs] [Bug 1278391] New: Data Tiering:Files not getting migrated from hot to cold when the hot brick is full

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1278391

            Bug ID: 1278391
           Summary: Data Tiering:Files not getting migrated from hot to
                    cold when the hot brick is full
           Product: Red Hat Gluster Storage
           Version: 3.1
         Component: glusterfs
     Sub Component: tiering
          Severity: urgent
          Priority: urgent
          Assignee: rhs-bugs at redhat.com
          Reporter: nchilaka at redhat.com
        QA Contact: nchilaka at redhat.com
                CC: bugs at gluster.org, dlambrig at redhat.com,
                    nbalacha at redhat.com, vagarwal at redhat.com
        Depends On: 1271151
            Blocks: 1260923



+++ This bug was initially created as a clone of Bug #1271151 +++

Description of problem:
=======================
When a brick is full, files are not getting migrated at all.
I created a hot tier with 2x2 with each brick of 1GB size. That means
effectively the hot tier is of 2GB size.
I started to create a file of 1.5GB using dd command, which was getting created
on 1st brick. But as the 1GB gets filled, the new file stops to grow with below
error:
dd: error writing ‘zerobyte.50’: Input/output error
dd: closing output file ‘zerobyte.50’: Input/output error

Now the file is not getting demoted at all and hence no more writes are going
to the hot tier.

Similarly, if i fill the other brick too, then it essentially means that the
whole hot tier is full.

In this case, no more new creates are being allowed even to the cold tier (also
refer bz#1259312)

Version-Release number of selected component (if applicable):
===========================================================
glusterfs-server-3.7.5-0.18.git2def6bb.el7.centos.x86_64






Steps to Reproduce:
==================
1.create a 2x2 vol and start it
2.attach a tier of 2x2 with limited size, say each brick of 1GB.
Also turn on ctr (note in this case quota was not turned on)
3.Now create a file of 1.5GB such that it goes to one of the hot bricks
It will fail after it creates 1GB
4. Now this file doesnt get demoted at all 
5.Now again create a 1.5GB file such that it goes to the other hot brick(to
make hot layer full)
5. Now file doenst get demoted at all



Expected results:
================
File must get demoted once it is old and idle

--- Additional comment from nchilaka on 2015-10-13 05:39:20 EDT ---

vol name:estonia



sosreports:rhsqe-repo.lab.eng.blr.redhat.com:/home/repo/sosreports/bug.1271151

--- Additional comment from nchilaka on 2015-10-13 06:43 EDT ---



--- Additional comment from Dan Lambright on 2015-10-29 17:01:33 EDT ---

DHT does not accept creation of files beyond the brick size. So I am not sure
if we can fix the test the way it is structured, but nithya can confirm.

In practice the hot brick would be 10s or 100s of GB. A file created would be a
fraction of that size. As files are created on the hot tier it would become
more full, until eventually a watermarked was crossed. At that time files would
be created on the cold tier.


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1260923
[Bug 1260923] Tracker for tiering in 3.1.2
https://bugzilla.redhat.com/show_bug.cgi?id=1271151
[Bug 1271151] Data Tiering:Files not getting migrated from hot to cold when
the hot brick is full
-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=Bjqummmhx3&a=cc_unsubscribe