[Bugs] [Bug 1271757] New: add policy mechanism for promotion and demotion

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1271757

            Bug ID: 1271757
           Summary: add policy mechanism for promotion and demotion
           Product: Red Hat Gluster Storage
           Version: 3.1
         Component: glusterfs-devel
          Assignee: chrisw at redhat.com
          Reporter: dlambrig at redhat.com
        QA Contact: storage-qa-internal at redhat.com
                CC: bugs at gluster.org, nlevinki at redhat.com,
                    rhs-bugs at redhat.com, rwheeler at redhat.com,
                    vagarwal at redhat.com, vbellur at redhat.com
        Depends On: 1257911
            Blocks: 1260923, 1270527



+++ This bug was initially created as a clone of Bug #1257911 +++

Current mechanism for driving promotion/demotion is too rigorous. Fils are
demoted automatically and the hot tier will be underutilized. Need a more
flexible system that can be tuned according to customers needs according to
various parameters. Need to gracefully handle situations where the hot tier
becomes full.

--- Additional comment from Anand Avati on 2015-08-28 07:39:39 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: WIP add watermarks and
policy driver) posted (#1) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Anand Avati on 2015-08-28 16:53:20 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: WIP add watermarks and
policy driver) posted (#2) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Anand Avati on 2015-09-01 12:31:59 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: WIP add watermarks and
policy driver) posted (#3) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Vijay Bellur on 2015-09-03 17:30:17 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: WIP add watermarks and
policy driver) posted (#4) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Vijay Bellur on 2015-09-08 17:01:37 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: WIP add watermarks and
policy driver) posted (#5) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Vijay Bellur on 2015-09-10 17:56:30 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: add watermarks and
policy driver) posted (#6) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Vijay Bellur on 2015-09-11 20:37:14 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: add watermarks and
policy driver) posted (#7) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Vijay Bellur on 2015-09-18 01:29:54 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: add watermarks and
policy driver) posted (#8) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Vijay Bellur on 2015-10-09 20:46:29 EDT ---

REVIEW: http://review.gluster.org/12039 (cluster/tier: add watermarks and
policy driver) posted (#24) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Vijay Bellur on 2015-10-10 09:00:08 EDT ---

COMMIT: http://review.gluster.org/12039 committed in master by Dan Lambright
(dlambrig at redhat.com) 
------
commit 672baab88fb7f32e844cd4be22e0924e4e0e83fc
Author: Dan Lambright <dlambrig at redhat.com>
Date:   Fri Sep 18 00:49:06 2015 -0400

    cluster/tier: add watermarks and policy driver

    This fix introduces infrastructure to support different
    policies for promotion and demotion.

    Currently the tier feature automatically promotes and demotes
    files periodically based on access. This is good for testing
    but too stringent for most real workloads. It makes it
    difficult to fully utilize a hot tier- data will be demoted
    before it is touched- its unlikely a 100GB hot SSD will have
    all its data touched in a window of time.

    A new parameter "mode" allows the user to pick promotion/demotion
    polcies.

    The "test mode" will be used for *.t and other general testing.
    This is the current mechanism.

    The "cache mode" introduces watermarks. The watermarks
    represent levels of data residing on the hot tier.

    "cache mode" policy:

    The % the hot tier is full is called P.

    Do not promote or demote more than D MB or F files.

    A random number [0-100] is called R.

    Rules for migration:

    if (P < watermark_low) don't demote, always promote.

    if (P >= watermark_low) && (P < watermark_hi) demote if R < P; promote if R
> P.

    if (P > watermark_hi) always demote, don't promote.

    gluster volume set {vol} cluster.watermark-hi %
    gluster volume set {vol} cluster.watermark-low %
    gluster volume set {vol} cluster.tier-max-mb {D}
    gluster volume set {vol} cluster.tier-max-files {F}
    gluster volume set {vol} cluster.tier-mode {test|cache}

    Change-Id: I157f19667ec95aa1d53406041c1e3b073be127c2
    BUG: 1257911
    Signed-off-by: Dan Lambright <dlambrig at redhat.com>
    Reviewed-on: http://review.gluster.org/12039
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Atin Mukherjee <amukherj at redhat.com>


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1257911
[Bug 1257911] add policy mechanism for promotion and demotion
https://bugzilla.redhat.com/show_bug.cgi?id=1260923
[Bug 1260923] Tracker for tiering in 3.1.2
https://bugzilla.redhat.com/show_bug.cgi?id=1270527
[Bug 1270527] add policy mechanism for promotion and demotion
-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=8d6kF7X9gW&a=cc_unsubscribe