[Gluster-devel] tiering: emergency demotions

[Milind Changire]

Emergency demotions will be required whenever writes breach the
hi-watermark. Emergency demotions are required to avoid ENOSPC in case
of continuous writes that originate on the hot tier.

There are two concerns in this area:

1. enforcing max-cycle-time during emergency demotions
    max-cycle-time is the time the tiering daemon spends in promotions or
    demotions
    I tend to think that the tiering daemon skip this check for the
    emergency situation and continue demotions until the watermark drops
    below the hi-watermark

2. file demotion policy
    I tend to think that evicting the largest file with the most recent
    *write* should be chosen for eviction when write-freq-threshold is
    NON-ZERO.
    Choosing a least written file is just going to delay file migration
    of an active file which might consume hot tier disk space resulting
    in a ENOSPC, in the worst case.
    In cases where write-freq-threshold are ZERO, the most recently
    *written* file can be chosen for eviction.
    In the case of choosing the largest file within the
    write-freq-threshold, a stat() on the files would be required to
    calculate the number of files that need to be demoted to take the
    watermark below the hi-watermark. Finding the number of most recently
    written files to demote could also help make demotions in parallel
    rather than in the sequential manner currently in place.

Comments are requested.

-- 
Milind