<div dir="ltr"><div dir="ltr">Thank you Hari, <div><br></div><div>I have set:</div><div><div>cluster.tier-promote-frequency: 1800</div><div>cluster.tier-demote-frequency: 120</div></div><div><br></div><div>I will let you know if it makes a difference after some time. So far (10 minutes), nothing has changed. </div><div>I would agree with you, that by looking at the result of 'gluster volume tier FFPrimary status' it would seem that demoting is happening. However, for the last 24hrs, nothing has changed in the tier status report except the time. Could it be stuck? How would I know? Is there a way to restart it without restarting the cluster?</div><div><br></div><div><br></div><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Sat, Sep 29, 2018 at 11:08 AM Hari Gowtham <<a href="mailto:hgowtham@redhat.com">hgowtham@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
I can see that the demotion is happening from the status provided by you.<br>
Do verify it.<br>
I would recommend you to change the cluster.tier-demote-frequency to 120<br>
and cluster.tier-promote-frequency to 1800 to increase the demotions until the<br>
hot tier is emptied to a certain extent. Later you can use the values<br>
existing now.<br>
On Sat, Sep 29, 2018 at 5:39 PM David Brown <<a href="mailto:dbccemtp@gmail.com" target="_blank">dbccemtp@gmail.com</a>> wrote:<br>
><br>
> Hey Everyone,<br>
><br>
> I have a 3 node GlusterFS cluster that uses NVMe hot tier and a HDD cold tier.<br>
> I recently ran into some problems when the hot tier became full with df-h showing 100%.<br>
><br>
> I did not have a watermark-hi set, but it is my understanding that 90% is the default. In an attempt to get the cluster to demote some files, I set cluster.watermark-hi: 80 but it is still not demoting.<br>
><br>
><br>
> [root@Glus1 ~]# gluster volume info<br>
><br>
> Volume Name: FFPrimary<br>
> Type: Tier<br>
> Volume ID: 466ec53c-d1ef-4ebc-8414-d7d070dfe61e<br>
> Status: Started<br>
> Snapshot Count: 0<br>
> Number of Bricks: 9<br>
> Transport-type: tcp<br>
> Hot Tier :<br>
> Hot Tier Type : Replicate<br>
> Number of Bricks: 1 x 3 = 3<br>
> Brick1: Glus3:/data/glusterfs/FFPrimary/brick3<br>
> Brick2: Glus2:/data/glusterfs/FFPrimary/brick2<br>
> Brick3: Glus1:/data/glusterfs/FFPrimary/brick1<br>
> Cold Tier:<br>
> Cold Tier Type : Distributed-Replicate<br>
> Number of Bricks: 2 x 3 = 6<br>
> Brick4: Glus1:/data/glusterfs/FFPrimary/brick5<br>
> Brick5: Glus2:/data/glusterfs/FFPrimary/brick6<br>
> Brick6: Glus3:/data/glusterfs/FFPrimary/brick7<br>
> Brick7: Glus1:/data/glusterfs/FFPrimary/brick8<br>
> Brick8: Glus2:/data/glusterfs/FFPrimary/brick9<br>
> Brick9: Glus3:/data/glusterfs/FFPrimary/brick10<br>
> Options Reconfigured:<br>
> cluster.tier-promote-frequency: 120<br>
> cluster.tier-demote-frequency: 1800<br>
> cluster.watermark-low: 60<br>
> cluster.watermark-hi: 80<br>
> performance.flush-behind: on<br>
> performance.cache-max-file-size: 128MB<br>
> performance.cache-size: 25GB<br>
> diagnostics.count-fop-hits: off<br>
> diagnostics.latency-measurement: off<br>
> cluster.tier-mode: cache<br>
> features.ctr-enabled: on<br>
> transport.address-family: inet<br>
> nfs.disable: on<br>
> performance.client-io-threads: off<br>
> [root@Glus1 ~]# gluster volume tier FFPrimary status<br>
> Node Promoted files Demoted files Status run time in h:m:s<br>
> --------- --------- --------- --------- ---------<br>
> localhost 49 0 in progress 5151:30:45<br>
> Glus2 0 0 in progress 5151:30:45<br>
> Glus3 0 2075 in progress 5151:30:47<br>
> Tiering Migration Functionality: FFPrimary: success<br>
> [root@Glus1 ~]#<br>
><br>
> What can cause GlusterFS to stop demoting files and allow it to completely fill the Hot Tier?<br>
><br>
> Thank you!<br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
> <a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
<br>
<br>
<br>
-- <br>
Regards,<br>
Hari Gowtham.<br>
</blockquote></div>