[Gluster-users] Blocking IO when hot tier promotion daemon runs
hgowtham at redhat.com
Wed Jan 10 03:33:42 UTC 2018
Can you send the volume info, and volume status output and the tier logs.
And I need to know the size of the files that are being stored.
On Tue, Jan 9, 2018 at 9:51 PM, Tom Fite <tomfite at gmail.com> wrote:
> I've recently enabled an SSD backed 2 TB hot tier on my 150 TB 2 server / 3
> bricks per server distributed replicated volume.
> I'm seeing IO get blocked across all client FUSE threads for 10 to 15
> seconds while the promotion daemon runs. I see the 'glustertierpro' thread
> jump to 99% CPU usage on both boxes when these delays occur and they happen
> every 25 minutes (my tier-promote-frequency setting).
> I suspect this has something to do with the heat database in sqlite, maybe
> something is getting locked while it runs the query to determine files to
> promote. My volume contains approximately 18 million files.
> Has anybody else seen this? I suspect that these delays will get worse as I
> add more files to my volume which will cause significant problems.
> Here are my hot tier settings:
> # gluster volume get gv0 all | grep tier
> cluster.tier-pause off
> cluster.tier-promote-frequency 1500
> cluster.tier-demote-frequency 3600
> cluster.tier-mode cache
> cluster.tier-max-promote-file-size 10485760
> cluster.tier-max-mb 64000
> cluster.tier-max-files 100000
> cluster.tier-query-limit 100
> cluster.tier-compact on
> cluster.tier-hot-compact-frequency 86400
> cluster.tier-cold-compact-frequency 86400
> # gluster volume get gv0 all | grep threshold
> cluster.write-freq-threshold 2
> cluster.read-freq-threshold 5
> # gluster volume get gv0 all | grep watermark
> cluster.watermark-hi 92
> cluster.watermark-low 75
> Gluster-users mailing list
> Gluster-users at gluster.org
More information about the Gluster-users