[Gluster-users] Testing sharding on tiered volume

[Ben Turner]

----- Original Message -----
> From: "Viktor Nosov" <vnosov at stonefly.com>
> To: gluster-users at gluster.org
> Cc: vnosov at stonefly.com
> Sent: Friday, December 8, 2017 5:45:25 PM
> Subject: [Gluster-users] Testing sharding on tiered volume
> 
> Hi,
> 
> I'm looking to use sharding on tiered volume. This is very attractive
> feature that could benefit tiered volume to let it handle larger files
> without hitting the "out of (hot)space problem".
> I decided to set  test configuration on GlusterFS 3.12.3  when tiered volume
> has 2TB cold and 1GB hot segments. Shard size is set to be 16MB.
> For testing 100GB files are used. It seems writes and reads are going well.
> But I hit problem trying to delete files from the volume. One of GlusterFS
> processes hit segmentation fault.
> The problem is reproducible each time.  It was submitted to Red Hat Bugzilla
> bug list and has ID 1521119.
> You can find details at the attachments to the bug.
> 
> I'm wondering are there other users who are interested to apply sharding to
> tiered volumes and are experienced similar problems?
>  How  this problem can be resolved or could it be avoided?

This isn't a config I have tried before, from the BZ it mentions:

-The VOL is shared out over SMB to a windows client
-You have a 1GB hot tier, 2099GB cold tier
-You have features.shard-block-size: 16MB and cluster.tier-demote-frequency: 150

What are you using for the hot tier that has only 1GB, some sort of RAM disk or battery back flash or something?  

With that small of a hot tier you may run into some strange performance characteristics, AFAIK the current tiering implementation uses rebalance to move files between tiers when the tier demote freq times out.  You may end up spending alot of time waiting for your hot files to rebalance to the cold tier since its out of space, you will also probably have other files being written to the cold tier with the hot tier full, further using up your IOPs.  

I don't know how tiering would treat sharded files, would it only promote the shards of the file that are in use or would it try to put the whole file / all the shards on the hot tier?  

If you get a free min update me on what you are trying todo, happy to help however I can.

-b


> 
> Best regards,
> 
>  Viktor Nosov
> 
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>