[Gluster-devel] Data Classification: would it be possible to have a RAM-disk as caching tier?

[Joseph Fernandes]

JOE >> reply inline

----- Original Message -----
From: "Niels de Vos" <ndevos at redhat.com>
To: "Joseph Fernandes" <josferna at redhat.com>
Cc: gluster-devel at gluster.org, "Dan Lambright" <dlambrig at redhat.com>
Sent: Monday, February 16, 2015 4:41:20 PM
Subject: Re: Data Classification: would it be possible to have a RAM-disk as caching tier?

On Mon, Feb 16, 2015 at 05:54:15AM -0500, Joseph Fernandes wrote:
> Hi Niels,
> 
> Well the idea is good, RAM-Disk would fastest and with no extra cost +
> We may have gluster brick from RAM-Disk[1]
> The one and the biggest challenge would be durability of data on RAM-Disks
> Using RAM for caching is good, as the cache will have only the copy of the original data
> but in case of tiering, the original data sits on the tier(not the copy).

Thanks for the swift resonse!

I imagine a solution where the hot contents are not moved to the
RAM-disk, but are replicated on demand. When the contents gets cold, the
replication can be reduced again, which would make space on the
RAM-disk. Upon boot, the RAM-disk would be empty, and only the
hot-contents would need to get 'healed' onto the RAM-disk. (The fastest
brick of a replica pair handles the reads, I assume that this would be
the RAM-disk.)

JOE >>
Looks Interesting and might be a good idea!
Few points to be careful of,
1) Network usage for AFR Self heals required to have the RAM Disk and Actual Disk,
 When Bricks are owned by multiple nodes.

2) This kind of replica pair should be marked separately from the regular AFR replica, say "cache-replica"
   and should work like current tiering implemetation, i.e whenever there is a cache-replica miss
   on the HOT Replica instead of marking it a bad replica move data intelligently(using heat patterns)
   to the HOT replica(Promote) from the cold replica.

Well these are the things I can see for now, but the "devil is the details" :)

At the moment I can not say if the current data-classification proposal
is flexible enough to configure a policy like this. Or, how easy it
would be to extend the feature to allow configurations like this as an
improvement later on.

JOE >> The current implementation this wouldnt be possible as our implementation
sits above DHT layer. For the purposed "cache-replica" we might need to change AFR code(for current replication).
 

Cheers,
Niels


> 
> Dan your thoughts.
> 
> Thanks,
> Joe
> 
> 1. https://lists.gnu.org/archive/html/gluster-devel/2013-05/msg00118.html
> 
> ----- Original Message -----
> From: "Niels de Vos" <ndevos at redhat.com>
> To: gluster-devel at gluster.org
> Cc: "Dan Lambright" <dlambrig at redhat.com>, "Joseph Fernandes" <josferna at redhat.com>
> Sent: Monday, February 16, 2015 4:14:37 PM
> Subject: Data Classification: would it be possible to have a RAM-disk as caching tier?
> 
> Hi guys,
> 
> at FOSDEM one of our users spoke to me about their deployment and
> environment. It seems that they have a *very* good deal with their
> hardware vendor, which makes it possible to stuff their servers full
> with RAM for a minimal difference of the costs.
> 
> They expressed interest in having a RAM-disk as caching tier on the
> bricks. Would a configuration like this be possible with the
> data-classification feature [1]?
> 
> Thanks,
> Niels
> 
> 1. http://www.gluster.org/community/documentation/index.php/Features/data-classification