[Gluster-devel] Upcall state + Data Tiering

[Niels de Vos]

On Thu, Apr 16, 2015 at 04:58:29PM +0530, Soumya Koduri wrote:
> Hi Dan/Joseph,
> 
> As part of upcall support on the server-side, we maintain certain state to
> notify clients of the cache-invalidation and recall-leaselk events.
> 
> We have certain known limitations with Rebalance and Self-Heal. Details in
> the below link -
> http://www.gluster.org/community/documentation/index.php/Features/Upcall-infrastructure#Limitations
> 
> In case of Cache-invalidation,
> upcall state is not migrated and once the rebalance is finished, the file is
> deleted and we may falsely notify the client that file is deleted when in
> reality it isn't.
> 
> In case of Lease-locks,
> As the case with posix locks, we do not migrate lease-locks as well but will
> end-up recalling lease-lock.
> 
> Here rebalance is an admin driven job, but that is not the case with respect
> to Data tiering.
> 
> We would like to know when the files are moved from hot to cold tiers or
> vice-versa or rather when a file is considered to be migrated from cold to
> hot tier, where we see potential issues.
> Is it the first fop which triggers it? and
> where are the further fops processed - on hot tier or cold tier?

My understanding is the following:

- when a file is "cold" and gets accessed, the 1st FOP will mark the
  file for migration to the "hot" tier
- migration is async, so the initial responses on FOPs would come from
  the "cold" tier
- upon migration (similar to rebalance) locking state and upcall
  tracking is lost

I think this is a problem. There seems to be a window where a client can
get (posix) locks while the file is on the "cold" tier. After migrating
the file from "cold" to "hot", these locks would get lost. The same
counts for tracking access in the upcall xlator.

> Please provide your inputs on the same. We may need to document the same or
> provide suggestions to the customers while deploying this solution.

Some ideas on how this can get solved would be most welcome.

Thanks,
Niels
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150419/da8700db/attachment.sig>