[Gluster-devel] Feature proposal - FS-Cache support in FUSE

[Dan Lambright]

Hi Vimal,

One suggestion is to consider making the cache read only. A read cache would give you many of the performance benefits, without having to deal with some of the more difficult problems.

* you would not need a background "flush" process to move dirty data from the cache to persistent store.
* no need to distinguish "dirty" data from "clean" data
* the read cache would not need to be persistent, so if power went out, you would be fine.
* the code could be written such that a write cache could exist in the future.

Another suggestion is to look at memcached.org, which is a distributed object system. I know of a company that uses it to implement their caching layer. They use it across multiple clients. Each client sees the same "view" of cached data.

Dan


----- Original Message -----
> From: "Vimal A R" <arvimal at yahoo.in>
> To: fuse-devel at lists.sourceforge.net, linux-cachefs at redhat.com, gluster-devel at gluster.org, ndevos at redhat.com
> Sent: Monday, September 1, 2014 9:07:02 AM
> Subject: [Gluster-devel] Feature proposal - FS-Cache support in FUSE
> 
> Hello fuse-devel / fs-cache / gluster-devel lists,
> 
> I would like to propose the idea of implementing FS-Cache support in the fuse
> kernel module, which I am planning to do as part of my UG university course.
> This proposal is by no means final, since I have just started to look into
> this.
> 
> There are several user-space filesystems which are based on the FUSE kernel
> module. As of now, if I understand correct, the only networked filesystems
> having FS-Cache support are NFS and AFS.
> 
> Implementing support hooks for fs-cache in the fuse module would provide
> networked filesystems such as GlusterFS the benefit of  a client-side
> caching mechanism, which should decrease the access times.
> 
> When enabled, FS-Cache would maintain a virtual indexing tree to cache the
> data or object-types per network FS. Indices in the tree are used by
> FS-Cache to find objects faster. The tree or index structure under the main
> network FS index depends on the filesystem. Cookies are used to represent
> the indices, the pages etc..
> 
> The tree structure would be as following:
> 
> a) The virtual index tree maintained by fs-cache would look like:
> 
> * FS-Cache master index -> The network-filesystem indice (NFS/AFS etc..) ->
> per-share indices -> File-handle indices -> Page indices
> 
> b) In case of FUSE-based filesystems, the tree would be similar to :
> 
> * FS-Cache master index -> FUSE indice -> Per FS indices -> file-handle
> indices -> page indices.
> 
> c) In case of FUSE based filesystems as GlusterFS, the tree would as :
> 
> * FS-Cache master index -> FUSE indice (fuse.glusterfs) -> GlusterFS volume
> ID (a UUID exists for each volume) - > GlusterFS file-handle indices (based
> on the GFID of a file) -> page indices.
> 
> The idea is to enable FUSE to work with the FS-Cache network filesystem API,
> which is documented at
> 'https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/caching/netfs-api.txt'.
> 
> The implementation of FS-Cache support in NFS can be taken as a guideline to
> understand and start off.
> 
> I will reply to this mail with any other updates that would come up whilst
> pursuing this further. I request any sort of feedback/suggestions, ideas,
> any pitfalls etc.. that can help in taking this further.
> 
> Thank you,
> 
> Vimal
> 
> References:
> *
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/caching/fscache.txt
> *
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/caching/netfs-api.txt
> *
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/caching/object.txt
> * http://people.redhat.com/dhowells/fscache/FS-Cache.pdf
> * http://people.redhat.com/steved/fscache/docs/HOWTO.txt
> * https://en.wikipedia.org/wiki/CacheFS
> * https://lwn.net/Articles/160122/
> * http://www.linux-mag.com/id/7378/
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>