[Gluster-devel] [Gluster-users] PLEASE READ ! We need your opinion. GSOC-2014 and the Gluster community

Tue Mar 18 18:35:45 UTC 2014

On Tue, 2014-03-18 at 12:35 +0530, Kaushal M wrote:
> I had a discussion with some developers here in the office regarding
> this. We created a list of ideas which we thought could be suitable
> for student projects. I've added these to [1]. But I'm also putting
> them on here for more visibility.
> 
> (I've tried to arrange the list in descending order of difficulty as I find it)
> 
> . Glusterd services high availablity
>     Glusterd should restart the processes it manages, bricks, nfs
> server, self-heal daemon & quota daemon, whenever it detects they have
> died.

It might make sense to think about the interplay between this and the
systemd feature set... 

> . glusterfsiostat - Top like utility for glusterfs
>     These are client side tools which will display stats from the
> io-stats translator. I'm not currently sure of the difference between
> the two.
> . ovirt gui for stats
>     Have pretty graphs and tables in ovirt for the GlusterFS top and
> profile commands.
> . monitoring integrations - munin others.
>     The more monitoring support we have for GlusterFS the better.
> . More compression algorithms for compression xlator
>     The onwire compression translator should be extended to support
> more compression algorithms. Ideally it should be pluggable.
> . cinder glusterfs backup driver
>     Write a driver for cinder, a part of openstack, to allow backup
> onto GlusterFS volumes
> . rsockets - sockets for rdma transport
>     Coding for RDMA using the familiar socket api should lead to a
> more robust rdma transport
> . data import tool
>     Create a tool which will allow already importing already existing
> data in the brick directories into the gluster volume. This is most
> likely going to be a special rebalance process.
> . rebalance improvements
>     Improve rebalance preformance.
> . Improve the meta translator
>     The meta xlator provides a /proc like interface to GlusterFS
> xlators. We could further improve this and make it a part of the
> standard volume graph.
> . geo-rep using rest-api
>     This might be suitable for geo replication over WAN. Using
> rsync/ssh over WAN isn't too nice.
> . quota using underlying fs quota
>     GlusterFS quota is currently maintained completely in GlusterFSs
> namespace using xattrs. We could make use of the quota capabilities of
> the underlying fs (XFS) for better performance.
> . snapshot pluggability
>     Snapshot should be able to make use of snapshot support provided
> by btrfs for example.

This would be very useful :)

> . compression at rest
>     Lessons learnt while implementing encryption at rest can be used
> with the compression at rest.
> . file-level deduplication
>     GlusterFS works on files. So why not have dedup at the level files as well.
> . composition xlator for small files
>     Merge smallfiles into a designated large file using our own custom
> semantics. This can improve our small file performance.
> . multi master geo-rep
>     Nothing much to say here. This has been discussed many times.
> 
> Any comments on this list?
> ~kaushal
> 
> [1] http://www.gluster.org/community/documentation/index.php/Projects
> 
> On Tue, Mar 18, 2014 at 9:07 AM, Lalatendu Mohanty <lmohanty at redhat.com> wrote:
> > On 03/13/2014 11:49 PM, John Mark Walker wrote:
> >>
> >> ----- Original Message -----
> >>
> >>> Welcome, Carlos.  I think it's great that you're taking initiative here.
> >>
> >> +1 - I love enthusiastic fresh me^H^H^H^H^H^H^H^Hcommunity members! :)
> >>
> >>
> >>> However, it's also important to set proper expectations for what a GSoC
> >>> intern
> >>> could reasonably be expected to achieve.  I've seen some amazing stuff
> >>> out of
> >>> GSoC, but if we set the bar too high then we end up with incomplete code
> >>> and
> >>> the student doesn't learn much except frustration.
> >>
> >> This. The reason we haven't really participated in GSoC is not because we
> >> don't want to - it's because it's exceptionally difficult for a project of
> >> our scope, but that doesn't mean there aren't any possibilities. As an
> >> example, last year the Open Source Lab at OSU worked with a student to
> >> create an integration with Ganeti, which was mostly successful, and I think
> >> work has continued on that project. That's an example of a project with the
> >> right scope.
> >
> >
> > IMO integration projects are ideal fits for GSoc. I can see some information
> > in Trello back log i.e. under "Ecosystem Integration". But not sure of their
> > current status. I think we should again take look on these and see if
> > something can be done through GSoc.
> >
> >
> >>>> 3) Accelerator node project. Some storage solutions out there offer an
> >>>> "accelerator node", which is, in short, a, extra node with a lot of RAM,
> >>>> eventually fast disks (SSD), and that works like a proxy to the regular
> >>>> volumes. active chunks of files are moved there, logs (ZIL style) are
> >>>> recorded on fast media, among other things. There is NO active project
> >>>> for
> >>>> this, or trello entry, because it is something I started discussing with
> >>>> a
> >>>> few fellows just a couple of days ago. I thought of starting to play
> >>>> with
> >>>> RAM disks (tmpfs) as scratch disks, but, since we have an opportunity to
> >>>> do
> >>>> something more efficient, or at the very least start it, why not ?
> >>>
> >>> Looks like somebody has read the Isilon marketing materials.  ;)
> >>>
> >>> A full production-level implementation of this, with cache consistency
> >>> and
> >>> so on, would be a major project.  However, a non-consistent prototype
> >>> good
> >>> for specific use cases - especially Hadoop, as Jay mentions - would be
> >>> pretty easy to build.  Having a GlusterFS server (for the real clients)
> >>> also be a GlusterFS client (to the real cluster) is pretty
> >>> straightforward.
> >>> Testing performance would also be a significant component of this, and
> >>> IMO
> >>> that's something more developers should learn about early in their
> >>> careers.
> >>> I encourage you to keep thinking about how this could be turned into a
> >>> real
> >>> GSoC proposal.
> >>
> >> Excellent. This has possibilities.
> >>
> >> Another possibility is in the mobile app space. I think it would be
> >> awesome to port GFAPI to Android, for example. Or to make use of the python
> >> or ruby bindings for GFAPI to create a server-side RESTful API that a mobile
> >> app can access.
> >>
> >> -JM
> >>
> >>
> >>
> >> _______________________________________________
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> http://supercolony.gluster.org/mailman/listinfo/gluster-users
> >
> >
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://supercolony.gluster.org/mailman/listinfo/gluster-users
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140318/30578429/attachment-0001.sig>