[Gluster-users] [Gluster-devel] PLEASE READ ! We need your opinion. GSOC-2014 and the Gluster community

[Kaushal M]

On Tue, Mar 18, 2014 at 9:08 PM, John Mark Walker <johnmark at gluster.org> wrote:
> Thanks, Kaushal!
>
> The next steps are the following:
>
> - find out the deadlines for student proposals. If we're going with the Fedora umbrella, we need to get any student proposals submitted very soon
The deadline is this Friday, March 21. We've got 3 days more.

> - write out proposals on fedora project ideas page
This is the ideas page -
http://fedoraproject.org/wiki/Summer_coding_ideas_for_2014
I'm not sure if we should be adding all the ideas I listed earlier
there as some are pretty easy to finish whereas others will take
longer than the GSOC timeline to finish. Also, the ideas need to be in
a specific format.
If I could get help doing this, we could add all.

> - submit proposals, along with proposed mentors
> - track down Vipul and see if he's still interested - I believe he said he was working with KP? KP - can you confirm this?
KP is on vacation and will be back in April, but I can confirm on his
behalf as he had talked to me about this. Since KP isn't around, Vijay
said he'll be reaching out to Vipul.

>
> We need to move quickly if we have any hope of getting projects submitted for this year. Please ask me for any help you need - if you don't know the Fedora folks involved, I can intro you.
>
> Thanks, everyone!
>
> -JM
>
>
> ----- Original Message -----
>> I had a discussion with some developers here in the office regarding
>> this. We created a list of ideas which we thought could be suitable
>> for student projects. I've added these to [1]. But I'm also putting
>> them on here for more visibility.
>>
>> (I've tried to arrange the list in descending order of difficulty as I find
>> it)
>>
>> . Glusterd services high availablity
>>     Glusterd should restart the processes it manages, bricks, nfs
>> server, self-heal daemon & quota daemon, whenever it detects they have
>> died.
>> . glusterfsiostat - Top like utility for glusterfs
>>     These are client side tools which will display stats from the
>> io-stats translator. I'm not currently sure of the difference between
>> the two.
>> . ovirt gui for stats
>>     Have pretty graphs and tables in ovirt for the GlusterFS top and
>> profile commands.
>> . monitoring integrations - munin others.
>>     The more monitoring support we have for GlusterFS the better.
>> . More compression algorithms for compression xlator
>>     The onwire compression translator should be extended to support
>> more compression algorithms. Ideally it should be pluggable.
>> . cinder glusterfs backup driver
>>     Write a driver for cinder, a part of openstack, to allow backup
>> onto GlusterFS volumes
>> . rsockets - sockets for rdma transport
>>     Coding for RDMA using the familiar socket api should lead to a
>> more robust rdma transport
>> . data import tool
>>     Create a tool which will allow already importing already existing
>> data in the brick directories into the gluster volume. This is most
>> likely going to be a special rebalance process.
>> . rebalance improvements
>>     Improve rebalance preformance.
>> . Improve the meta translator
>>     The meta xlator provides a /proc like interface to GlusterFS
>> xlators. We could further improve this and make it a part of the
>> standard volume graph.
>> . geo-rep using rest-api
>>     This might be suitable for geo replication over WAN. Using
>> rsync/ssh over WAN isn't too nice.
>> . quota using underlying fs quota
>>     GlusterFS quota is currently maintained completely in GlusterFSs
>> namespace using xattrs. We could make use of the quota capabilities of
>> the underlying fs (XFS) for better performance.
>> . snapshot pluggability
>>     Snapshot should be able to make use of snapshot support provided
>> by btrfs for example.
>> . compression at rest
>>     Lessons learnt while implementing encryption at rest can be used
>> with the compression at rest.
>> . file-level deduplication
>>     GlusterFS works on files. So why not have dedup at the level files as
>>     well.
>> . composition xlator for small files
>>     Merge smallfiles into a designated large file using our own custom
>> semantics. This can improve our small file performance.
>> . multi master geo-rep
>>     Nothing much to say here. This has been discussed many times.
>>
>> Any comments on this list?
>> ~kaushal
>>
>> [1] http://www.gluster.org/community/documentation/index.php/Projects
>>
>> On Tue, Mar 18, 2014 at 9:07 AM, Lalatendu Mohanty <lmohanty at redhat.com>
>> wrote:
>> > On 03/13/2014 11:49 PM, John Mark Walker wrote:
>> >>
>> >> ----- Original Message -----
>> >>
>> >>> Welcome, Carlos.  I think it's great that you're taking initiative here.
>> >>
>> >> +1 - I love enthusiastic fresh me^H^H^H^H^H^H^H^Hcommunity members! :)
>> >>
>> >>
>> >>> However, it's also important to set proper expectations for what a GSoC
>> >>> intern
>> >>> could reasonably be expected to achieve.  I've seen some amazing stuff
>> >>> out of
>> >>> GSoC, but if we set the bar too high then we end up with incomplete code
>> >>> and
>> >>> the student doesn't learn much except frustration.
>> >>
>> >> This. The reason we haven't really participated in GSoC is not because we
>> >> don't want to - it's because it's exceptionally difficult for a project of
>> >> our scope, but that doesn't mean there aren't any possibilities. As an
>> >> example, last year the Open Source Lab at OSU worked with a student to
>> >> create an integration with Ganeti, which was mostly successful, and I
>> >> think
>> >> work has continued on that project. That's an example of a project with
>> >> the
>> >> right scope.
>> >
>> >
>> > IMO integration projects are ideal fits for GSoc. I can see some
>> > information
>> > in Trello back log i.e. under "Ecosystem Integration". But not sure of
>> > their
>> > current status. I think we should again take look on these and see if
>> > something can be done through GSoc.
>> >
>> >
>> >>>> 3) Accelerator node project. Some storage solutions out there offer an
>> >>>> "accelerator node", which is, in short, a, extra node with a lot of RAM,
>> >>>> eventually fast disks (SSD), and that works like a proxy to the regular
>> >>>> volumes. active chunks of files are moved there, logs (ZIL style) are
>> >>>> recorded on fast media, among other things. There is NO active project
>> >>>> for
>> >>>> this, or trello entry, because it is something I started discussing with
>> >>>> a
>> >>>> few fellows just a couple of days ago. I thought of starting to play
>> >>>> with
>> >>>> RAM disks (tmpfs) as scratch disks, but, since we have an opportunity to
>> >>>> do
>> >>>> something more efficient, or at the very least start it, why not ?
>> >>>
>> >>> Looks like somebody has read the Isilon marketing materials.  ;)
>> >>>
>> >>> A full production-level implementation of this, with cache consistency
>> >>> and
>> >>> so on, would be a major project.  However, a non-consistent prototype
>> >>> good
>> >>> for specific use cases - especially Hadoop, as Jay mentions - would be
>> >>> pretty easy to build.  Having a GlusterFS server (for the real clients)
>> >>> also be a GlusterFS client (to the real cluster) is pretty
>> >>> straightforward.
>> >>> Testing performance would also be a significant component of this, and
>> >>> IMO
>> >>> that's something more developers should learn about early in their
>> >>> careers.
>> >>> I encourage you to keep thinking about how this could be turned into a
>> >>> real
>> >>> GSoC proposal.
>> >>
>> >> Excellent. This has possibilities.
>> >>
>> >> Another possibility is in the mobile app space. I think it would be
>> >> awesome to port GFAPI to Android, for example. Or to make use of the
>> >> python
>> >> or ruby bindings for GFAPI to create a server-side RESTful API that a
>> >> mobile
>> >> app can access.
>> >>
>> >> -JM
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Gluster-users mailing list
>> >> Gluster-users at gluster.org
>> >> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>> >
>> >
>> > _______________________________________________
>> > Gluster-users mailing list
>> > Gluster-users at gluster.org
>> > http://supercolony.gluster.org/mailman/listinfo/gluster-users
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>>