[Gluster-users] [Gluster-devel] PLEASE READ ! We need your opinion. GSOC-2014 and the Gluster community

[Kaushal M]

I had a discussion with some developers here in the office regarding
this. We created a list of ideas which we thought could be suitable
for student projects. I've added these to [1]. But I'm also putting
them on here for more visibility.

(I've tried to arrange the list in descending order of difficulty as I find it)

. Glusterd services high availablity
    Glusterd should restart the processes it manages, bricks, nfs
server, self-heal daemon & quota daemon, whenever it detects they have
died.
. glusterfsiostat - Top like utility for glusterfs
    These are client side tools which will display stats from the
io-stats translator. I'm not currently sure of the difference between
the two.
. ovirt gui for stats
    Have pretty graphs and tables in ovirt for the GlusterFS top and
profile commands.
. monitoring integrations - munin others.
    The more monitoring support we have for GlusterFS the better.
. More compression algorithms for compression xlator
    The onwire compression translator should be extended to support
more compression algorithms. Ideally it should be pluggable.
. cinder glusterfs backup driver
    Write a driver for cinder, a part of openstack, to allow backup
onto GlusterFS volumes
. rsockets - sockets for rdma transport
    Coding for RDMA using the familiar socket api should lead to a
more robust rdma transport
. data import tool
    Create a tool which will allow already importing already existing
data in the brick directories into the gluster volume. This is most
likely going to be a special rebalance process.
. rebalance improvements
    Improve rebalance preformance.
. Improve the meta translator
    The meta xlator provides a /proc like interface to GlusterFS
xlators. We could further improve this and make it a part of the
standard volume graph.
. geo-rep using rest-api
    This might be suitable for geo replication over WAN. Using
rsync/ssh over WAN isn't too nice.
. quota using underlying fs quota
    GlusterFS quota is currently maintained completely in GlusterFSs
namespace using xattrs. We could make use of the quota capabilities of
the underlying fs (XFS) for better performance.
. snapshot pluggability
    Snapshot should be able to make use of snapshot support provided
by btrfs for example.
. compression at rest
    Lessons learnt while implementing encryption at rest can be used
with the compression at rest.
. file-level deduplication
    GlusterFS works on files. So why not have dedup at the level files as well.
. composition xlator for small files
    Merge smallfiles into a designated large file using our own custom
semantics. This can improve our small file performance.
. multi master geo-rep
    Nothing much to say here. This has been discussed many times.

Any comments on this list?
~kaushal

[1] http://www.gluster.org/community/documentation/index.php/Projects

On Tue, Mar 18, 2014 at 9:07 AM, Lalatendu Mohanty <lmohanty at redhat.com> wrote:
> On 03/13/2014 11:49 PM, John Mark Walker wrote:
>>
>> ----- Original Message -----
>>
>>> Welcome, Carlos.  I think it's great that you're taking initiative here.
>>
>> +1 - I love enthusiastic fresh me^H^H^H^H^H^H^H^Hcommunity members! :)
>>
>>
>>> However, it's also important to set proper expectations for what a GSoC
>>> intern
>>> could reasonably be expected to achieve.  I've seen some amazing stuff
>>> out of
>>> GSoC, but if we set the bar too high then we end up with incomplete code
>>> and
>>> the student doesn't learn much except frustration.
>>
>> This. The reason we haven't really participated in GSoC is not because we
>> don't want to - it's because it's exceptionally difficult for a project of
>> our scope, but that doesn't mean there aren't any possibilities. As an
>> example, last year the Open Source Lab at OSU worked with a student to
>> create an integration with Ganeti, which was mostly successful, and I think
>> work has continued on that project. That's an example of a project with the
>> right scope.
>
>
> IMO integration projects are ideal fits for GSoc. I can see some information
> in Trello back log i.e. under "Ecosystem Integration". But not sure of their
> current status. I think we should again take look on these and see if
> something can be done through GSoc.
>
>
>>>> 3) Accelerator node project. Some storage solutions out there offer an
>>>> "accelerator node", which is, in short, a, extra node with a lot of RAM,
>>>> eventually fast disks (SSD), and that works like a proxy to the regular
>>>> volumes. active chunks of files are moved there, logs (ZIL style) are
>>>> recorded on fast media, among other things. There is NO active project
>>>> for
>>>> this, or trello entry, because it is something I started discussing with
>>>> a
>>>> few fellows just a couple of days ago. I thought of starting to play
>>>> with
>>>> RAM disks (tmpfs) as scratch disks, but, since we have an opportunity to
>>>> do
>>>> something more efficient, or at the very least start it, why not ?
>>>
>>> Looks like somebody has read the Isilon marketing materials.  ;)
>>>
>>> A full production-level implementation of this, with cache consistency
>>> and
>>> so on, would be a major project.  However, a non-consistent prototype
>>> good
>>> for specific use cases - especially Hadoop, as Jay mentions - would be
>>> pretty easy to build.  Having a GlusterFS server (for the real clients)
>>> also be a GlusterFS client (to the real cluster) is pretty
>>> straightforward.
>>> Testing performance would also be a significant component of this, and
>>> IMO
>>> that's something more developers should learn about early in their
>>> careers.
>>> I encourage you to keep thinking about how this could be turned into a
>>> real
>>> GSoC proposal.
>>
>> Excellent. This has possibilities.
>>
>> Another possibility is in the mobile app space. I think it would be
>> awesome to port GFAPI to Android, for example. Or to make use of the python
>> or ruby bindings for GFAPI to create a server-side RESTful API that a mobile
>> app can access.
>>
>> -JM
>>
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users