[Gluster-devel] Gluster API out of Unix environment

[XoD]

Hello, thanks for your responses.

The swift interface will add an undesired extra interface to the data (plus
probably the need of extra servers ?).
And we don't have the resource internally to take the time to better
understand Gluster and create our custom client library/api.

We will ask the same question on the Ceph mailing list, and if there is the
same difficulties, we probably abandon the idea of using Cluster/Ceph as
part of our server infrastructure.

2017-06-02 7:08 GMT+02:00 Niels de Vos <ndevos at redhat.com>:

> On Fri, Jun 02, 2017 at 12:13:53AM -0400, Prashanth Pai wrote:
> >
> > ----- Original Message -----
> > > From: "XoD" <xoddark at gmail.com>
> > > To: gluster-devel at gluster.org
> > > Sent: Thursday, 1 June, 2017 8:08:08 PM
> > > Subject: [Gluster-devel] Gluster API out of Unix environment
> > >
> > > Hello, in my company we are looking for a most efficient solution to
> > > distribute a lot of small files, for a lot of clients.
> > >
> > > We are considering using Glusterfs for this (Or Ceph) as our server
> > > infrastructure.
> > > On the client side we want to access data as object directly from the
> > > application.
> > > Our applications will not run in an Unix environment (Windows among
> others).
> > > But unfortunately the Gluster API (Or Ceph API) depends on Unix
> technologies
> > > and headers.
> > >
> > > So to summarize our needs for the API are :
> > > - put new files on a volume
> > > - read files on the volume
> > > - remove files on the volume
> > > We do not need to update any files after the first write.
> > >
> > > For the moment, here is our current investigation status:
> > > We have tried to build the api (libglusterfs) in a Msys/Mingw
> environment,
> > > but all dependencies are not available.
> > > We are considering to modify libglusterfs to replace Unix
> dependencies, but
> > > it's seems a lot of work.
>
> The interface you would use in applications is in libgfapi.so. This
> library depends on libglusterfs and others. There is already support for
> Linux, NetBSD and partially (untested?) FreeBSD and Mac OSX. Adding
> support for an other UNIX-like implementation (Mingw, Cygwin, ..) should
> be doable, but might not be trivial.
>
> > > We are considering to reimplement the client library, but I haven't
> found
> > > documentation about the communication protocol of libglusterfs api.
> > > We are also considering to create a new (simpler) API, possibly based
> on
> > > http, but we need to know how implement the server part.
> >
> >
> > You can put and get files as objects using gluster-swift project. This
> happens
> > over HTTP protocol with requests and responses conforming to Swift or S3
> API.
> > https://github.com/gluster/gluster-swift
>
> This is my suggestion as well. gluster-swift (with the Swift3 add-on) is
> an OS independent way of doing this.
>
> > > Without any knowledge of the internal Gluster’s functionality it's not
> easy,
> > > and I haven't found any documentation about it.
>
> The protocol is very sparsely documented, so writing a new
> implementation from scratch that will be difficult and a *lot* of work.
> Because Gluster is based on translators on both the client- and
> server-side, behaviour may change subtly depending on what features are
> enabled on the Gluster volume. Adding a new implementation will also
> require you to keep it updated whenever new functionalities are added to
> the protocol.
>
> > > So here are my questions:
> > > Does anybody have advices/warnings about how we can achieve any of the
> afore
> > > mentioned ports ?
> > > Or know any open source library project to access to Gluster
> files/objects
> > > from (at least) a Windows application.
>
> The best advise we can give is to abandon the idea of a Windows native
> Gluster implementation. Either use the (OpenStack) Swift, (Amazon) S3 or
> NFS/SMB protocols, the last two might have native Windows libraries too.
>
> Please keep us informed about your approach and plans, even if you
> decide to use something else as Gluster.
>
> Thanks and good luck!
> Niels
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20170608/74becd44/attachment.html>