[Gluster-devel] Question about file copy through libgfapi

Fri Aug 22 11:26:02 UTC 2014

Hi there,

Thanks to both Soumya and Prashanth. Actually you are both right. With the
approach proposed by Soumya I would avoid the FUSE overhead but, as
Prashanth says, the network transfer overhead would be always present. This
is particularly important for me because I deal with very big files
(usually around 100 GB and even more), so that network transfer have a big
impact, while I don't think the impact of the FUSE overhead is that big.
That's why what I would like to get is a "brick to brick" copy (just server
side), so I would like to use the APIs to order the server to make a copy,
so that the network transfer can be avoided.

As far as I understood, it is not currently possible with libgfapi. Do you
think it would be difficult to implement? Are there any other ways?
Thank you and best regards,

*Giacomo Fazio*
IT Engineer

Tel. +41 91 910 7690
E-mail: giacomo.fazio at wcpmediaservices.com  |  Web: www.wcpmediaservices.com

Europe Office: Via Zurigo 35, 6900 Lugano, Switzerland
USA Office: 7083 Hollywood Boulevard Los Angeles, CA 90028

On Fri, Aug 22, 2014 at 9:36 AM, Prashanth Pai <ppai at redhat.com> wrote:

> Hi,
>
> Even with that approach, data would still be read (over the n/w) at the
> client (the app using libgfapi). I think what he is looking for is a server
> side copy (brick to brick) or within same brick _without_ the need for data
> to go through client.
>
> Swift has this feature[1] and it would be really cool for glusterfs to
> have it (may be as an external tool or as a API in libgfapi) :)
>
> # gluster-copy <src> <dest>
> or
> glfs_copy(src,dest)
>
> [1]
> http://programmerthoughts.com/openstack/server-side-object-copy-in-openstack-storage/
>
>
>
> Regards,
>  -Prashanth Pai
>
> ----- Original Message -----
> From: "Soumya Koduri" <skoduri at redhat.com>
> To: "Giacomo Fazio" <giacomo.fazio at wcpmediaservices.com>, "John Mark
> Walker" <johnmark at gluster.org>
> Cc: gluster-devel at gluster.org, "Giovanni Contri" <
> giovanni.contri at wcpmediaservices.com>, forge-admin at gluster.org
> Sent: Friday, August 22, 2014 12:40:01 PM
> Subject: Re: [Gluster-devel] Question about file copy through libgfapi
>
> Hi Giacomo,
>
> If your requirement is to get away with fuse/protocol clients and do
> server-side operations, I think its doable by writing a simple libgfapi
> application. But since there is no libgfapi API equivalent to "cp"
> command, you may need to implement that functionality using "glfs_open,
> glfs_read & glfs_write" APIs.
>
> Here are the few links which Humble has documented on how to use
> libgfapi and different APIs supported by it-
>
> http://humblec.com/libgfapi-interface-glusterfs/
> https://github.com/gluster/glusterfs/blob/master/doc/features/libgfapi.md
>
>
> Few sample examples (written in 'C' and 'python') are copied to -
> https://github.com/gluster/glusterfs/tree/master/api/examples
>
>
> Thanks,
> Soumya
>
>
>
> On 08/21/2014 08:45 PM, Giacomo Fazio wrote:
> > Hi John,
> >
> > Thanks for your quick answer. Do you mean that my question can be
> > summarized in "can we do server-only operations?"? Yes, I think so.
> > Please let me know as soon as you receive any answer or provide me a
> > link where I can follow directly this case.
> > Thanks in advance and best regards,
> >
> > *Giacomo Fazio*
> > IT Engineer
> >
> > Tel. +41 91 910 7690
> > E-mail:Â giacomo.fazio at wcpmediaservices.com
> > <mailto:giacomo.fazio at wcpmediaservices.com>Â  |Â Â Web:Â
> > www.wcpmediaservices.com <http://www.wcpmediaservices.com>
> >
> > Europe Office:Â Via Zurigo 35, 6900 Lugano, Switzerland
> > USA Office:Â 7083 Hollywood Boulevard Los Angeles, CA 90028
> >
> >
> > On Thu, Aug 21, 2014 at 5:04 PM, John Mark Walker <johnmark at gluster.org
> > <mailto:johnmark at gluster.org>> wrote:
> >
> >     Thanks, Giacomo. I'm sending this to the gluster-devel list - it's
> >     an interesting question. Basically, can we do server-only operations?
> >
> >     -JM
> >
> >
> >
>  ------------------------------------------------------------------------
> >
> >         Hello,
> >
> >         I am currently using GlusterFS version 3.5 with two bricks. What
> >         I currently do is mounting the whole storage in some Linux
> >         clients (RedHat) through fuse.glusterfs that (I think) uses NFS
> >         in the background.
> >         What I would like to do is copying a file from a directory to
> >         another one in the storage in the quickest way. Using a "cp
> >         file1 file2" from my RedHat client is not the best option
> >         because the data flows from the storage to my RedHat client
> >         through the network and then back to the storage. I would like
> >         instead to avoid this waste of time and copy the file directly
> >         from the 1st directory to the 2nd one. So, in a nutshell, I
> >         would like to have file1 -> file2Â  , instead of file1 ->
> >         RedHatclient -> file2
> >         Do you think is it possible, for example using libgfapi? Any
> >         example to show me?
> >         Thank you in advance and best regards,
> >
> >         *Giacomo Fazio*
> >         IT Engineer
> >
> >         Tel. +41 91 910 7690 <tel:%2B41%2091%20910%207690>
> >         E-mail:Â giacomo.fazio at wcpmediaservices.com
> >         <mailto:giacomo.fazio at wcpmediaservices.com>Â  |Â Â Web:Â
> >         www.wcpmediaservices.com <http://www.wcpmediaservices.com>
> >
> >         Europe Office:Â Via Zurigo 35, 6900 Lugano, Switzerland
> >         USA Office:Â 7083 Hollywood Boulevard Los Angeles, CA 90028
> >
> >
> >
> >
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> >
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140822/441cf94f/attachment.html>