[Gluster-devel] [Gluster-users] uWSGI plugin and some question

[Anand Avati]

On Mon, Jul 29, 2013 at 10:55 PM, Anand Avati <anand.avati at gmail.com> wrote:

>
> On Mon, Jul 29, 2013 at 8:36 AM, Roberto De Ioris <roberto at unbit.it>wrote:
>
>>
>> Hi everyone, i have just committed a plugin for the uWSGI application
>> server
>> for exposing glusterfs filesystems using the new native api:
>>
>> https://github.com/unbit/uwsgi-docs/blob/master/GlusterFS.rst
>>
>> Currently it is very simple, but works really well.
>>
>> I have studied the whole api, and i have two questions:
>>
>> why there is no glfs_stat_async() ?
>>
>> if i understand the code well, even stat() is a blocking operation.
>>
>
> Can you show some code in uwsgi which makes use of asynchronous stat
> calls? Adding an async stat call in gfapi is not hard, but the use case
> hasn't been clear.
>
>
>> My objective is avoiding the use of threads and processes and use the
>> uWSGI async api to implement a non blocking-approach (mixable with other
>> engines like gevent or Coro::AnyEvent)
>>
>
> Are there any requirements that the callback happen only in specific
> threads? That is typically a common requirement, and the async callbacks
> would end up requiring special "wiring" to bring the callbacks to desired
> threads. But I guess that wiring would already be done with the IO
> callbacks anyways in your case.
>
> Do you have some prototype of the module using gfapi out somewhere? I'm
> hoping to understand the use case of gfapi and see if something can be done
> to make it integrate with Coro::AnyEvent more "naturally".
>


I am assuming the module in question is this -
https://github.com/unbit/uwsgi/blob/master/plugins/glusterfs/glusterfs.c. I
see that you are not using the async variants of any of the glfs calls so
far. I also believe you would like these "synchronous" calls to play nicely
with Coro:: by yielding in a compatible way (and getting woken up when
response arrives in a compatible way) - rather than implementing an
explicit glfs_stat_async(). The ->request() method does not seem to be be
naturally allowing the use of "explictly asynchronous" calls within.

Can you provide some details of the event/request management in use? If
possible, I would like to provide hooks for yield and wakeup primitives in
gfapi (which you can wire with Coro:: or anything else) such that these
seemingly synchronous calls (glfs_open, glfs_stat etc.) don't starve the
app thread without yielding.

I can see those hooks having a benefit in the qemu gfapi driver too,
removing a bit of code there which integrates callbacks into the event loop
using pipes.

Avati



> Another thing is the bison/yacc nameclash. uWSGI allows you to load
>> various external libraries and the use of the default 'yy' prefix causes
>> nameclashes with common libraries (like matheval).
>>
>> I understand that matheval too should choose a better approach, but why
>> not prefixing it like glusterfsyy ? This would avoid headaches, even for
>> when people will start using the library in higher level languages.
>>
>> Currently i have tried the YFLAGS env var hack for ./configure but it did
>> not work (i am using bison)
>>
>> YFLAGS="-Dapi.prefix=glusterfsyy -d" ./configure --prefix=/opt/glusterfs/
>>
>
> Hmm, this is nice to get fixed. Do you already have a patch which you have
> used (other than just the "technique" shown above)?
>
>
> Thanks!
> Avati
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20130729/4c9d7cbb/attachment-0001.html>