[Gluster-devel] rpc problems when using syncops in callbacks

[Krishnan Parthasarathi]

Fog,

On 04/29/2013 01:57 PM, fog - wrote:
> Hello Avati,
>
> I am wrapping the syncop call in a synctask_new (otherwise glusterFS 
> will run into a null pointer @ synctask_get in the SYNCOP macro & 
> crash). Below is some code to show how I do it currently to test the 
> syncops.
>
> typedef struct{
>     xlator_t *this;    loc_t *loc;    dict_t *dic;
> }syncstore_args;
>
> int32_t __xattr_store_sync(void* data)
> {
>     syncstore_args *args = (syncstore_args*)data;
>     return syncop_setxattr(FIRST_CHILD(args->this), args->loc, 
> args->dic, 0);
> }
>
> int32_t xattr_store_sync(xlator_t *this, call_frame_t *frame, loc_t 
> *loc, dict_t *dic)
> {
>     syncstore_args args = {this, loc, dic};
>     return synctask_new(this->ctx->env, __xattr_store_sync, NULL, 
> NULL, &args);
If you don't provide a synctask_cbk_t to synctask_new, you are using 
synctask in a 'blocking' mode.
That is, the thread calling synctask_new would block until the 
synctask_fn_t function (ie, __xattr_store_sync) returns.
An alternative way to do this would be,

int32_t xattr_store_sync(xlator_t *this, call_frame_t *frame, loc_t 
*loc, dict_t *dic)
{
     syncstore_args args = {this, loc, dic};
     return synctask_new(this->ctx->env, __xattr_store_sync, 
__xattr_store_sync_cbk, NULL, &args);
}

int32_t __xattr_store_sync_cbk (int ret, /*and the other args*/)
{
     // Your code goes here
     return ret;
}

Now, all file operations performed using syncop_* inside 
__xattr_store_sync would have the synchronous flavour, while leaving the 
calling thread (thread calling xattr_store_sync fn) 'free'. This should 
avoid the hang issue.

HTH,
krish


> }
>
> ------------------------------------------------------------------------
> Date: Mon, 29 Apr 2013 00:19:11 -0700
> Subject: Re: [Gluster-devel] rpc problems when using syncops in callbacks
> From: anand.avati at gmail.com
> To: fog_is_my_name at hotmail.com
> CC: gluster-devel at nongnu.org
>
> Note that you need to place your syncop code in a synctask function 
> strictly within a syncenv (by calling synctask_new(). You're probably 
> calling syncop_XXX() directly in your xlator code?
>
> Avati
>
>
> On Fri, Apr 26, 2013 at 2:40 AM, fog - <fog_is_my_name at hotmail.com 
> <mailto:fog_is_my_name at hotmail.com>> wrote:
>
>     Hello everyone,
>
>     I am trying to use syncops in a custom translator to keep my code
>     at least borderline readable, but I am having limited success.
>
>     Problem Symptoms:
>     Using a syncop in a regular fop is fine. However, in a callback it
>     causes a 'freeze' (synctask_yield called by the SYNCOP macro
>     doesn't return).
>
>     What seems to be the Problem:
>     Looking at the traces, there is no corresponding trace from
>     rpc_clnt_reply_init on the client to the trace from
>     rpcsvc_submit_generic on the server. In other words, the rpc reply
>     gets sent but isn't correctly received. Obviously this is not
>     really a networking problem but something else... I'd guess it's a
>     deadlock somewhere on the client?
>     From the point of the syncop call onwards the client doesn't 'get'
>     any rpc replies any more (the next GlusterFS Handshake sent by the
>     client, which is received by the server and replied to, leads to a
>     disconnection accordingly).
>
>     Again: This problem is only occurring when calling a syncop from a
>     callback function inside my translator, if I call the same syncop
>     in a fop call it completes fine.
>
>     I hope you can make sense out of the above problem description.
>     Thanks for your time ~
>
>
>     _______________________________________________
>     Gluster-devel mailing list
>     Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
>     https://lists.nongnu.org/mailman/listinfo/gluster-devel
>
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20130429/75db7772/attachment-0001.html>