[Gluster-devel] swapcontest usage in syncio.c

[Emmanuel Dreyfus]

On Tue, Aug 07, 2012 at 12:44:02PM +0000, Emmanuel Dreyfus wrote:
> Here is the kind of trace I can get in rebalance.log, after the first
> synctask_new invocation (self is pthread_self() return value).
> 
> [syncop.c:216:synctask_new] 0-: self = 0xb8000000, newtask = 0xb7f02400
> [syncop.c:291:synctask_switchto] 0-: self = 0xb9e00000, synctask_set(0xb7f02400)
> [syncop.c:120:synctask_wrap] 0-: self = 0xb8000000, synctask_get() = 0x0, old_task = 0xb7f02400

I posted the full explanation of the phenomenon, but I notice I forgot to 
Cc gluster-devel@ so here is it again.

The problem is caused by a non portable behavior of swapcontext(): on 
Linux it remains in the same thread, on NetBSD it switches to the
thread where getcontext()/makecontext() was run.

Here is a test case that exhibit the difference:

/* 
 * cc -o tss tss.c
 *
 * linux# ./tss 
 * before swapcontext self = 0x4002
 * after swapcontext self = 0x4002
 *
 * NetBSD# ./tss
 * before swapcontext self = 0xbb600000
 * after swapcontext self = 0xbfa00000
 */
#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
#include <err.h>
#include <sysexits.h>
#include <signal.h>
#ifdef __linux__
#include <bits/sigstack.h> /* For stack_t */
#endif
#include <ucontext.h>

char stack[66536];
ucontext_t ctx;
ucontext_t octx;

void wrap(int);
void *thread1(void *);

void
wrap(cookie)
        int cookie;
{
        printf("after swapcontext self = %p\n", (void *)pthread_self());
        return;
}

void *
thread1(arg)
        void *arg;
{

        sleep(1);

        ctx.uc_stack.ss_sp = stack;
        ctx.uc_stack.ss_size = sizeof(stack);
        
        makecontext(&ctx, (void *)*wrap, 2, (int)0xdeadbeef);

        printf("before swapcontext self = %p\n", (void *)pthread_self());
        if (swapcontext(&octx, &ctx) != 0)
                err(EX_OSERR, "swapcontext failed");
        
        errx(EX_SOFTWARE, "swapcontext returned");
        /* NOTREACHED */
        return NULL;
}


int
main(argc, argv)
        int argc;
        char **argv;
{
        int error;
        pthread_t th;

        if ((error = pthread_create(&th, NULL, *thread1, NULL)) != 0)
                err(EX_OSERR, "pthread_create returns error = %d", error);

        if (getcontext(&ctx) != 0)
                err(EX_OSERR, "getcontext failed");

        sleep(2);

        return EX_OK;
}


-- 
Emmanuel Dreyfus
manu at netbsd.org