[Gluster-devel] RPM spec file and starting glusterd in no daemon mode

[Raghavendra Talur]

On Wednesday 29 April 2015 09:26 PM, Kaushal M wrote:
> If we were to run without `-N`, in daemon mode, yum wouldn't know when
> glusterd actually finished the upgrade process. Yum would consider the
> parent process returning to be the end.

I did an experiment and here is what I found,

[root at 192 ~]# time glusterd --xlator-option *.upgrade=on -N
librdmacm: Warning: couldn't read ABI version.
librdmacm: Warning: assuming: 4
librdmacm: Fatal: unable to get RDMA device list

real	0m13.645s
user	0m8.374s
sys	0m2.177s
[root at 192 ~]# echo $?
0
[root at 192 ~]# time glusterd --xlator-option *.upgrade=on

real	0m13.678s
user	0m0.001s
sys	0m0.004s
[root at 192 ~]# echo $?
1

So,
1. parent process would return to yum only after upgrade completes
    in both daemon and no-daemon cases.
2. What would be wrong is the return value, in no-daemon it is 0
    indicating upgrade was a success. In daemon mode, it is 1
    because child exited without informing anything back to parent.
    (Although, upgrade was a success).

Proposal for a change:
Is it ok to call emancipate in
glusterd_handle_upgrade_downgrade() just before exit? Something like
this..

         if (terminate && (ret == 0)) {
	  ----->emancipate(ctx, 0);
                 kill (getpid(), SIGTERM);
	}

instead of the current method of just kill

         if (terminate && (ret == 0))
                 kill (getpid(), SIGTERM);

If would send the patch if that is fine.
Code details are below for those interested.

* In the daemonize funtion, we have

default:
                 /* parent */
                 /* close write */
                 close (ctx->daemon_pipe[1]);

                 if (ctx->mnt_pid > 0) {
                         ret = waitpid (ctx->mnt_pid, &cstatus, 0);
                         if (!(ret == ctx->mnt_pid && cstatus == 0)) {
                                 gf_msg ("daemonize", GF_LOG_ERROR, 0,
                                         glusterfsd_msg_25);
                                 exit (1);
                         }
                 }

                 err = 1;
                 read (ctx->daemon_pipe[0], (void *)&err, sizeof (err));
                 _exit (err);

* and in emancipate we have,

void
emancipate (glusterfs_ctx_t *ctx, int ret)
{
         /* break free from the parent */
         if (ctx->daemon_pipe[1] != -1) {
                 write (ctx->daemon_pipe[1], (void *) &ret, sizeof(ret));
                 close (ctx->daemon_pipe[1]);
                 ctx->daemon_pipe[1] = -1;
         }
}


So ideally, parent waits for child to process volfile and give out its
ret before parent exits.(Process volfile is called before emancipate)

However, in glusterd upgrade option we call

int32_t
glusterd_handle_upgrade_downgrade (dict_t *options, glusterd_conf_t *conf)
{
......
......
         if (!upgrade && !downgrade)
                 ret = 0;
         else
                 terminate = _gf_true;
         if (regenerate_volfiles) {
                 ret = glusterd_recreate_volfiles (conf);
         }
out:
         if (terminate && (ret == 0))
                 kill (getpid(), SIGTERM);
         return ret;
}


>
>
>
> On Wed, Apr 29, 2015 at 9:17 PM, Raghavendra Talur <rtalur at redhat.com> wrote:
>> Hi,
>>
>> As part of the yum upgrade procedure, when glusterfs-server is updated
>> we run glusterd in no daemon mode along with upgrade option with this
>> command.
>>
>> glusterd --xlator-option *.upgrade=on -N
>>
>> This helps us update our vol files with new defaults along with
>> few other things.(say we added a new xlator which we want as default).
>>
>> Starting in no daemon mode has a problem though, we leave our stdout,
>> stdin and stderr open. This can cause messages to be printed on the
>> console from any of the libs that we load.
>>
>> We have seen this problem with librdmacm, it prints out these
>> messages on screen
>>
>> librdmacm: Warning: couldn't read ABI version.
>> librdmacm: Warning: assuming: 4
>> librdmacm: Fatal: unable to get RDMA device list
>>
>> I looked through the code and did not find any real requirement
>> to use no-daemon option(-N). Anyhow, we init logging very early
>> in our process and don't need the stderr open.
>>
>> I am missing something or can we exclude -N from our spec files
>> while performing upgrade?
>>
>>
>> Thanks,
>> Raghavendra Talur
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel