[Gluster-devel] Cores generated with ./tests/geo-rep/georep-basic-dr-tarssh.t

[Raghavendra G]

Hi Soumya,

Can you send a fix to this regression on upstream master too? This patch is
merged there.

regards,
Raghavendra

On Tue, Mar 1, 2016 at 10:34 PM, Kotresh Hiremath Ravishankar <
khiremat at redhat.com> wrote:

> Hi Soumya,
>
> I analysed the issue and found out that crash has happened because of the
> patch [1].
>
> The patch doesn't set transport object to NULL in 'rpc_clnt_disable' but
> instead does it on
> 'rpc_clnt_trigger_destroy'. So if there are pending rpc invocations on the
> rpc object that
> is disabled (those instances are possible as happening now in changelog),
> it will trigger a
> CONNECT notify again with 'mydata' that is freed causing a crash. This
> happens because
> 'rpc_clnt_submit' reconnects if rpc is not connected.
>
>  rpc_clnt_submit (...) {
>    ...
>                 if (conn->connected == 0) {
>                         ret = rpc_transport_connect (conn->trans,
>
>  conn->config.remote_port);
>                 }
>    ...
>  }
>
> Without your patch, conn->trans was set NULL and hence CONNECT fails not
> resulting with
> CONNECT notify call. And also the cleanup happens in failure path.
>
> So the memory leak can happen, if there is no try for rpc invocation after
> DISCONNECT.
> It will be cleaned up otherwise.
>
>
> [1] http://review.gluster.org/#/c/13507/
>
> Thanks and Regards,
> Kotresh H R
>
> ----- Original Message -----
> > From: "Kotresh Hiremath Ravishankar" <khiremat at redhat.com>
> > To: "Soumya Koduri" <skoduri at redhat.com>
> > Cc: avishwan at redhat.com, "Gluster Devel" <gluster-devel at gluster.org>
> > Sent: Monday, February 29, 2016 4:15:22 PM
> > Subject: Re: Cores generated with
> ./tests/geo-rep/georep-basic-dr-tarssh.t
> >
> > Hi Soumya,
> >
> > I just tested that it is reproducible only with your patch both in
> master and
> > 3.76 branch.
> > The geo-rep test cases are marked bad in master. So it's not hit in
> master.
> > rpc is introduced
> > in changelog xlator to communicate to applications via libgfchangelog.
> > Venky/Me will check
> > why is the crash happening and will update.
> >
> >
> > Thanks and Regards,
> > Kotresh H R
> >
> > ----- Original Message -----
> > > From: "Soumya Koduri" <skoduri at redhat.com>
> > > To: avishwan at redhat.com, "kotresh" <khiremat at redhat.com>
> > > Cc: "Gluster Devel" <gluster-devel at gluster.org>
> > > Sent: Monday, February 29, 2016 2:10:51 PM
> > > Subject: Cores generated with ./tests/geo-rep/georep-basic-dr-tarssh.t
> > >
> > > Hi Aravinda/Kotresh,
> > >
> > > With [1], I consistently see cores generated with the test
> > > './tests/geo-rep/georep-basic-dr-tarssh.t' in release-3.7 branch. From
> > > the cores, looks like we are trying to dereference a freed
> > > changelog_rpc_clnt_t(crpc) object in changelog_rpc_notify(). Strangely
> > > this was not reported in master branch.
> > >
> > > I tried debugging but couldn't find any possible suspects. I request
> you
> > > to take a look and let me know if [1] caused any regression.
> > >
> > > Thanks,
> > > Soumya
> > >
> > > [1] http://review.gluster.org/#/c/13507/
> > >
> >
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Raghavendra G
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160303/8bef3f9f/attachment.html>