[Gluster-devel] [Gluster-infra] tests/basic/tier/tier-file-create.t dumping core on Linux

Tue Mar 8 17:36:58 UTC 2016

On Tue, Mar 08, 2016 at 08:45:12AM -0500, Dan Lambright wrote:
> 
> 
> ----- Original Message -----
> > From: "Niels de Vos" <ndevos at redhat.com>
> > To: "Krutika Dhananjay" <kdhananj at redhat.com>, "Vijay Bellur" <vbellur at redhat.com>
> > Cc: "Dan Lambright" <dlambrig at redhat.com>, "Gluster Devel" <gluster-devel at gluster.org>, "Pranith Karampuri"
> > <pkarampu at redhat.com>, "gluster-infra" <gluster-infra at gluster.org>, "RHGS tiering mailing list"
> > <rhgs-tiering at redhat.com>
> > Sent: Tuesday, March 8, 2016 8:37:14 AM
> > Subject: Re: [Gluster-infra] tests/basic/tier/tier-file-create.t dumping core on Linux
> > 
> > On Tue, Mar 08, 2016 at 07:00:07PM +0530, Krutika Dhananjay wrote:
> > > I did talk to the author and he is going to look into the issue.
> > > 3.7.9 is round the corner and we certainly don't want bad tests to block
> > > patches that need to go in.
> > 
> > 3.7.9 has been delayed already. There is hardly a good reason for
> > changes to delay the relase even more. 3.7.10 will be done in a few
> > weeks at the end of March, anything non-regression should probably not
> > be included at this point anymore.
> > 
> > Vijay is the release manager for 3.7.9 and .10, you'll need to come with
> > extreme strong points for getting more patches included.
> 
> I've no desire to disrupt downstream releases and if we are indeed out
> of time for 3.7.9, then thats life.

3.7.x releases are planned for each 30th of the month, 3.7.9 was
supposed to be tagged for relaese the end of February. We are trying to
get back to this schdule, and are failing hard :-(

Upstream has a very strict schedule (that we fail to follow because the
release managers are occupied elsewhere), downstream (RHGS) should be
aware of that as is it documented on
https://www.gluster.org/community/release-schedule/ . Please pass that
link on to anyone that expects changes to get merged in certain upstream
releases. In general a release needs a few days of preparation, so
patches for a release need to be ready at least ~5 working days in
advance.

> That said, obviously a test failure is a failure, and once its masked
> it tends to be forgotton.

When a test is marked as bad_test, we require a bug to be filed for that
test. It should not be forgotten, the component maintainers are expected
to track (and eventually fix) the open bugs for their components.

> I suggest we do not mask this test upstream (does it happen there?)
> and give some urgency to chasing it down.

I assumed this is all for upstream. Not sure why an internal Red Hat
list (rhgs-tiering) was put on CC.

HTH,
Niels

> 
> 
> > 
> > Niels
> > 
> > 
> > > 
> > > -Krutika
> > > 
> > > On Tue, Mar 8, 2016 at 6:50 PM, Dan Lambright <dlambrig at redhat.com> wrote:
> > > 
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Krutika Dhananjay" <kdhananj at redhat.com>
> > > > > To: "Pranith Karampuri" <pkarampu at redhat.com>
> > > > > Cc: "gluster-infra" <gluster-infra at gluster.org>, "Gluster Devel" <
> > > > gluster-devel at gluster.org>, "RHGS tiering mailing
> > > > > list" <rhgs-tiering at redhat.com>, "Dan Lambright" <dlambrig at redhat.com>
> > > > > Sent: Tuesday, March 8, 2016 12:15:20 AM
> > > > > Subject: Re: [Gluster-infra] tests/basic/tier/tier-file-create.t
> > > > > dumping
> > > > core on Linux
> > > > >
> > > > > It has been failing rather frequently.
> > > > > Have reported a bug at
> > > > https://bugzilla.redhat.com/show_bug.cgi?id=1315560
> > > > > For now, have moved it to bad tests here:
> > > > > http://review.gluster.org/#/c/13632/1
> > > > >
> > > >
> > > >
> > > > Masking tests is a bad habit. It would be better to fix the problem, and
> > > > it looks like a real bug.
> > > > The author of the test should help chase this down.
> > > >
> > > > > -Krutika
> > > > >
> > > > > On Mon, Mar 7, 2016 at 4:17 PM, Krutika Dhananjay <kdhananj at redhat.com>
> > > > > wrote:
> > > > >
> > > > > > +Pranith
> > > > > >
> > > > > > -Krutika
> > > > > >
> > > > > >
> > > > > > On Sat, Mar 5, 2016 at 11:34 PM, Dan Lambright <dlambrig at redhat.com>
> > > > > > wrote:
> > > > > >
> > > > > >>
> > > > > >>
> > > > > >> ----- Original Message -----
> > > > > >> > From: "Dan Lambright" <dlambrig at redhat.com>
> > > > > >> > To: "Shyam" <srangana at redhat.com>
> > > > > >> > Cc: "Krutika Dhananjay" <kdhananj at redhat.com>, "Gluster Devel" <
> > > > > >> gluster-devel at gluster.org>, "Rafi Kavungal Chundattu
> > > > > >> > Parambil" <rkavunga at redhat.com>, "Nithya Balachandran" <
> > > > > >> nbalacha at redhat.com>, "Joseph Fernandes"
> > > > > >> > <josferna at redhat.com>, "gluster-infra" <gluster-infra at gluster.org>
> > > > > >> > Sent: Friday, March 4, 2016 9:51:18 AM
> > > > > >> > Subject: Re: [Gluster-infra] tests/basic/tier/tier-file-create.t
> > > > > >> dumping core on Linux
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> > ----- Original Message -----
> > > > > >> > > From: "Shyam" <srangana at redhat.com>
> > > > > >> > > To: "Krutika Dhananjay" <kdhananj at redhat.com>, "Gluster Devel"
> > > > > >> > > <gluster-devel at gluster.org>, "Rafi Kavungal Chundattu
> > > > > >> > > Parambil" <rkavunga at redhat.com>, "Nithya Balachandran"
> > > > > >> > > <nbalacha at redhat.com>, "Joseph Fernandes"
> > > > > >> > > <josferna at redhat.com>, "Dan Lambright" <dlambrig at redhat.com>
> > > > > >> > > Cc: "gluster-infra" <gluster-infra at gluster.org>
> > > > > >> > > Sent: Friday, March 4, 2016 9:45:17 AM
> > > > > >> > > Subject: Re: [Gluster-infra] tests/basic/tier/tier-file-create.t
> > > > > >> dumping
> > > > > >> > > core on Linux
> > > > > >> > >
> > > > > >> > > Facing the same problem in the following runs as well,
> > > > > >> > >
> > > > > >> > > 1)
> > > > > >> > >
> > > > > >>
> > > > https://build.gluster.org/job/rackspace-regression-2GB-triggered/18767/console
> > > > > >> > > 2)
> > > > https://build.gluster.org/job/regression-test-burn-in/546/console
> > > > > >> > > 3)
> > > > https://build.gluster.org/job/regression-test-burn-in/547/console
> > > > > >> > > 4)
> > > > https://build.gluster.org/job/regression-test-burn-in/549/console
> > > > > >> > >
> > > > > >> > > Last successful burn-in was: 545 (but do not see the test having
> > > > been
> > > > > >> > > run here, so this is inconclusive)
> > > > > >> > >
> > > > > >> > > burn-in test 544 is hung on the same test here,
> > > > > >> > > https://build.gluster.org/job/regression-test-burn-in/544/console
> > > > > >> > >
> > > > > >> > > (and at this point I am stopping the hunt for when this last
> > > > > >> succeeded :) )
> > > > > >> > >
> > > > > >> > > Let's know if anyone is taking a peek at the cores.
> > > > > >> >
> > > > > >> > hm. Not familiar with this test. Written by Pranith? I'll look.
> > > > > >>
> > > > > >> We are doing lookup everywhere, and building up a dict of the
> > > > > >> extended
> > > > > >> attributes of a file as we traverse each sub volume across the hot
> > > > > >> and
> > > > > >> cold
> > > > > >> tiers. The length field of one of the EC keys is corrupted.
> > > > > >>
> > > > > >> Not clear why this is happening.. I see no tiering relationship as
> > > > > >> of
> > > > > >> yet, its possible the file is being demoted in parallel to the
> > > > foreground
> > > > > >> script operation.
> > > > > >>
> > > > > >> The test runs fine on my machines.  Does this reproduce consistently
> > > > on
> > > > > >> one of the Jenkins machines? If so, getting onto it would be the
> > > > > >> next
> > > > > >> step.
> > > > > >> I think that would be preferable to masking this test case.
> > > > > >>
> > > > > >>
> > > > > >> >
> > > > > >> > >
> > > > > >> > > Thanks,
> > > > > >> > > Shyam
> > > > > >> > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > On 03/04/2016 07:40 AM, Krutika Dhananjay wrote:
> > > > > >> > > > Could someone from tiering dev team please take a look?
> > > > > >> > > >
> > > > > >> > > >
> > > > > >>
> > > > https://build.gluster.org/job/rackspace-regression-2GB-triggered/18793/console
> > > > > >> > > >
> > > > > >> > > > -Krutika
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > _______________________________________________
> > > > > >> > > > Gluster-infra mailing list
> > > > > >> > > > Gluster-infra at gluster.org
> > > > > >> > > > http://www.gluster.org/mailman/listinfo/gluster-infra
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > 
> > > _______________________________________________
> > > Gluster-infra mailing list
> > > Gluster-infra at gluster.org
> > > http://www.gluster.org/mailman/listinfo/gluster-infra
> > 
> > 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160308/0ecf0849/attachment-0001.sig>