fuel library CI jobs return false positive +1 due to reusing snapshots

Bug #1658063 reported by Vladimir Kuklin
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Confirmed
Critical
Fuel QA Team
Mitaka
Confirmed
Critical
Fuel QA Team
Newton
New
High
Fuel QA Team
Ocata
Confirmed
Critical
Fuel QA Team

Bug Description

If we look into
https://ci.fuel-infra.org/job/master.fuel-library.pkgs.ubuntu.neutron_vlan_ha/buildTimeTrend

we can see that some of the builds take only 18-20

e.g. here is the job https://ci.fuel-infra.org/job/master.fuel-library.pkgs.ubuntu.neutron_vlan_ha/13300/ it took only 18 minutes and did not do deploy actually.

we have false positive CI for some reason and this is really critical

Also, if we look into https://review.openstack.org/#/c/422731 https://ci.fuel-infra.org/job/mitaka.fuel-library.pkgs.ubuntu.neutron_vlan_ha/1855/
fuel-library9.0 version 9.0.0-1.8710 on the master node is vs 9.0.0-8720.2 built for the review

This actually means that we

1) do not destroy the snapshot of a successful run which leads to it being reverted on the subsequent run when only OSTF tests are run

2) we do not update packages on the master node during deployment phase, thus breaking things completely.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix merged to fuel-infra/jenkins-jobs (master)

Reviewed: https://review.fuel-infra.org/30004
Submitter: Dmitry Kaigarodsev <email address hidden>
Branch: master

Commit: 9561170492360874eb856bc091baa603b85c7f6c
Author: Dmitry Kaigarodsev <email address hidden>
Date: Fri Jan 20 11:46:00 2017

[fuel-ci]disable KEEP_BEFORE for fuel-library

Related-bug: #1658063
Change-Id: Id826448fb1082da2dbcf1d6f3e575dab71b8f8ac

Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

here is another set of jobs which actually have a workaround above merged already, although packages are not updated

https://ci.fuel-infra.org/job/mitaka.fuel-library.pkgs.ubuntu.neutron_vlan_ha/1861
https://ci.fuel-infra.org/job/mitaka.fuel-library.pkgs.ubuntu.neutron_vlan_ha/1862

Revision history for this message
Alexey Shtokolov (ashtokolov) wrote :

Raised to Critical due to blocked CI on stable/mitaka branch

Revision history for this message
Alexey Shtokolov (ashtokolov) wrote :

Looks like fix https://review.fuel-infra.org/30004 doesn't work.

E.g. https://ci.fuel-infra.org/job/mitaka.fuel-library.pkgs.ubuntu.neutron_vlan_ha/1862/
I see the fuel package on the env: fuel-library9.0.noarch 9.0.0-1.mos8710 @proposed
But should be fuel-library9.0-9.0.0-8720.2.gerrit422726.3.git56ba2b5.noarch.rpm

Revision history for this message
Roman Vyalov (r0mikiam) wrote :

1. https://review.fuel-infra.org/30004 is is not a fix its workaround.
2. its works for master branch , for stable branch was implemented another workaround https://review.fuel-infra.org/#/c/30016/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-library (stable/mitaka)

Related fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/423616

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-qa (stable/mitaka)

Related fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/423622

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-library (stable/mitaka)

Reviewed: https://review.openstack.org/423616
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=d600a302c63b8c9de2b9945a4469abaea2a01ec1
Submitter: Jenkins
Branch: stable/mitaka

commit d600a302c63b8c9de2b9945a4469abaea2a01ec1
Author: Vladimir Kuklin <email address hidden>
Date: Sat Jan 21 14:34:27 2017 +0300

    Set aux repos priority to 5

    Change-Id: I19932d6dc53e57347a5f496edc4ad16df8531a11
    Related-bug: #1658063

tags: added: in-stable-mitaka
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix proposed to fuel-infra/jenkins-jobs (master)

Related fix proposed to branch: master
Change author: Vladimir Kuklin <email address hidden>
Review: https://review.fuel-infra.org/30020

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix merged to fuel-infra/jenkins-jobs (master)

Reviewed: https://review.fuel-infra.org/30020
Submitter: Dmitry Kaigarodsev <email address hidden>
Branch: master

Commit: e3eec1041befe7bfeb75ac8dfebb542e185ad957
Author: Vladimir Kuklin <email address hidden>
Date: Sat Jan 21 16:43:50 2017

Update Fuel Master manifests for neutron_ha_vlan test

Change-Id: I3478a3a725b9ce06907dbe752a266572ded1bf12
Related-bug: #1658063

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-library (stable/newton)

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/432298

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-library (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/432299

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-library (master)

Change abandoned by Vladimir Kuklin (<email address hidden>) on branch: master
Review: https://review.openstack.org/432299

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-library (stable/newton)

Change abandoned by Vladimir Kuklin (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/432298

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-qa (stable/mitaka)

Change abandoned by Vladimir Kuklin (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/423622

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.