[9.x swarm] cluster deployment with detached haproxy fails due to the broken package "percona-xtrabackup"

Bug #1651182 reported by Sergey Novikov
20
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Vladimir Kuklin
8.0.x
Fix Committed
High
MOS Maintenance
Mitaka
Fix Released
High
Ivan Suzdal
Newton
Fix Released
High
Vladimir Kuklin
Ocata
Fix Committed
High
Vladimir Kuklin

Bug Description

Detailed bug description:
the issue was found by https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.plugins.thread_separate_haproxy/153/testReport/(root)/separate_haproxy/separate_haproxy/

Steps to reproduce:
            1. install external-lb and detach-haproxy plugins
            2. Bootstrap 3 slaves from default nodegroup
            3. Create cluster with Neutron VXLAN and custom nodegroups
            4. Run 'fuel-mirror' to replace cluster repositories
               with local mirrors
            5. Bootstrap 2 slaves nodes from second nodegroup
               and one node from third node group
            6. Enable plugins for cluster
            7. Add 2 controllers from default nodegroup and 1 controller
               from second node group
            8. Add 1 compute+cinder from default node group
               and 1 compute+cinder from second node group
            9. Add haproxy node from third node group
            10. Verify networks
            11. Deploy cluster

Deployment fails on Task[primary-database]
part of puppet's log http://paste.openstack.org/show/592811/

Additional information:
snapshot #646

Revision history for this message
Sergey Novikov (snovikov) wrote :
Changed in fuel:
assignee: nobody → Fuel Sustaining (fuel-sustaining-team)
status: New → Confirmed
Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

The log seems to show issues with repository connectivity to the repos. I would firstly wait for a reproduction of the issue.

Changed in fuel:
status: Confirmed → Incomplete
Revision history for this message
Nastya Urlapova (aurlapova) wrote :

Vlamimir, the new fail you can find by link:
https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.plugins.thread_separate_haproxy/153/

>>> (/Stage[main]/Galera/Package[percona-xtrabackup]/ensure) change from purged to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o APT::Get::AllowUnauthenticated=1 install percona-xtrabackup' returned 100: Reading package lists...

Revision history for this message
Nastya Urlapova (aurlapova) wrote :

Version 9.2 snapshot #684

Changed in fuel:
status: Incomplete → Opinion
status: Opinion → Confirmed
Revision history for this message
Nastya Urlapova (aurlapova) wrote :
Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

Okay, the issue is that the package is being installed before we install the client package that replaces the upstream mysql one, i. e. libmysqlclient18 or its replacement should be present before we try to install libdbd-mysql-perl

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/416587

Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Vladimir Kuklin (vkuklin)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/417439

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/417440

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/416587
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=7583f5cb17ededd3eccebe12fb04025d8dbe9e6a
Submitter: Jenkins
Branch: master

commit 7583f5cb17ededd3eccebe12fb04025d8dbe9e6a
Author: Vladimir Kuklin <email address hidden>
Date: Wed Jan 4 18:27:58 2017 +0300

    Fix apt discrepancy around mysql client package

    When we run galera/mysql packages installation
    we can get into dependencies issues with mysql
    client package as it is not installed at the time
    when perl dbd module requires it. At the same time
    libperl-dbd-mysql is depending on it, so it fails.
    Instead we need to ensure that mysql client is installed
    along with its libraries prior to installing additional
    packages for galera.

    Change-Id: Idc91a399a46cd63c2201660bb248c0a0ad3fb9b8
    Closes-bug: #1651182

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/mitaka)

Reviewed: https://review.openstack.org/417439
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=abeaf1f50a8a93515aa5c6fcdc14b06ed46aa6b5
Submitter: Jenkins
Branch: stable/mitaka

commit abeaf1f50a8a93515aa5c6fcdc14b06ed46aa6b5
Author: Vladimir Kuklin <email address hidden>
Date: Wed Jan 4 18:27:58 2017 +0300

    Fix apt discrepancy around mysql client package

    When we run galera/mysql packages installation
    we can get into dependencies issues with mysql
    client package as it is not installed at the time
    when perl dbd module requires it. At the same time
    libperl-dbd-mysql is depending on it, so it fails.
    Instead we need to ensure that mysql client is installed
    along with its libraries prior to installing additional
    packages for galera.

    Change-Id: Idc91a399a46cd63c2201660bb248c0a0ad3fb9b8
    Closes-bug: #1651182

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-mirror (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/418990

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/newton)

Reviewed: https://review.openstack.org/417440
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=f9f7ec7c4cf9aea5c2888d76d4cba957c4527c22
Submitter: Jenkins
Branch: stable/newton

commit f9f7ec7c4cf9aea5c2888d76d4cba957c4527c22
Author: Vladimir Kuklin <email address hidden>
Date: Wed Jan 4 18:27:58 2017 +0300

    Fix apt discrepancy around mysql client package

    When we run galera/mysql packages installation
    we can get into dependencies issues with mysql
    client package as it is not installed at the time
    when perl dbd module requires it. At the same time
    libperl-dbd-mysql is depending on it, so it fails.
    Instead we need to ensure that mysql client is installed
    along with its libraries prior to installing additional
    packages for galera.

    Change-Id: Idc91a399a46cd63c2201660bb248c0a0ad3fb9b8
    Closes-bug: #1651182

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-mirror (stable/mitaka)

Reviewed: https://review.openstack.org/418990
Committed: https://git.openstack.org/cgit/openstack/fuel-mirror/commit/?id=c0d6c904bbcb83e66c1d6ff5a3bdba6e9a407c7b
Submitter: Jenkins
Branch: stable/mitaka

commit c0d6c904bbcb83e66c1d6ff5a3bdba6e9a407c7b
Author: Vladimir Kuklin <email address hidden>
Date: Wed Jan 11 18:07:56 2017 +0300

    Add libmysqlclient18 to the partial mirror pkg list

    Otherwise it will not be mirrored and it will be marked
    as a virtual package by APT, for which there is no ability
    to do provides via mysql-wsrep-libmysqlclient18 package
    which will break mysql library installation dependencies

    Change-Id: I487032cfa2acc9e22c6b8df46fa5d11019debd3b
    Closes-bug: #1651182

Revision history for this message
Ekaterina Shutova (eshutova) wrote :
tags: added: on-verification
tags: removed: on-verification
Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

The issue itself cannot be resolved without changing the mirror creation workflow. We need to download libmysqlclient18 without actually using MOS repos at the first time. Then we need to create mirrors again with MOS mirrors enabled. Then the mirrors will be complete and include all the required packages.

Here is how it should look like to make things work

http://paste.openstack.org/show/595182/

tags: added: docs release-notes
Revision history for this message
Vladimir Khlyunev (vkhlyunev) wrote :

QA team can not do anything with missing package in repo - probably we should put it to 9.2 repos instead of creating new spike in tests

Revision history for this message
Roman Vyalov (r0mikiam) wrote :

moved to mos linux , because the bug related to build new packages to 9.2 (according to comment from QA)

Revision history for this message
Dmitry Teselkin (teselkin-d) wrote :

percona-xtrabackup / libdb-mysql-perl are upstream (Ubuntu) packages, we're not rebuilding them. There is no excuse to adding a new package into our repositories without introducing new functionality / fixing bugs in packages. This should be fixed on fuel-qa / fuel-build side, so moving to fuel-qa because of [1].

[1] https://bugs.launchpad.net/fuel/+bug/1651182/comments/16

Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

guys, this bug requires thorough refactoring of how be build repositories. more precisely, we need to do it in a different way as I described in https://bugs.launchpad.net/fuel/+bug/1651182/comments/16

the mirror should be built with 2 passes of fuel-mirror script with different configuration options. Thus, this bug is a won't fix with release notes based on my comment

tags: added: release-notes-done
removed: release-notes
tags: added: area-docs
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-library 11.0.0.0rc1

This issue was fixed in the openstack/fuel-library 11.0.0.0rc1 release candidate.

Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

It's very sad that nobody considered a proper fix for this. After short investigation we've found out that the problem is apparently might be properly solved, so I'm going to re-open it for 9.2-mu-1.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix merged to packages/trusty/mysql-wsrep-5.6 (9.0)

Reviewed: https://review.fuel-infra.org/30000
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0

Commit: 949a6e053f9ce3c75aa81ff10c944629e85898b7
Author: Ivan Suzdal <email address hidden>
Date: Tue Mar 21 13:20:56 2017

Add libmysqlclient18 metapackage

Related-Bug: #1651182

Change-Id: I221beafa7d2760b919a676f70dbcce0e6c370c18

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix proposed to packages/trusty/mysql-wsrep-5.6 (mcp/1.0/mitaka)

Related fix proposed to branch: mcp/1.0/mitaka
Change author: Ivan Suzdal <email address hidden>
Review: https://review.fuel-infra.org/32408

Revision history for this message
TatyanaGladysheva (tgladysheva) wrote :
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to packages/trusty/mysql-wsrep-5.6 (8.0)

Reviewed: https://review.fuel-infra.org/35867
Submitter: Pkgs Jenkins <email address hidden>
Branch: 8.0

Commit: 7f7d79e0816961acec585dbfb7f0c4b1f2a089af
Author: Ivan Suzdal <email address hidden>
Date: Fri Jul 14 10:06:26 2017

MySQL client metapackage and break condition

This patch introduces two changes:
 - added the break condition statement for mysql-server-wsrep-core-5.6
   which is needed to allow easy upgrades from pre 5.6 mysql-wsrep
 - added the libmysqlclient18 metapackage which is needed to avoid
   conflicts during installation of 3rd-party packages which depend on
   the real libmysqlclient18 package and cannot be satisfied with the
   virtual one

Closes-Bug: #1651182
Change-Id: Id3794f47ec6d45cdaeaba9943cb89e9eb22c202c

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.