Failed tasks: Task[glance/1], Task[glance/3], Task[glance/6] Stopping the deployment process!'

[Ann Taraday]

Reproduced on https://ci.fuel-infra.org/job/10.0-community.main.ubuntu.bvt_2/471/console

2016-08-12 01:27:48,866 - ERROR decorators.py:123 -- Traceback (most recent call last):
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 117, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/tests/test_ceph.py", line 505, in ceph_rados_gw
    self.fuel_web.deploy_cluster_wait(cluster_id)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 459, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 444, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 495, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 502, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 386, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 931, in deploy_cluster_wait
    self.assert_task_success(task, interval=interval, timeout=timeout)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/__init__.py", line 60, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-community.main.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 333, in assert_task_success
    task["name"], task['status'], 'ready', _message(task)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/asserts.py", line 55, in assert_equal
    raise ASSERTION_ERROR(message)
AssertionError: Task 'deploy' has incorrect status. error != ready, 'Deployment has failed. All nodes are finished. Failed tasks: Task[glance/1], Task[glance/3], Task[glance/6] Stopping the deployment process!'

[Oleksiy Molchanov (omolchanov)]

10.109.15.6 2016-08-12T01:19:58.837336 err: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o APT::Get::AllowUnauthenticated=1 install glance-registry' returned 100: Reading package lists...
10.109.15.6 2016-08-12T01:19:58.837863 err: Building dependency tree...
10.109.15.6 2016-08-12T01:19:58.838476 err: Reading state information...
10.109.15.6 2016-08-12T01:19:58.839159 err: Some packages could not be installed. This may mean that you have
10.109.15.6 2016-08-12T01:19:58.839746 err: requested an impossible situation or if you are using the unstable
10.109.15.6 2016-08-12T01:19:58.840463 err: distribution that some required packages have not yet been created
10.109.15.6 2016-08-12T01:19:58.841561 err: or been moved out of Incoming.
10.109.15.6 2016-08-12T01:19:58.842434 err: The following information may help to resolve the situation:
10.109.15.6 2016-08-12T01:19:58.843212 err:
10.109.15.6 2016-08-12T01:19:58.843851 err: The following packages have unmet dependencies:
10.109.15.6 2016-08-12T01:19:58.844512 err: glance-registry : Depends: glance-common (= 2:13.0.0~b2-0ubuntu1~cloud0) but it is not going to be installed
10.109.15.6 2016-08-12T01:19:58.845233 err: E: Unable to correct problems, you have held broken packages.

[Igor Yozhikov (iyozhikov)]

Community build is using UCA packages not MOS - so please fail a bug to Canonical

[Vladimir Kozhukalov (kozhukalov)]

Community ISO uses a mixture of UCA and MOS and one of MOS packages was updated and made MOS incompatible with UCA. We also need to update some other packages in MOS to fix this issue.

[Igor Yozhikov (iyozhikov)]

Confirming that fuel community build is using mixed packages as from mos as from canonical package repositories. To be able to use canonical packages for OS and dependencies required to setup proper apt pinning. This pin setup should disable any packages from mos package repositories except of very necessary packages like mysql + galera, haproxy, erlang, rabbit.
Another option is - to use snapshot of package repository without introduced updates according to Newton requirements.

To be able to use clean UCA - it is required to analyse all current fuel features and create a switch option for deployment tasks. This set op options will define amount of features depending on chosen distribution.

[Alex Schultz (alex-schultz)]

We already configure pinning but we did not merge the UCA Xenial pin change yet for the ancillary packages, https://review.openstack.org/#/c/347034/

We shouldn't be picking up MOS openstack packages and if we are we need to know which ones are coming from MOS and why.

[Dmitry Teselkin (teselkin-d)]

We're testing a bit different pinning schema:

* set pin-priority == 550 for all packages with origin=Mirantis
* set pin-priority == 499 for all python packages with origin=Mirantis

This should satisfy all requirements:
* to not install our python packages
* to install any package (except python-*) from our repo if
  - package not exists at all
  - package exists only in Xenial (not UCA) repositories

Using that schema there is no need to pin rabbitmq, haproxy, and so on, because they will be pinned from repo pin-priority.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Alex Schultz (<email address hidden>) on branch: master
Review: https://review.openstack.org/347034