[SWARM][9.2] nova-compute service missed on the ol-based compute

[Vladimir Jigulin]

Test steps:
        1. Check required image.
        2. Revert snapshot 'ready_with_3_slaves'.
        3. Create a Fuel cluster.
        4. Update cluster nodes with required roles.
        5. Deploy the Fuel cluster.
        6. Run OSTF.
        7. Backup astute.yaml and ssh keys from compute.
        8. Boot compute with OL image.
        9. Prepare node for Puppet run.
        10. Execute modular tasks for compute. <= Fail
        11. Run OSTF.

Expected results: All services are up and available

Actual result:
Test failed with error
Expected service count is 5, but get 4 count, actual list [<Service: nova-cert>, <Service: nova-consoleauth>, <Service: nova-scheduler>, <Service: nova-conductor>]
traceback: http://paste.openstack.org/show/588721/

On controller:
nova-compute not present in "nova-manage service list":
http://paste.openstack.org/show/588722/

root@node-1:~# grep "DELETE.*os-services" /var/log/nova/nova-api.log
2016-11-10 00:24:13.537 29164 INFO nova.osapi_compute.wsgi.server [req-494bb311-531d-4fe5-8366-511043728cb5 02b6b9239caf4c3eaf53f01eb6b6778e 8fc775a023f14d799f10186cb7b5be80 - - -] 10.109.21.6 "DELETE /v2.1/os-services/8 HTTP/1.1" status: 204 len: 270 time: 0.0601859

On compute:
[root@ol-1 ~]# yum list installed|grep nova
python-novaclient.noarch 1:3.3.1-2.el7~mos1 @mos
[root@ol-1 ~]# yum list|grep nova-compute
openstack-nova-compute.noarch 1:13.0.0-3.el7~mos43 mos

Fuel build: 495
shotgun2 short-report: http://paste.openstack.org/show/588724/

Similar bug: https://bugs.launchpad.net/fuel/+bug/1617392
Ci fail: https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.ol/120/testReport/(root)/deploy_ol_compute_ha_one_controller_tun/deploy_ol_compute_ha_one_controller_tun/

[Oleksiy Molchanov (omolchanov)]

2016-11-10 00:20:56 +0000 Puppet (err): Execution of '/usr/bin/yum -d 0 -e 0 -y install python-nova' returned 1: Error: Package: python2-cryptography-1.3.1-3.el7.x86_64 (ol7_latest)
           Requires: python-cffi >= 1.4.1
           Available: python-cffi-1.1.2-3.el7~mos1.x86_64 (mos)
               python-cffi = 1.1.2-3.el7~mos1
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
2016-11-10 00:20:56 +0000 /Stage[main]/Nova/Package[python-nova]/ensure (err): change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install python-nova' returned 1: Error: Package:
 python2-cryptography-1.3.1-3.el7.x86_64 (ol7_latest)
           Requires: python-cffi >= 1.4.1
           Available: python-cffi-1.1.2-3.el7~mos1.x86_64 (mos)
               python-cffi = 1.1.2-3.el7~mos1
 You could try using --skip-broken to work around the problem

[Dmitry Teselkin (teselkin-d)]

Seems like mos-packaging issue

[Ivan Udovichenko (iudovichenko)]

Proposed fix: https://review.fuel-infra.org/#/c/28515/

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/28515
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0

Commit: a9cf57580b26459c49943a5a10cd3a516e9fb74b
Author: Ivan Udovichenko <email address hidden>
Date: Tue Nov 15 12:56:43 2016

[MOS 9.2] Update python-cryptography packages

Version: 1.0-1.el7~mos3

- Add python2-* packages.
- LP#1640771

Related-Bug: #1640771

Change-Id: If166dd2d5bceb27c701912386b5a127f7b681656

[Ivan Udovichenko (iudovichenko)]

We've merged python-cryptography packages.
They are placed in "proposed" repository.

Currently we add only package repository with stable 9.0 which is frozen.

# cat /etc/yum.repos.d/mos.repo
[mos]
name=mos
type=rpm-md
baseurl=http://perestroika-repo-tst.infra.mirantis.net/mos-repos/centos/mos9.0-centos7/os/x86_64/
gpgcheck=0
enabled=1
priority=5

Please add proposed package repository with the fix to yum.repos.d :
http://perestroika-repo-tst.infra.mirantis.net/mos-repos/centos/mos9.0-centos7/proposed/x86_64/Packages/python2-cryptography-1.0-1.el7~mos3.x86_64.rpm

[Ivan Udovichenko (iudovichenko)]

Ivan Remizov assured me that proposed package repository should be available (passed) to the compute node.

[Ivan Udovichenko (iudovichenko)]

CI team, please check:
https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.ol/126/console

[Vitaly Sedelnik (vsedelnik)]

The fix was merged at 11/15, please retest with any 9.2 snapshot after 11/16, see http://perestroika-repo-tst.infra.mirantis.net/mos-repos/centos/mos9.0-centos7/snapshots/

Setting to Fix Committed

[Vladimir Jigulin (vjigulin)]

#549 iso 11/23 still broken

https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.ol/lastCompletedBuild/testReport/(root)/deploy_ol_compute_ha_one_controller_tun/deploy_ol_compute_ha_one_controller_tun/

2016-11-23 23:03:46 +0000 Puppet (debug): Executing '/usr/bin/yum -d 0 -e 0 -y install python-nova'
2016-11-23 23:03:50 +0000 Puppet (err): Execution of '/usr/bin/yum -d 0 -e 0 -y install python-nova' returned 1: Error: Package: python2-cryptography-1.3.1-3.el7.x86_64 (ol7_latest)
           Requires: python-cffi >= 1.4.1
           Available: python-cffi-1.1.2-3.el7~mos1.x86_64 (mos)
               python-cffi = 1.1.2-3.el7~mos1
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
2016-11-23 23:03:50 +0000 /Stage[main]/Nova/Package[python-nova]/ensure (err): change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install python-nova' returned 1: Error: Package: python2-cryptography-1.3.1-3.el7.x86_64 (ol7_latest)
           Requires: python-cffi >= 1.4.1
           Available: python-cffi-1.1.2-3.el7~mos1.x86_64 (mos)
               python-cffi = 1.1.2-3.el7~mos1
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
2016-11-23 23:03:50 +0000 /Stage[main]/Nova/Package[nova-common] (notice): Dependency Package[python-nova] has failures: true

[Ivan Udovichenko (iudovichenko)]

As I can see, 9.0-proposed package repository wasn't added nor 9.0-updates
Please add them as I mentioned before.

[root@ol-1 ~]# ll /etc/yum.repos.d/
total 8
-rw-r--r--. 1 root root 157 Nov 23 22:56 mos.repo
-rw-r--r--. 1 root root 2560 Nov 23 22:56 public-yum-ol7.repo

[root@ol-1 ~]# cat /etc/yum.repos.d/mos.repo
[mos]
name=mos
type=rpm-md
baseurl=http://perestroika-repo-tst.infra.mirantis.net/mos-repos/centos/mos9.0-centos7/os/x86_64/
gpgcheck=0
enabled=1
priority=5

[Ivan Udovichenko (iudovichenko)]

I've checked it on 9.x.system_test.ubuntu.ol.134 env (error_deploy_ol_compute_ha_one_controller_tun snapshot).

https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.ol/lastCompletedBuild/console

[Oleksiy Molchanov (omolchanov)]

Reproduced on las ci run

--> Finished Dependency Resolution
Error: Package: python2-cryptography-1.3.1-3.el7.x86_64 (ol7_latest)
           Requires: python-cffi >= 1.4.1
           Available: python-cffi-1.1.2-3.el7~mos1.x86_64 (mos)
               python-cffi = 1.1.2-3.el7~mos1
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

[Timur Nurlygayanov (tnurlygayanov)]

Dmitry Kaigarodtsev [12:35 PM]
@tnurlygayanov it's included, check logs there

[12:35]
https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.ol/139/parameters/

[12:35]
 ```+ __repo_name='mos-proposed,deb http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-11-28-130322 mos9.0-proposed main restricted'
++ join '|' '' 'mos-proposed,deb http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-11-28-130322 mos9.0-proposed main restricted'
++ local '__sep=|'
++ local __head=
++ local '__tail=mos-proposed,deb http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-11-28-130322 mos9.0-proposed main restricted'
++ [[ -n '' ]]
++ echo 'mos-proposed,deb http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-11-28-130322 mos9.0-proposed main restricted'
+ EXTRA_DEB_REPOS='mos-proposed,deb http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-11-28-130322 mos9.0-proposed main restricted'```

[12:35]
https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.ol/lastCompletedBuild/console

[Timur Nurlygayanov (tnurlygayanov)]

The verification of the issue is blocked by another blocker issue:

https://bugs.launchpad.net/fuel/+bug/1640093

[Vladimir Jigulin (vjigulin)]

"ol-based compute" feature no longer supported in 9.x and 10.0. These tests are disabled, so this bug is not relevant anymore