Upgrades, rollback for bootstrap brakes slaves bootstrapping

Bug #1348166 reported by Evgeniy L
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Evgeniy L
5.0.x
Fix Committed
High
Evgeniy L

Bug Description

1. install 5.0 master node
2. run 5.0.1/5.1 upgrade, make it fail (add exception raising in post bootstrap upgarde section)
3. wait until rollback is done
4. try to reboot slave

Actual result:
Node bootstrapping fails

Expected result:
Node booted with bootstrap image

It happens because of symlink magic in cobbler container

[root@c3afdaa2107d nailgun]# ls -l /var/www/nailgun/
total 0
lrwxrwxrwx 1 root root 15 Jun 4 11:51 bootstrap -> /repo/bootstrap
lrwxrwxrwx 1 root root 12 Jun 4 11:51 centos -> /repo/centos
lrwxrwxrwx 1 root root 12 Jun 4 11:51 docker -> /repo/docker
lrwxrwxrwx 1 root root 5 Jul 24 12:44 dump -> /dump
lrwxrwxrwx 1 root root 22 Jun 4 11:51 puppet-slave.tgz -> /repo/puppet-slave.tgz
lrwxrwxrwx 1 root root 12 Jun 4 11:51 ubuntu -> /repo/ubuntu
[root@c3afdaa2107d nailgun]# ls -l /repo/
total 1236
drwxr-xr-x 2 root root 38 Jun 4 11:31 5.0_bootstrap
drwxr-xr-x 4 root root 32 Jul 23 13:55 5.1
drwxr-xr-x 2 root root 38 Jun 4 13:25 5.1_bootstrap
lrwxrwxrwx 1 root root 31 Jul 24 12:46 bootstrap -> /var/www/nailgun/5.0_bootstrap/
drwxr-xr-x. 3 root root 20 Jun 4 11:31 centos
drwxr-xr-x. 4 root root 49 Jun 4 11:32 docker
drwxr-xr-x 2 root root 6 Jun 4 11:51 dump
-rwxr-xr-x. 1 root root 1265298 Jun 4 11:31 puppet-slave.tgz
drwxr-xr-x. 3 root root 20 Jun 4 11:31 ubuntu

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "5.1"
  api: "1.0"
  build_number: "70"
  build_id: "2014-07-23_15-32-11"
  astute_sha: "fd9b8e3b6f59b2727b1b037054f10e0dd7bd37f1"
  fuellib_sha: "fb0e84c954a33c912584bf35054b60914d2a2360"
  ostf_sha: "c1b60d4bcee7cd26823079a86e99f3f65414498e"
  nailgun_sha: "06b9becc1e6831351aac693290ac619c92e56e99"
  fuelmain_sha: "7b03801a70506b1c9efc2aec8a6203bc916a5b89"

Tags: upgrade
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/109285

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/5.0)

Fix proposed to branch: stable/5.0
Review: https://review.openstack.org/109293

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/109285
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=a74fa60388d658aff04aedbc5d7ba7d091713c56
Submitter: Jenkins
Branch: master

commit a74fa60388d658aff04aedbc5d7ba7d091713c56
Author: Evgeniy L <email address hidden>
Date: Thu Jul 24 17:03:28 2014 +0400

    Upgrades, fix rollback for bootstrap engine

    * changed rollback strategy for bootstrap
      upgrade engine, in previous strategy we
      made symlink from previous version of
      bootstrap to /var/www/nailgun/bootstrap
      it doesn't work because in 5.0 version
      cobbler container mounts /var/www/nailgun
      directory as /repo, as result symlinks
      which we make on the host system are
      broken inside of container.

      Instead of symlink command we rename
      previous version to /var/www/nailgun/bootstrap

    * use lexists instead of exists because
      exists returns False on broken symlinks

    * fixed docker engine rollback, there
      was wrong definition of previous version
      of fuel.

    Change-Id: Id0b50835410e005fde76f421981b4616211ddcbf
    Closes-bug: #1348166

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (stable/5.0)

Reviewed: https://review.openstack.org/109293
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=3f19a97e4f250c49742655ef94edd6e33cc9845a
Submitter: Jenkins
Branch: stable/5.0

commit 3f19a97e4f250c49742655ef94edd6e33cc9845a
Author: Evgeniy L <email address hidden>
Date: Thu Jul 24 17:03:28 2014 +0400

    Upgrades, fix rollback for bootstrap engine

    * changed rollback strategy for bootstrap
      upgrade engine, in previous strategy we
      made symlink from previous version of
      bootstrap to /var/www/nailgun/bootstrap
      it doesn't work because in 5.0 version
      cobbler container mounts /var/www/nailgun
      directory as /repo, as result symlinks
      which we make on the host system are
      broken inside of container.

      Instead of symlink command we rename
      previous version to /var/www/nailgun/bootstrap

    * use lexists instead of exists because
      exists returns False on broken symlinks

    * fixed docker engine rollback, there
      was wrong definition of previous version
      of fuel

    Change-Id: Id0b50835410e005fde76f421981b4616211ddcbf
    Closes-bug: #1348166

Dmitry Pyzhov (dpyzhov)
no longer affects: fuel/5.1.x
Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :

verified on fuel-5.1-upgrade-9-2014-09-17_04-49-39.tar.lrz

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.