IndexError: list index out of range when upgrading from trusty to xenial

Bug #1765805 reported by Felipe Reyes on 2018-04-20
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Charm Helpers
High
Felipe Reyes
OpenStack ceph-mon charm
High
Felipe Reyes
OpenStack ceph-osd charm
High
Felipe Reyes

Bug Description

[Impact]

When upgrading a cluster from trusty-mitaka to xenial(-distro) the config-changed hook fails when trying to figure out the ceph version

unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed Traceback (most recent call last):
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed File "/var/lib/juju/agents/unit-ceph-osd-1/charm/hooks/config-changed", line 562, in <module>
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed hooks.execute(sys.argv)
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed File "/var/lib/juju/agents/unit-ceph-osd-1/charm/hooks/charmhelpers/core/hookenv.py", line 801, in execute
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed self._hooks[hook_name]()
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed File "/var/lib/juju/agents/unit-ceph-osd-1/charm/hooks/charmhelpers/contrib/hardening/harden.py", line 79, in _harden_inner2
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed return f(*args, **kwargs)
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed File "/var/lib/juju/agents/unit-ceph-osd-1/charm/hooks/config-changed", line 351, in config_changed
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed check_for_upgrade()
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed File "/var/lib/juju/agents/unit-ceph-osd-1/charm/hooks/config-changed", line 93, in check_for_upgrade
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed 'distro')
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed File "lib/ceph/utils.py", line 2526, in resolve_ceph_version
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed os_release = get_os_codename_install_source(source)
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed File "/var/lib/juju/agents/unit-ceph-osd-1/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 309, in get_os_codename_install_source
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed ca_rel = ca_rel.split('%s-' % ubuntu_rel)[1].split('/')[0]
unit-ceph-osd-1: 14:28:23 DEBUG unit.ceph-osd/1.config-changed IndexError: list index out of range
unit-ceph-osd-1: 14:28:23 ERROR juju.worker.uniter.operation hook "config-changed" failed: exit status 1

[Test Case]

cat <<EOF > ceph-trusty-mitaka.yaml
series: trusty
applications:
  ceph-mon:
    charm: "cs:~openstack-charmers-next/ceph-mon"
    num_units: 3
    options:
      expected-osd-count: 3
      source: cloud:trusty-mitaka
  ceph-osd:
    charm: "cs:~openstack-charmers-next/ceph-osd"
    num_units: 3
    options:
      osd-devices: /dev/vdb
      osd-reformat: true
      ephemeral-unmount: "/mnt"
      source: cloud:trusty-mitaka
relations:
  - [ "ceph-osd:mon", "ceph-mon:osd" ]
EOF
juju deploy ./ceph-trusty-mitaka.yaml
# once things are deployed
./upgrade-series.sh ceph-osd/0 # script available at https://gist.github.com/freyes/5327cc554bd18d4f9f6059a9a1af153b
juju ssh ceph-osd/1 sudo reboot

Expected result: the workload for ceph-osd/1 is active

Actual result: workload is in error state, workload message is ' hook failed: "config-changed" '

Felipe Reyes (freyes) on 2018-04-20
tags: added: sts
tags: added: openstack
Frode Nordahl (fnordahl) on 2018-04-24
Changed in charm-helpers:
status: New → Triaged
importance: Undecided → High
Changed in charm-ceph-mon:
status: New → Triaged
importance: Undecided → High
Changed in charm-ceph-osd:
status: New → Triaged
importance: Undecided → High
Felipe Reyes (freyes) wrote :
Changed in charm-helpers:
status: Triaged → Fix Committed
assignee: nobody → Felipe Reyes (freyes)
Changed in charm-ceph-mon:
assignee: nobody → Felipe Reyes (freyes)
Changed in charm-ceph-osd:
assignee: nobody → Felipe Reyes (freyes)
Changed in charm-ceph-mon:
status: Triaged → In Progress
Changed in charm-ceph-osd:
status: Triaged → In Progress

Reviewed: https://review.openstack.org/565874
Committed: https://git.openstack.org/cgit/openstack/charm-ceph-osd/commit/?id=644dea83defe44210bffb2d22cba5aab994c5eda
Submitter: Zuul
Branch: master

commit 644dea83defe44210bffb2d22cba5aab994c5eda
Author: Felipe Reyes <email address hidden>
Date: Wed May 2 16:06:36 2018 -0300

    Sync charm-helpers

    Change-Id: I8e565d86b15c959ba224136b15e9db037892f6f5
    Closes-Bug: #1765805

Changed in charm-ceph-osd:
status: In Progress → Fix Committed

Change abandoned by Felipe Reyes (<email address hidden>) on branch: master
Review: https://review.openstack.org/565865

James Page (james-page) on 2018-06-06
Changed in charm-ceph-osd:
milestone: none → 18.05
David Ames (thedac) on 2018-06-11
Changed in charm-ceph-osd:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers