Resume action appears to fail with 'tuple.index(x): x not in tuple'

[Andrea Ieri]

This occurred with v281.

How to reproduce:

$ juju run-action --wait neutron-api/0 pause
unit-neutron-api-0:
  id: 16235bb4-8286-4f11-8a5a-9c093a544352
  status: completed
  timing:
    completed: 2019-11-21 11:16:06 +0000 UTC
    enqueued: 2019-11-21 11:15:54 +0000 UTC
    started: 2019-11-21 11:15:55 +0000 UTC
  unit: neutron-api/0

$ juju run-action --wait neutron-api/0 resume
unit-neutron-api-0:
  id: eed41740-b805-4e50-8604-291775a60829
  message: 'tuple.index(x): x not in tuple'
  status: failed
  timing:
    completed: 2019-11-21 11:17:13 +0000 UTC
    enqueued: 2019-11-21 11:17:00 +0000 UTC
    started: 2019-11-21 11:17:02 +0000 UTC
  unit: neutron-api/0

Excerpt from the debug logs:

2019-11-21 11:17:04 DEBUG resume Removed /etc/systemd/system/apache2.service.
2019-11-21 11:17:04 DEBUG resume Synchronizing state of apache2.service with SysV service script with /lib/systemd/systemd-sysv-install.
2019-11-21 11:17:04 DEBUG resume Executing: /lib/systemd/systemd-sysv-install enable apache2
2019-11-21 11:17:04 DEBUG resume inactive
2019-11-21 11:17:04 DEBUG resume Removed /etc/systemd/system/haproxy.service.
2019-11-21 11:17:04 DEBUG resume Synchronizing state of haproxy.service with SysV service script with /lib/systemd/systemd-sysv-install.
2019-11-21 11:17:04 DEBUG resume Executing: /lib/systemd/systemd-sysv-install enable haproxy
2019-11-21 11:17:05 DEBUG resume failed
2019-11-21 11:17:05 DEBUG resume Removed /etc/systemd/system/neutron-server.service.
2019-11-21 11:17:05 DEBUG resume Synchronizing state of neutron-server.service with SysV service script with /lib/systemd/systemd-sysv-install.
2019-11-21 11:17:05 DEBUG resume Executing: /lib/systemd/systemd-sysv-install enable neutron-server
2019-11-21 11:17:06 DEBUG resume inactive
2019-11-21 11:17:06 DEBUG resume Removed /etc/systemd/system/memcached.service.
2019-11-21 11:17:06 DEBUG resume Synchronizing state of memcached.service with SysV service script with /lib/systemd/systemd-sysv-install.
2019-11-21 11:17:06 DEBUG resume Executing: /lib/systemd/systemd-sysv-install enable memcached
2019-11-21 11:17:06 DEBUG resume inactive
2019-11-21 11:17:11 DEBUG juju-log Generating template context for amqp
2019-11-21 11:17:12 DEBUG resume ERROR no relation id specified
2019-11-21 11:17:12 DEBUG juju-log Generating template context for identity-service

Since haproxy appears to actually have been unmasked and started, this seems to be only cosmetic. Although the juju status output keeps reporting the unit to be paused, this will be corrected as soon as the update-status hook runs.

[Alex Kavanagh (ajkavanagh)]

Hi Andrea

I've note been able to reproduce this bug. Please could provide a bundle which can be used to reproduce it? Thanks.

[Andrea Ieri (aieri)]

You're right... this doesn't happen on my openstack-on-lxd cloud. It must be something specific to the cloud I'm working on.

You can find a redacted bundle here: https://pastebin.canonical.com/p/QRPFKWJnPj/

I have tested this once again on the affected cloud and I can consistently reproduce the issue.

[Alex Kavanagh (ajkavanagh)]

I'm setting this to incomplete as this may no longer be a bug as some work has been done in this area. If it's still a problem please comment on the bug. Thanks.

[Launchpad Janitor (janitor)]

[Expired for OpenStack neutron-api charm because there has been no activity for 60 days.]