Juju 1.25.10: Conversation scope is missing sometimes - causes relation hook failure
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
juju-core |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
Strange to describe bug. Essentially, it looks like 1.25.10 (and possible 1.25.9) have a bug where during a hook execution the relation is essentially missing for the hook that was invoked during that hook's execution.
This is happening with reactive charms, but doesn't occur (in my testing) with 2.1-beta5. I don't know if it's happening with non-reactive charms.
The problem is occurring with Amulet tests which bootstrap an environment, deploy the charms and relate them as quickly as possible. It doesn't happen every time; perhaps 80% of the time it will fail.
One other oddity, which may be pertinent, is that if you bootstrap the environment, leave it for a good 10 minutes or so and then run the Amulet tests without re-bootstrapping it then completes without issue.
Relevant evidence from ceph-fs charm (from /var/log/
2017-02-09 14:59:18 INFO worker.uniter.jujuc server.go:172 running hook tool "juju-log" ["-l" "INFO" "Invoking reactive handler: hooks/relations
2017-02-09 14:59:18 DEBUG worker.uniter.jujuc server.go:173 hook context id "ceph-fs/
...
snip
...
2017-02-09 14:59:20 INFO worker.uniter.jujuc server.go:172 running hook tool "relation-ids" ["--format=json" "mon"]
2017-02-09 14:59:20 DEBUG worker.uniter.jujuc server.go:173 hook context id "ceph-fs/
2017-02-09 14:59:20 INFO worker.uniter.jujuc server.go:172 running hook tool "status-set" ["blocked" "No MDS detected using current configuration"]
2017-02-09 14:59:20 DEBUG worker.uniter.jujuc server.go:173 hook context id "ceph-fs/
2017-02-09 14:59:20 DEBUG juju.worker.
2017-02-09 14:59:21 INFO juju.worker.
2017-02-09 14:59:21 INFO juju.worker.
2017-02-09 14:59:21 INFO juju.worker.
and from the manila charm (var/log/
2017-02-08 15:46:53 DEBUG juju.worker.uniter modes.go:31 [AGENT-STATUS] executing: running amqp-relation-
...
snip
...
2017-02-08 15:46:57 DEBUG worker.uniter.jujuc server.go:173 hook context id "manila/
2017-02-08 15:46:57 INFO juju-log amqp:5: Writing file /etc/memcached.conf root:root 444
2017-02-08 15:47:04 INFO worker.uniter.jujuc server.go:172 running hook tool "juju-log" ["Application Version: 2.0.0"]
2017-02-08 15:47:04 DEBUG worker.uniter.jujuc server.go:173 hook context id "manila/
2017-02-08 15:47:04 INFO juju-log amqp:5: Application Version: 2.0.0
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO worker.uniter.jujuc server.go:172 running hook tool "status-set" ["blocked" "Services not running that should be: manila-scheduler, manila-data"]
2017-02-08 15:47:04 DEBUG worker.uniter.jujuc server.go:173 hook context id "manila/
2017-02-08 15:47:04 DEBUG juju.worker.
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO amqp-relation-
2017-02-08 15:47:04 INFO juju.worker.
2017-02-08 15:47:04 ERROR juju.worker.
Resources:
OSCI output for ceph-fs: https:/
OSCI output for manila-fs: https:/
edited: Changed 1.2.9 -> 1.25.9 and 1.2.19 -> 1.25.10 as prompted by @anastasia below.
summary: |
- Juju 1.2.10: Conversation scope is missing sometimes - causes relation + Juju 1.25.10: Conversation scope is missing sometimes - causes relation hook failure |
@Alex,
Please confirm - when you say 1.2.9/12.10, you mean 1.25.9/1.25.10?
It does sound that there is a timing issue, potentially a race. As in your description, giving bootstrap enough time to settle seem to allow you to run Amulet testing successful, we cannot consider this issue as Critical and are unlikely to address it in 1.25 series.
As you say, the problem does not occur with Juju 2.1.