OpenStack Snap

sunbeam cluster join fails with Error: Timed out while waiting for units openstack-hypervisor/1 to be ready

Bug #2065866 reported by Nobuto Murata
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Snap
Fix Committed
Medium
Nobuto Murata

Bug Description

$ snap list openstack
Name Version Rev Tracking Publisher Notes
openstack 2024.1 503 2024.1/edge canonical✓ -

The cluster join times out at 20 min. However, it took 21 minutes and 14 seconds in my case.

$ time sunbeam cluster join --role control --role compute --role storage --token eyJuYW1lIjoic3VuYmVhbS0yLmxvY2FsZG9tYWluIiwic2VjcmV0IjoiNzFjZDBhN2RjYTRjNzU3ODQ0OWRiZTUzNTJhNjgxOTY0NmEzY2E2NDVlZTI1YjRmOWIwZWQwY2UxNzZlYWEyOCIsImZpbmdlcnByaW50IjoiMjIyNTQyYjU3YTVmMGJhN2RmZTMyYjA1NDBjZWQyYzcwNjkzMDc3ZWU1NTg2YmFjZWM1MzI1Mjg4YjA4NGM4OSIsImpvaW5fYWRkcmVzc2VzIjpbIjEwLjAuMTIzLjI0MTo3MDAwIl19
⠼ Adding Openstack Hypervisor unit to machine(s) ... Timed out while waiting for units openstack-hypervisor/1 to be ready
Error: Timed out while waiting for units openstack-hypervisor/1 to be ready

real 20m13.762s
user 0m22.555s
sys 0m2.121s

Time Type Status Message
16 May 2024 08:46:07Z juju-unit allocating
16 May 2024 08:46:07Z workload waiting waiting for machine
16 May 2024 08:46:07Z workload waiting installing agent
16 May 2024 08:46:08Z workload waiting agent initialising
16 May 2024 08:46:11Z workload maintenance installing charm software
16 May 2024 08:46:11Z juju-unit executing running install hook
16 May 2024 08:46:13Z workload waiting no status set yet
16 May 2024 08:46:14Z workload blocked (ovsdb-cms) integration missing
16 May 2024 08:46:14Z juju-unit executing running identity-credentials-relation-created hook
16 May 2024 08:46:15Z workload waiting no status set yet
16 May 2024 08:46:15Z workload blocked (ovsdb-cms) integration missing
16 May 2024 08:46:15Z workload active
16 May 2024 08:46:15Z workload blocked (nova-service) integration missing
16 May 2024 08:46:15Z workload active
16 May 2024 08:46:15Z workload waiting (certificates) integration incomplete
16 May 2024 08:46:15Z workload blocked (amqp) integration missing
16 May 2024 08:46:16Z workload waiting (certificates) integration incomplete
16 May 2024 08:46:16Z workload blocked (identity-credentials) integration missing
16 May 2024 08:46:35Z workload waiting (certificates) integration incomplete
16 May 2024 08:46:35Z juju-unit executing running nova-service-relation-created hook
16 May 2024 08:46:36Z workload waiting no status set yet
16 May 2024 08:46:36Z workload active
16 May 2024 08:46:56Z workload waiting (certificates) integration incomplete
16 May 2024 08:46:57Z juju-unit executing running ovsdb-cms-relation-created hook
16 May 2024 08:46:57Z workload waiting no status set yet
16 May 2024 08:46:57Z workload active
16 May 2024 08:47:17Z workload waiting (certificates) integration incomplete
16 May 2024 08:47:17Z juju-unit executing running receive-ca-cert-relation-created hook
16 May 2024 08:47:17Z workload waiting no status set yet
16 May 2024 08:47:18Z workload active
16 May 2024 08:47:37Z workload waiting (certificates) integration incomplete
16 May 2024 08:47:37Z juju-unit executing running amqp-relation-created hook
16 May 2024 08:47:38Z workload waiting no status set yet
16 May 2024 08:47:38Z workload active
16 May 2024 08:47:57Z workload waiting (certificates) integration incomplete
16 May 2024 08:47:58Z juju-unit executing running certificates-relation-created hook
16 May 2024 08:47:58Z workload waiting no status set yet
16 May 2024 08:47:58Z workload active
16 May 2024 08:48:18Z workload waiting (certificates) integration incomplete
16 May 2024 08:48:18Z juju-unit executing running ceph-access-relation-created hook
16 May 2024 08:48:19Z workload waiting no status set yet
16 May 2024 08:48:19Z workload active
16 May 2024 08:48:38Z workload waiting (certificates) integration incomplete
16 May 2024 08:48:39Z juju-unit executing running leader-settings-changed hook
16 May 2024 08:48:39Z workload waiting no status set yet
16 May 2024 08:48:39Z workload active
16 May 2024 08:48:59Z workload waiting (certificates) integration incomplete
16 May 2024 08:48:59Z juju-unit executing running config-changed hook
16 May 2024 08:49:00Z workload waiting no status set yet
16 May 2024 08:49:00Z workload active
16 May 2024 08:49:19Z workload waiting (certificates) integration incomplete
16 May 2024 08:49:27Z juju-unit executing running start hook
16 May 2024 08:49:48Z juju-unit executing running identity-credentials-relation-changed hook
16 May 2024 08:50:16Z juju-unit executing running receive-ca-cert-relation-joined hook for cert-distributor/0
16 May 2024 08:50:38Z juju-unit executing running receive-ca-cert-relation-changed hook for cert-distributor/0
16 May 2024 08:51:06Z juju-unit executing running nova-service-relation-changed hook
16 May 2024 08:51:34Z juju-unit executing running ovsdb-cms-relation-joined hook for ovn-relay/0
16 May 2024 08:51:55Z juju-unit executing running nova-service-relation-joined hook for nova/0
16 May 2024 08:52:15Z juju-unit executing running amqp-relation-changed hook
16 May 2024 08:52:43Z juju-unit executing running amqp-relation-joined hook for rabbitmq/0
16 May 2024 08:53:04Z juju-unit executing running amqp-relation-changed hook for rabbitmq/0
16 May 2024 08:55:08Z workload blocked (workload) Error in charm (see logs): Snap: 'openstack-hypervisor'; command ['snap', 'set', 'openstack-hypervisor', '-t', 'rabbitmq.url=null'] failed with output = ''
16 May 2024 08:55:09Z juju-unit executing running nova-service-relation-changed hook for nova/0
16 May 2024 08:56:40Z juju-unit executing running ovsdb-cms-relation-changed hook for ovn-relay/0
16 May 2024 08:58:08Z juju-unit executing running ceph-access-relation-changed hook
16 May 2024 08:59:49Z juju-unit executing running certificates-relation-changed hook
16 May 2024 09:01:19Z juju-unit executing running ceph-access-relation-joined hook for cinder-ceph/0
16 May 2024 09:01:40Z juju-unit executing running ceph-access-relation-changed hook for cinder-ceph/0
16 May 2024 09:03:21Z juju-unit executing running identity-credentials-relation-joined hook for keystone/0
16 May 2024 09:03:42Z juju-unit executing running certificates-relation-joined hook for certificate-authority/0
16 May 2024 09:04:02Z juju-unit executing running certificates-relation-changed hook for certificate-authority/0
16 May 2024 09:05:32Z juju-unit executing running identity-credentials-relation-changed hook for keystone/0
16 May 2024 09:07:21Z juju-unit idle
16 May 2024 09:10:51Z workload active

Revision history for this message
Nobuto Murata (nobuto) wrote :
Revision history for this message
Nobuto Murata (nobuto) wrote :

Timeout is not configurable.
https://github.com/canonical/snap-openstack/blob/74ca5bd0a698c901c52121dec0f0f277a2c62239/sunbeam-python/sunbeam/commands/hypervisor.py#L44-L48

Revision history for this message
Nobuto Murata (nobuto) wrote (last edit ):

So in my success case:

[node-1] sunbeam cluster bootstrap --role control --role compute --role storage

-> real 21m7.678s

[node-2] sunbeam cluster join --role control --role compute --role storage --token TOKEN

-> real 23m19.413s

[node-3] sunbeam cluster join --role control --role compute --role storage --token TOKEN

-> real 22m15.997s

sunbeam cluster resize

-> real 31m39.274s

Total: 98 minutes and 59 seconds

Nobuto Murata (nobuto)
affects: sunbeam-charms → snap-openstack
Revision history for this message
Nobuto Murata (nobuto) wrote :

https://github.com/canonical/snap-openstack/pull/234

James Page (james-page)
Changed in snap-openstack:
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Nobuto Murata (nobuto)
Nobuto Murata (nobuto)
Changed in snap-openstack:
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.