Impossible to cancel bootstrap

Bug #1840367 reported by Andreas Hasenack
50
This bug affects 7 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Ben Hoyt
2.9
Fix Released
High
Ben Hoyt
3.0
Fix Released
High
Ben Hoyt

Bug Description

Bootstrapping was taking too long (10min+), so I tried to cancel it. It doens't work.

Host is bionic, juju snap is:
juju 2.6.6 8594 stable canonical✓ classic

lxd works just fine on this host.

$ juju bootstrap lxd --keep-broken
Creating Juju controller "localhost-localhost" on lxd/localhost
Looking for packaged Juju agent version 2.6.6 for amd64
To configure your system to better support LXD containers, please see: https://github.com/lxc/lxd/blob/master/doc/production-setup.md
Launching controller instance(s) on localhost/localhost...
 - juju-a83592-0 (arch=amd64)
Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.0.100.216:22
Connected to 10.0.100.216
Running machine configuration script...
^CInterrupt signalled: waiting for bootstrap to exit
Bootstrap agent now started
Contacting Juju controller at 10.0.100.216 to verify accessibility...
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit
^CInterrupt signalled: waiting for bootstrap to exit

Revision history for this message
Tim McNamara (tim-clicks) wrote :

This is intentional.

Aborting immediately will leave things running on the host. To get out of this straight away, use Ctrl+Z to kill the process, then run `juju kill-controller`.

Changed in juju:
status: New → Invalid
Revision history for this message
Tim Penhey (thumper) wrote :

While this may be intentional, it isn't good user experience.

We should be able to determine that there is a machine deployed by the cloud provider, and kill that.

Changed in juju:
status: Invalid → Triaged
importance: Undecided → Medium
Revision history for this message
Heather Lanigan (hmlanigan) wrote :

We now have at least 4 duplicates of this bug. Any issues making a high priority?

Changed in juju:
importance: Medium → High
Revision history for this message
Ben Hoyt (benhoyt) wrote :

This is addressed in https://github.com/juju/juju/pull/12475: now when you press Ctrl-C, Juju responds right away with "Ctrl-C pressed, stopping bootstrap and cleaning up resources", and (after the current operation) terminates the bootstrap instance and exits.

It's not perfect, as the code only checks for cancellation after various points (rather than wiring context.Context through to the "bottom" layers, as that would require a lot more churn). And of course you still have to wait for teardown. For LXD this is quite quick -- just a few seconds -- but for cloud providers like AWS teardown can take a minute or two. If you press Ctrl-C again during teardown, the Juju client says "Ctrl-C pressed, cleaning up failed bootstrap".

Changed in juju:
assignee: nobody → Ben Hoyt (benhoyt)
milestone: none → 2.8.8
status: Triaged → Fix Committed
Revision history for this message
Ben Hoyt (benhoyt) wrote :

PR to merge fix into 2.9 and develop:

* 2.9: https://github.com/juju/juju/pull/12480
* develop: https://github.com/juju/juju/pull/12481

Changed in juju:
status: Fix Committed → Fix Released
Ian Booth (wallyworld)
Changed in juju:
status: Fix Released → Fix Committed
status: Fix Committed → Fix Released
Revision history for this message
Jordan Barrett (barrettj12) wrote :

Marking this as "fix released" since Ben's fix did go into 3.0, however there are still issues interrupting bootstrap, as detailed in my bug report here:
https://bugs.launchpad.net/juju/+bug/1968122

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.