| 2015-05-14 22:47:18 |
Curtis Hovey |
bug |
|
|
added bug |
| 2015-05-14 22:47:32 |
Curtis Hovey |
bug task added |
|
juju-quickstart |
|
| 2015-05-14 22:47:43 |
Curtis Hovey |
bug task added |
|
juju-deployer |
|
| 2015-05-14 22:48:59 |
Curtis Hovey |
bug task added |
|
cloud-installer |
|
| 2015-05-14 22:49:31 |
Curtis Hovey |
nominated for series |
|
juju-core/1.24 |
|
| 2015-05-14 22:49:31 |
Curtis Hovey |
bug task added |
|
juju-core/1.24 |
|
| 2015-05-14 22:49:41 |
Curtis Hovey |
juju-core/1.24: milestone |
|
1.24.0 |
|
| 2015-05-14 22:49:48 |
Curtis Hovey |
juju-core/1.24: status |
New |
Triaged |
|
| 2015-05-14 22:49:52 |
Curtis Hovey |
juju-core/1.24: importance |
Undecided |
High |
|
| 2015-05-14 22:54:03 |
David Britton |
bug task deleted |
cloud-installer |
|
|
| 2015-05-15 01:42:23 |
Curtis Hovey |
description |
This is a meta bug that describes a problem with many symptoms and many advisable workarounds.
Enterprises commonly automate the deployment of services. They can use juju-quickstart, juju-deployer, landscape autopilot, or their own bespoke script to bootstrap an environment and deploy services. It works repeatedly, reliably for many weeks or months, until a new micro version of juju is placed in the streams. The enterprise sees failures in many ways, commonly the deployment fails because the script lost connection to its watcher, or the juju failed to upgrade at the same moment that charms are installing and configuring services.
When a juju state-server is bootstrapped one its first actions, is to query streams, and start an upgrade to the highest micro version for its major.minor version. eg, the juju client installed 1.22.1, and the current version in streams is 1.22.3, start upgrading. This upgrade will complete in less than a minute. A savvy script bootstrapping an env would wait for an upgrade to complete because upgrading a single state-server is faster and more reliable that upgrading services too.
Enterprises do not like default behaviour however, and their tools were not written to account for this "surprising" behaviour. There are several strategies employed to ensure the state-server is exactly the version that was tested previously:
A. Juju CI and a few others set "agent-version: 1.22.1" to ensure the state-server matches the version under test. But many parties do not like this method because environments.yaml must change each time they upgrade to a new juju client (and server).
B. Canonical IS and many customers use --upload-tools to force the state-server to be a known version. This however make explicit upgrades VERY unpredictable because the juju-client selects agents based on the localhost's arch, series, and $PATH (which might include development jujus). There are several bugs about failed upgrades, and --upload-tools was a factor.
C. The company never uses current juju. They choose to use a version that is not getting updates, like 1.22.x which is not 1.23.x, except that we have delivered updates to their surprise.
Juju chooses to implicitly upgrade because it is a way to deliver compatibility fixes. Azure, AWS, and HP have changed their clouds, and we delivered a new micro version of juju a few days later to ensure juju "Just worked".
Several changes are needed to ensure enterprises have a reliable and repeatable experience:
1. All API clients must reconnect watchers when they are disconnected. They will be disconnected because of network issues as well as explicit disconnects durin upgrade-juju. The clients need to resume their work.
2.A If juju must upgrade, it needs to prevent clients from starting work until the upgrade is complete, this might mean bootstrap doesn't let go until the state-server is upgraded.
2.B Or juju stops implicit upgrades. No enterprises uses this feature; they work hard to disable it. Juju could instead inform the party that an upgrade is available (as is done for charms). |
This is a meta bug that describes a problem with many symptoms and many unadvisable workarounds.
Enterprises commonly automate the deployment of services. They can use juju-quickstart, juju-deployer, landscape autopilot, or their own bespoke script to bootstrap an environment and deploy services. It works repeatedly, reliably for many weeks or months, until a new micro version of juju is placed in the streams. The enterprise sees failures in many ways, commonly the deployment fails because the script lost connection to its watcher, or the juju failed to upgrade at the same moment that charms are installing and configuring services.
When a juju state-server is bootstrapped one its first actions, is to query streams, and start an upgrade to the highest micro version for its major.minor version. eg, the juju client installed 1.22.1, and the current version in streams is 1.22.3, start upgrading. This upgrade will complete in less than a minute. A savvy script bootstrapping an env would wait for an upgrade to complete because upgrading a single state-server is faster and more reliable that upgrading services too.
Enterprises do not like default behaviour however, and their tools were not written to account for this "surprising" behaviour. There are several strategies employed to ensure the state-server is exactly the version that was tested previously:
A. Juju CI and a few others set "agent-version: 1.22.1" to ensure the state-server matches the version under test. But many parties do not like this method because environments.yaml must change each time they upgrade to a new juju client (and server).
B. Canonical IS and many customers use --upload-tools to force the state-server to be a known version. This however make explicit upgrades VERY unpredictable because the juju-client selects agents based on the localhost's arch, series, and $PATH (which might include development jujus). There are several bugs about failed upgrades, and --upload-tools was a factor.
C. The company never uses current juju. They choose to use a version that is not getting updates, like 1.22.x which is not 1.23.x, except that we have delivered updates to their surprise.
Juju chooses to implicitly upgrade because it is a way to deliver compatibility fixes. Azure, AWS, and HP have changed their clouds, and we delivered a new micro version of juju a few days later to ensure juju "Just worked".
Several changes are needed to ensure enterprises have a reliable and repeatable experience:
1. All API clients must reconnect watchers when they are disconnected. They will be disconnected because of network issues as well as explicit disconnects durin upgrade-juju. The clients need to resume their work.
2.A If juju must upgrade, it needs to prevent clients from starting work until the upgrade is complete, this might mean bootstrap doesn't let go until the state-server is upgraded.
2.B Or juju stops implicit upgrades. No enterprises uses this feature; they work hard to disable it. Juju could instead inform the party that an upgrade is available (as is done for charms). |
|
| 2015-05-15 01:46:18 |
Curtis Hovey |
description |
This is a meta bug that describes a problem with many symptoms and many unadvisable workarounds.
Enterprises commonly automate the deployment of services. They can use juju-quickstart, juju-deployer, landscape autopilot, or their own bespoke script to bootstrap an environment and deploy services. It works repeatedly, reliably for many weeks or months, until a new micro version of juju is placed in the streams. The enterprise sees failures in many ways, commonly the deployment fails because the script lost connection to its watcher, or the juju failed to upgrade at the same moment that charms are installing and configuring services.
When a juju state-server is bootstrapped one its first actions, is to query streams, and start an upgrade to the highest micro version for its major.minor version. eg, the juju client installed 1.22.1, and the current version in streams is 1.22.3, start upgrading. This upgrade will complete in less than a minute. A savvy script bootstrapping an env would wait for an upgrade to complete because upgrading a single state-server is faster and more reliable that upgrading services too.
Enterprises do not like default behaviour however, and their tools were not written to account for this "surprising" behaviour. There are several strategies employed to ensure the state-server is exactly the version that was tested previously:
A. Juju CI and a few others set "agent-version: 1.22.1" to ensure the state-server matches the version under test. But many parties do not like this method because environments.yaml must change each time they upgrade to a new juju client (and server).
B. Canonical IS and many customers use --upload-tools to force the state-server to be a known version. This however make explicit upgrades VERY unpredictable because the juju-client selects agents based on the localhost's arch, series, and $PATH (which might include development jujus). There are several bugs about failed upgrades, and --upload-tools was a factor.
C. The company never uses current juju. They choose to use a version that is not getting updates, like 1.22.x which is not 1.23.x, except that we have delivered updates to their surprise.
Juju chooses to implicitly upgrade because it is a way to deliver compatibility fixes. Azure, AWS, and HP have changed their clouds, and we delivered a new micro version of juju a few days later to ensure juju "Just worked".
Several changes are needed to ensure enterprises have a reliable and repeatable experience:
1. All API clients must reconnect watchers when they are disconnected. They will be disconnected because of network issues as well as explicit disconnects durin upgrade-juju. The clients need to resume their work.
2.A If juju must upgrade, it needs to prevent clients from starting work until the upgrade is complete, this might mean bootstrap doesn't let go until the state-server is upgraded.
2.B Or juju stops implicit upgrades. No enterprises uses this feature; they work hard to disable it. Juju could instead inform the party that an upgrade is available (as is done for charms). |
This is a meta bug that describes a problem with many symptoms and many unadvisable workarounds.
Enterprises commonly automate the deployment of services. They can use juju-quickstart, juju-deployer, landscape autopilot, or their own bespoke script to bootstrap an environment and deploy services. It works repeatedly, reliably for many weeks or months, until a new micro version of juju is placed in the streams. The enterprise sees failures in many ways, commonly the deployment fails because the script lost connection to its watcher, or the juju failed to upgrade at the same moment that charms are installing and configuring services.
When a juju state-server is bootstrapped one its first actions, is to query streams, and start an upgrade to the highest micro version for its major.minor version. eg, the juju client installed 1.22.1, and the current version in streams is 1.22.3, start upgrading. This upgrade will complete in less than a minute. A savvy script bootstrapping an env would wait for an upgrade to complete because upgrading a single state-server is faster and more reliable that upgrading services too.
Enterprises do not like default behaviour however, and their tools were not written to account for this "surprising" behaviour. There are several strategies employed to ensure the state-server is exactly the version that was tested previously:
A. Juju CI and a few others set "agent-version: 1.22.1" to ensure the state-server matches the version under test. But many parties do not like this method because environments.yaml must change each time they upgrade to a new juju client (and server).
B. Canonical IS and many customers use --upload-tools to force the state-server to be a known version. This however makes explicit upgrades VERY unpredictable because the juju-client selects agents based on the localhost's arch, series, and $PATH (which might include development jujus). There are several bugs about failed upgrades, and --upload-tools was a factor.
C. The company never uses current juju. They choose to use a version that is not getting updates, like 1.22.x which is not 1.23.x, except that we have delivered updates to their surprise. They are also not getting the benefits of current juju.
Juju chooses to implicitly upgrade because it is a way to deliver compatibility fixes. Azure, AWS, and HP have changed their clouds, and we delivered a new micro version of juju a few days later to ensure juju "Just worked".
Several changes are needed to ensure enterprises have a reliable and repeatable experience:
1. All API clients must reconnect watchers when they are disconnected. They will be disconnected because of network issues as well as explicit disconnects durin upgrade-juju. The clients need to resume their work.
2.A If juju must upgrade, it needs to prevent clients from starting work until the upgrade is complete, this might mean bootstrap doesn't let go until the state-server is upgraded.
2.B Or juju stops implicit upgrades. No enterprises uses this feature; they work hard to disable it. Juju could instead inform the party that an upgrade is available (as is done for charms). |
|
| 2015-05-15 11:21:50 |
Ian Booth |
juju-core/1.24: assignee |
|
Ian Booth (wallyworld) |
|
| 2015-05-15 11:21:56 |
Ian Booth |
juju-core/1.24: milestone |
1.24.0 |
1.24-beta4 |
|
| 2015-05-15 11:32:54 |
Adam Collard |
juju-deployer: status |
New |
Confirmed |
|
| 2015-05-15 11:33:17 |
Adam Collard |
bug |
|
|
added subscriber Landscape |
| 2015-05-15 14:07:25 |
David Britton |
bug task deleted |
juju-deployer |
|
|
| 2015-05-15 14:07:45 |
David Britton |
bug task added |
|
python-jujuclient |
|
| 2015-05-15 14:07:53 |
David Britton |
python-jujuclient: status |
New |
Confirmed |
|
| 2015-05-19 04:19:03 |
Ian Booth |
juju-core/1.24: assignee |
Ian Booth (wallyworld) |
|
|
| 2015-05-19 04:19:14 |
Ian Booth |
juju-core/1.24: milestone |
1.24-beta4 |
1.24.0 |
|
| 2015-05-21 13:05:27 |
Lei Wang |
bug |
|
|
added subscriber Ray Wang |
| 2015-05-21 13:12:30 |
Nobuto Murata |
bug |
|
|
added subscriber Nobuto Murata |
| 2015-05-25 07:56:57 |
Ian Booth |
juju-core/1.24: assignee |
|
Ian Booth (wallyworld) |
|
| 2015-05-27 04:35:14 |
Ian Booth |
juju-core/1.24: milestone |
1.24.0 |
1.24-beta6 |
|
| 2015-05-27 04:35:20 |
Ian Booth |
juju-core/1.24: status |
Triaged |
In Progress |
|
| 2015-05-29 03:56:59 |
Ian Booth |
juju-core: assignee |
|
Ian Booth (wallyworld) |
|
| 2015-05-29 03:57:06 |
Ian Booth |
juju-core: status |
Triaged |
In Progress |
|
| 2015-05-29 04:02:00 |
Ian Booth |
bug task deleted |
juju-core/1.24 |
|
|
| 2015-05-31 14:48:00 |
Ian Booth |
branch linked |
|
lp:~wallyworld/python-jujuclient/retry-on-upgrade |
|
| 2015-05-31 14:48:29 |
Ian Booth |
branch unlinked |
lp:~wallyworld/python-jujuclient/retry-on-upgrade |
|
|
| 2015-06-16 11:52:20 |
Ian Booth |
juju-core: status |
In Progress |
Fix Committed |
|
| 2015-08-27 14:22:16 |
Curtis Hovey |
juju-core: status |
Fix Committed |
Fix Released |
|
| 2015-10-02 03:47:43 |
David Britton |
tags |
bootstrap ci cloud-installer deployer landscape oil quickstart |
bootstrap ci cloud-installer deployer kanban-cross-team landscape oil quickstart |
|
| 2015-10-02 03:49:40 |
David Britton |
tags |
bootstrap ci cloud-installer deployer kanban-cross-team landscape oil quickstart |
bootstrap ci cloud-installer deployer landscape oil quickstart |
|
| 2017-05-15 14:00:38 |
Curtis Hovey |
removed subscriber Curtis Hovey |
|
|
|
| 2018-07-03 04:38:07 |
Lei Wang |
removed subscriber Ray Wang |
|
|
|
