Update Juju to make a "safe mode" for the provisioner

[Mark Ramm]

It should be possible to update the juju database to make the provisioner stop destroying machines that it does not know about.

This will allow us to bring a state server machine up, do upgrades, and otherwise take "unsafe" actions with a little bit more confidence that juju won't do anything destructive.

[Kapil Thangavelu (hazmat)]

any reason this isn't default enabled.. ie its not a specially enabled
mode, its just the sane default policy? afaics the branch defaulted to this
off which perpetuates the problem.

On Thu, Dec 5, 2013 at 10:12 AM, Curtis Hovey <email address hidden> wrote:

> ** Changed in: juju-core/trunk
> Importance: Critical => High
>
> --
> You received this bug notification because you are subscribed to trunk.
> https://bugs.launchpad.net/bugs/1254729
>
> Title:
> Update Juju to make a "safe mode" for the provisioner
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1254729/+subscriptions
>

[Ian Booth (wallyworld)]

This isn't an authoritative answer, but I can see a few reasons. First, we didn't want to change the out-of-the-box behaviour of Juju that people have come to expect and rely on in this regard. Which leads to the second point, that I believe the reason for the provisioner cleaning up unknown instances is so that people don't accidentally waste money running instances which for whatever reason should not be there as part of the current Juju environment. This change was for a point 1.16 release which in general should only be a bug fix release. The fact that a new feature was introduced was a necessary deviation from preferred policy to satisfy a customer need; it would have been unacceptable to compound this further by changing default system behaviour.

[Kapil Thangavelu (hazmat)]

sure its a bug fix release, but the fix was never released imo, a
consulting ware option was added so that people in the know could avoid the
bug.

the reason for the behavior was a slavish copying of pyjuju semantics which
were themselves implemented during the first pyjuju sprint with no real
reasoning behind it outside of I didn't trust juju to even bootstrap
correctly at the time.

The reality is that this behavior is almost never a good idea, as has been
demonstrated, and that its basically always a bug for production
environments. Having an option to disable it is nice for a particular site
fix, but means this is a bug that is waiting to bite all users by default,
at which point we point out that they should have enabled the provisioner
**sane** mode, that its if they bother to ask instead of moving on.

please at least in 1.17/1.18 change the default to 'sane' mode on.

On Thu, Dec 5, 2013 at 3:33 PM, Ian Booth <email address hidden> wrote:

> This isn't an authoritative answer, but I can see a few reasons. First,
> we didn't want to change the out-of-the-box behaviour of Juju that
> people have come to expect and rely on in this regard. Which leads to
> the second point, that I believe the reason for the provisioner cleaning
> up unknown instances is so that people don't accidentally waste money
> running instances which for whatever reason should not be there as part
> of the current Juju environment. This change was for a point 1.16
> release which in general should only be a bug fix release. The fact that
> a new feature was introduced was a necessary deviation from preferred
> policy to satisfy a customer need; it would have been unacceptable to
> compound this further by changing default system behaviour.
>
> --
> You received this bug notification because you are subscribed to trunk.
> https://bugs.launchpad.net/bugs/1254729
>
> Title:
> Update Juju to make a "safe mode" for the provisioner
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1254729/+subscriptions
>

[Dave Cheney (dave-cheney)]

On Fri, Dec 6, 2013 at 11:05 AM, Kapil Thangavelu
<email address hidden> wrote:
> sure its a bug fix release, but the fix was never released imo, a
> consulting ware option was added so that people in the know could avoid the
> bug.
>
> the reason for the behavior was a slavish copying of pyjuju semantics which
> were themselves implemented during the first pyjuju sprint with no real
> reasoning behind it outside of I didn't trust juju to even bootstrap
> correctly at the time.

weeeeell, I don't know if that is correct. Certainly I cannot speak
for the wealth of Juju development that happened before I joined last
year.

My impression of Juju, circa April last year was a tool primarily
focussed at public clouds, and so one of the primary goals was not to
cost the user money by leaking machines. With that in mind, the
provisioner always shut down machines that it didn't have a record
for. I believe this is the correct, and least surprising behaviour
when juju is managing environments on public clouds.

Even if cost were not the primary issue, most clouds have limits on
the number of machines you can spawn (hello HP Cloud) so leaking
machines means future deployments may fail mysteriously because you've
hit a limit which is poorly communicated to the Juju user.

All that being said, I feel that in the last 12-18 months a second
audience for Juju has emerged, that is Juju + MAAS as a tool to deploy
Openstack clouds. These customers have different requirements. For
example the per hour cost of 'leaking' a machine is irrelevant,
because the hardware is bought and paid for.

So, I think it's unfair to say that the current behaviour of the
provisioner is wrong, or that safe mode is a consultant ware switch.
Juju has two major markets, and they have different requirements,
hence a switch.

>
> The reality is that this behavior is almost never a good idea, as has been
> demonstrated, and that its basically always a bug for production
> environments. Having an option to disable it is nice for a particular site
> fix, but means this is a bug that is waiting to bite all users by default,
> at which point we point out that they should have enabled the provisioner
> **sane** mode, that its if they bother to ask instead of moving on.
>
> please at least in 1.17/1.18 change the default to 'sane' mode on.
>
>
> On Thu, Dec 5, 2013 at 3:33 PM, Ian Booth <email address hidden>
> wrote:
>
>> This isn't an authoritative answer, but I can see a few reasons. First,
>> we didn't want to change the out-of-the-box behaviour of Juju that
>> people have come to expect and rely on in this regard. Which leads to
>> the second point, that I believe the reason for the provisioner cleaning
>> up unknown instances is so that people don't accidentally waste money
>> running instances which for whatever reason should not be there as part
>> of the current Juju environment. This change was for a point 1.16
>> release which in general should only be a bug fix release. The fact that
>> a new feature was introduced was a necessary deviation from preferred
>> policy to satisfy a customer need; it would have been unacceptable to
>> compound this further by changing default ...

[Rolf Leggewie (r0lf)]

saucy has seen the end of its life and is no longer receiving any updates. Marking the saucy task for this ticket as "Won't Fix".