cannot deploy with manual provider

[Andrew McLeod]

juju 2.6.2 on an amd64 instance as the controller, with s390x manual machines

I have provisioned 5 s390x machines, which are successfully added to the model:

Model Controller Cloud/Region Version SLA Timestamp
s390x-mosci-s390x s390x-mosci-s390x manual 2.6.2 unsupported 15:52:46Z

Machine State DNS Inst id Series AZ Message
5 started 10.13.3.10 manual:10.13.3.10 bionic Manually provisioned machine
6 started 10.13.3.11 manual:10.13.3.11 bionic Manually provisioned machine
7 started 10.13.3.12 manual:10.13.3.12 bionic Manually provisioned machine
8 started 10.13.3.13 manual:10.13.3.13 bionic Manually provisioned machine
9 started 10.13.3.14 manual:10.13.3.14 bionic Manually provisioned machine

Note re unit numbering, the exact same issues were present with 0-4.

I am unable to deploy a bundle with --map-machines=existing, e.g.:

$ juju deploy bundle.yaml --map-machines=existing
...
Resolving charm: cs:~openstack-charmers-next/swift-storage
Executing changes:
- upload charm cs:~openstack-charmers-next/cinder-374 for series bionic
- deploy application cinder on bionic using cs:~openstack-charmers-next/cinder-374
ERROR cannot deploy bundle: cannot deploy application "cinder": cannot add application "cinder": use "juju add-machine ssh:[user@]<host>" to provision machines

debug log: https://pastebin.canonical.com/p/xmydKcJJcs/
bundle url: https://raw.githubusercontent.com/ubuntu-openstack/zopenstack/master/bundles/lpar/bionic-stein-stable.yaml

I am able to manually deploy a principal charm using --to:

$ juju deploy cs:~openstack-charmers-next/cinder-374 --to lxd:5
Located charm "cs:~openstack-charmers-next/cinder-374".
Deploying charm "cs:~openstack-charmers-next/cinder-374".
$ juju status
Model Controller Cloud/Region Version SLA Timestamp
s390x-mosci-s390x s390x-mosci-s390x manual 2.6.2 unsupported 15:57:30Z

App Version Status Scale Charm Store Rev OS Notes
cinder waiting 0/1 cinder jujucharms 374 ubuntu

Unit Workload Agent Machine Public address Ports Message
cinder/2 waiting allocating 5/lxd/0 waiting for machine

Machine State DNS Inst id Series AZ Message
5 started 10.13.3.10 manual:10.13.3.10 bionic Manually provisioned machine
5/lxd/0 pending pending bionic

So I can use this method to deploy charms manually, until I get to a subordinate charm:

$ juju deploy neutron-openvswitch
Located charm "cs:neutron-openvswitch-259".
Deploying charm "cs:neutron-openvswitch-259".
ERROR cannot add application "neutron-openvswitch": use "juju add-machine ssh:[user@]<host>" to provision machines
$ juju deploy neutron-openvswitch --num-units=0
ERROR --num-units must be a positive integer
$ juju deploy neutron-openvswitch --to 0
Located charm "cs:neutron-openvswitch-259".
Deploying charm "cs:neutron-openvswitch-259".
ERROR cannot use --num-units or --to with subordinate application

debug log:

https://pastebin.canonical.com/p/vZ8PScRWVr/

[Achilleas Anagnostopoulos (achilleasa)]

https://github.com/juju/juju/pull/10209 includes a tentative fix for the 2.6 branch.

[Andrew McLeod (admcleod)]

I've tested the fix and it resolves the problem

[Richard Harding (rharding)]

There was an issue around manual and 2.5 as well. Can you please confirm if 2.5 has the same issue and if so backport this and include it against the 2.5 tree?

Thanks

[Achilleas Anagnostopoulos (achilleasa)]

The actual code in the manual provider hasn't really changed since 2.4
(that is as far as I searched).

Unless something has radically changed in the state code between 2.5 and
2.6 backporting the fix will most likely fix it for 2.5 as well.

I will backport and merge first thing in the morning.

/A

On Thu, 23 May 2019, 8:00 pm Richard Harding <<email address hidden>
wrote:

> There was an issue around manual and 2.5 as well. Can you please confirm
> if 2.5 has the same issue and if so backport this and include it against
> the 2.5 tree?
>
> Thanks
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1829559
>
> Title:
> cannot deploy with manual provider - 2.6.2, s390x
>
> Status in juju:
> Fix Committed
> Status in Ubuntu on IBM z Systems:
> Fix Committed
>
> Bug description:
> juju 2.6.2 on an amd64 instance as the controller, with s390x manual
> machines
>
> I have provisioned 5 s390x machines, which are successfully added to
> the model:
>
> Model Controller Cloud/Region Version SLA
> Timestamp
> s390x-mosci-s390x s390x-mosci-s390x manual 2.6.2
> unsupported 15:52:46Z
>
> Machine State DNS Inst id Series AZ Message
> 5 started 10.13.3.10 manual:10.13.3.10 bionic Manually
> provisioned machine
> 6 started 10.13.3.11 manual:10.13.3.11 bionic Manually
> provisioned machine
> 7 started 10.13.3.12 manual:10.13.3.12 bionic Manually
> provisioned machine
> 8 started 10.13.3.13 manual:10.13.3.13 bionic Manually
> provisioned machine
> 9 started 10.13.3.14 manual:10.13.3.14 bionic Manually
> provisioned machine
>
> Note re unit numbering, the exact same issues were present with 0-4.
>
> I am unable to deploy a bundle with --map-machines=existing, e.g.:
>
> $ juju deploy bundle.yaml --map-machines=existing
> ...
> Resolving charm: cs:~openstack-charmers-next/swift-storage
> Executing changes:
> - upload charm cs:~openstack-charmers-next/cinder-374 for series bionic
> - deploy application cinder on bionic using
> cs:~openstack-charmers-next/cinder-374
> ERROR cannot deploy bundle: cannot deploy application "cinder": cannot
> add application "cinder": use "juju add-machine ssh:[user@]<host>" to
> provision machines
>
> debug log: https://pastebin.canonical.com/p/xmydKcJJcs/
> bundle url:
> https://raw.githubusercontent.com/ubuntu-openstack/zopenstack/master/bundles/lpar/bionic-stein-stable.yaml
>
>
> I am able to manually deploy a principal charm using --to:
>
> $ juju deploy cs:~openstack-charmers-next/cinder-374 --to lxd:5
> Located charm "cs:~openstack-charmers-next/cinder-374".
> Deploying charm "cs:~openstack-charmers-next/cinder-374".
> $ juju status
> Model Controller Cloud/Region Version SLA
> Timestamp
> s390x-mosci-s390x s390x-mosci-s390x manual 2.6.2
> unsupported 15:57:30Z
>
> App Version Status Scale Charm Store Rev OS Notes
> cinder waiting 0/1 cinder jujucharms 374 ubuntu
>
> Unit Workload Ag...

[Richard Harding (rharding)]

I just want to make sure we validate 2.5 does have the same exact issue
before the backport please.

On Thu, May 23, 2019 at 3:20 PM Achilleas Anagnostopoulos <
<email address hidden>> wrote:

> The actual code in the manual provider hasn't really changed since 2.4
> (that is as far as I searched).
>
> Unless something has radically changed in the state code between 2.5 and
> 2.6 backporting the fix will most likely fix it for 2.5 as well.
>
> I will backport and merge first thing in the morning.
>
> /A
>
> On Thu, 23 May 2019, 8:00 pm Richard Harding <<email address hidden>
> wrote:
>
> > There was an issue around manual and 2.5 as well. Can you please confirm
> > if 2.5 has the same issue and if so backport this and include it against
> > the 2.5 tree?
> >
> > Thanks
> >
> > --
> > You received this bug notification because you are a bug assignee.
> > https://bugs.launchpad.net/bugs/1829559
> >
> > Title:
> > cannot deploy with manual provider - 2.6.2, s390x
> >
> > Status in juju:
> > Fix Committed
> > Status in Ubuntu on IBM z Systems:
> > Fix Committed
> >
> > Bug description:
> > juju 2.6.2 on an amd64 instance as the controller, with s390x manual
> > machines
> >
> > I have provisioned 5 s390x machines, which are successfully added to
> > the model:
> >
> > Model Controller Cloud/Region Version SLA
> > Timestamp
> > s390x-mosci-s390x s390x-mosci-s390x manual 2.6.2
> > unsupported 15:52:46Z
> >
> > Machine State DNS Inst id Series AZ Message
> > 5 started 10.13.3.10 manual:10.13.3.10 bionic Manually
> > provisioned machine
> > 6 started 10.13.3.11 manual:10.13.3.11 bionic Manually
> > provisioned machine
> > 7 started 10.13.3.12 manual:10.13.3.12 bionic Manually
> > provisioned machine
> > 8 started 10.13.3.13 manual:10.13.3.13 bionic Manually
> > provisioned machine
> > 9 started 10.13.3.14 manual:10.13.3.14 bionic Manually
> > provisioned machine
> >
> > Note re unit numbering, the exact same issues were present with 0-4.
> >
> > I am unable to deploy a bundle with --map-machines=existing, e.g.:
> >
> > $ juju deploy bundle.yaml --map-machines=existing
> > ...
> > Resolving charm: cs:~openstack-charmers-next/swift-storage
> > Executing changes:
> > - upload charm cs:~openstack-charmers-next/cinder-374 for series bionic
> > - deploy application cinder on bionic using
> > cs:~openstack-charmers-next/cinder-374
> > ERROR cannot deploy bundle: cannot deploy application "cinder": cannot
> > add application "cinder": use "juju add-machine ssh:[user@]<host>" to
> > provision machines
> >
> > debug log: https://pastebin.canonical.com/p/xmydKcJJcs/
> > bundle url:
> >
> https://raw.githubusercontent.com/ubuntu-openstack/zopenstack/master/bundles/lpar/bionic-stein-stable.yaml
> >
> >
> > I am able to manually deploy a principal charm using --to:
> >
> > $ juju deploy cs:~openstack-charmers-next/cinder-374 --to lxd:5
> > Located charm "cs:~openstack-charmers-next/cinder-374".
> > Deploying charm "cs:~openstack-charmers-next/cinder-374".
> > $ j...

[Andrew McLeod (admcleod)]

I can confirm this is a problem with 2.5.7 as well

$ juju --version
2.5.7-bionic-amd64

$ juju deploy bundle.yaml --map-machines=existing --model=s390x-mosci-s390x:s390x-mosci-s390x
Resolving...
...
...
- deploy application ceph-mon on disco using cs:~openstack-charmers-next/ceph-mon-389
ERROR cannot deploy bundle: cannot deploy application "ceph-mon": cannot add application "ceph-mon": use "juju add-machine ssh:[user@]<host>" to provision machines

[Achilleas Anagnostopoulos (achilleasa)]

I have just replicated the bug on 2.5.8 and can confirm that applying the
patch
allows deployments to proceed.

I will go ahead and back-port the fix. Do we also need this backported to
2.4?

/A

On May 23, 2019 at 8:35:34 PM, Richard Harding (<email address hidden>)
wrote:

I just want to make sure we validate 2.5 does have the same exact issue
before the backport please.

On Thu, May 23, 2019 at 3:20 PM Achilleas Anagnostopoulos <
<email address hidden>> wrote:

> The actual code in the manual provider hasn't really changed since 2.4
> (that is as far as I searched).
>
> Unless something has radically changed in the state code between 2.5 and
> 2.6 backporting the fix will most likely fix it for 2.5 as well.
>
> I will backport and merge first thing in the morning.
>
> /A
>
> On Thu, 23 May 2019, 8:00 pm Richard Harding <<email address hidden>
> wrote:
>
> > There was an issue around manual and 2.5 as well. Can you please
confirm
> > if 2.5 has the same issue and if so backport this and include it
against
> > the 2.5 tree?
> >
> > Thanks
> >
> > --
> > You received this bug notification because you are a bug assignee.
> > https://bugs.launchpad.net/bugs/1829559
> >
> > Title:
> > cannot deploy with manual provider - 2.6.2, s390x
> >
> > Status in juju:
> > Fix Committed
> > Status in Ubuntu on IBM z Systems:
> > Fix Committed
> >
> > Bug description:
> > juju 2.6.2 on an amd64 instance as the controller, with s390x manual
> > machines
> >
> > I have provisioned 5 s390x machines, which are successfully added to
> > the model:
> >
> > Model Controller Cloud/Region Version SLA
> > Timestamp
> > s390x-mosci-s390x s390x-mosci-s390x manual 2.6.2
> > unsupported 15:52:46Z
> >
> > Machine State DNS Inst id Series AZ Message
> > 5 started 10.13.3.10 manual:10.13.3.10 bionic Manually
> > provisioned machine
> > 6 started 10.13.3.11 manual:10.13.3.11 bionic Manually
> > provisioned machine
> > 7 started 10.13.3.12 manual:10.13.3.12 bionic Manually
> > provisioned machine
> > 8 started 10.13.3.13 manual:10.13.3.13 bionic Manually
> > provisioned machine
> > 9 started 10.13.3.14 manual:10.13.3.14 bionic Manually
> > provisioned machine
> >
> > Note re unit numbering, the exact same issues were present with 0-4.
> >
> > I am unable to deploy a bundle with --map-machines=existing, e.g.:
> >
> > $ juju deploy bundle.yaml --map-machines=existing
> > ...
> > Resolving charm: cs:~openstack-charmers-next/swift-storage
> > Executing changes:
> > - upload charm cs:~openstack-charmers-next/cinder-374 for series bionic
> > - deploy application cinder on bionic using
> > cs:~openstack-charmers-next/cinder-374
> > ERROR cannot deploy bundle: cannot deploy application "cinder": cannot
> > add application "cinder": use "juju add-machine ssh:[user@]<host>" to
> > provision machines
> >
> > debug log: https://pastebin.canonical.com/p/xmydKcJJcs/
> > bundle url:
> >
>
https://raw.githubusercontent.com/ubuntu-openstack/zopenstack/master/bundles/lpar/bionic-stein-stable.yaml
> >
> >
> > I am able to manually deploy a principal charm using --to:
> >
> > $ juju deploy cs:~openstack-charmers-next/cinder-374 --to lxd:5
> > Located charm "cs:~openstack-charmers-nex...

[Achilleas Anagnostopoulos (achilleasa)]

https://github.com/juju/juju/pull/10229 backports the fix to the 2.5 branch

[Richard Harding (rharding)]

No need to go back to 2.4. They say 2.4 is working properly and is their
current fallback.

On Fri, May 24, 2019 at 5:51 AM Achilleas Anagnostopoulos <
<email address hidden>> wrote:

> I have just replicated the bug on 2.5.8 and can confirm that applying the
> patch
> allows deployments to proceed.
>
> I will go ahead and back-port the fix. Do we also need this backported to
> 2.4?
>
> /A
>
>
> On May 23, 2019 at 8:35:34 PM, Richard Harding (<email address hidden>
> )
> wrote:
>
> I just want to make sure we validate 2.5 does have the same exact issue
> before the backport please.
>
> On Thu, May 23, 2019 at 3:20 PM Achilleas Anagnostopoulos <
> <email address hidden>> wrote:
>
> > The actual code in the manual provider hasn't really changed since 2.4
> > (that is as far as I searched).
> >
> > Unless something has radically changed in the state code between 2.5 and
> > 2.6 backporting the fix will most likely fix it for 2.5 as well.
> >
> > I will backport and merge first thing in the morning.
> >
> > /A
> >
> > On Thu, 23 May 2019, 8:00 pm Richard Harding <<email address hidden>
> > wrote:
> >
> > > There was an issue around manual and 2.5 as well. Can you please
> confirm
> > > if 2.5 has the same issue and if so backport this and include it
> against
> > > the 2.5 tree?
> > >
> > > Thanks
> > >
> > > --
> > > You received this bug notification because you are a bug assignee.
> > > https://bugs.launchpad.net/bugs/1829559
> > >
> > > Title:
> > > cannot deploy with manual provider - 2.6.2, s390x
> > >
> > > Status in juju:
> > > Fix Committed
> > > Status in Ubuntu on IBM z Systems:
> > > Fix Committed
> > >
> > > Bug description:
> > > juju 2.6.2 on an amd64 instance as the controller, with s390x manual
> > > machines
> > >
> > > I have provisioned 5 s390x machines, which are successfully added to
> > > the model:
> > >
> > > Model Controller Cloud/Region Version SLA
> > > Timestamp
> > > s390x-mosci-s390x s390x-mosci-s390x manual 2.6.2
> > > unsupported 15:52:46Z
> > >
> > > Machine State DNS Inst id Series AZ Message
> > > 5 started 10.13.3.10 manual:10.13.3.10 bionic Manually
> > > provisioned machine
> > > 6 started 10.13.3.11 manual:10.13.3.11 bionic Manually
> > > provisioned machine
> > > 7 started 10.13.3.12 manual:10.13.3.12 bionic Manually
> > > provisioned machine
> > > 8 started 10.13.3.13 manual:10.13.3.13 bionic Manually
> > > provisioned machine
> > > 9 started 10.13.3.14 manual:10.13.3.14 bionic Manually
> > > provisioned machine
> > >
> > > Note re unit numbering, the exact same issues were present with 0-4.
> > >
> > > I am unable to deploy a bundle with --map-machines=existing, e.g.:
> > >
> > > $ juju deploy bundle.yaml --map-machines=existing
> > > ...
> > > Resolving charm: cs:~openstack-charmers-next/swift-storage
> > > Executing changes:
> > > - upload charm cs:~openstack-charmers-next/cinder-374 for series bionic
> > > - deploy application cinder on bionic using
> > > cs:~openstack-charmers-next/cinder-374
> > > ERROR cannot deploy bundle: cannot deploy application "cinder": cannot
> > > add application "cinder": use "juju add-machine ssh:[user@]<host>" to
> > > provision machines
> > >
> > > debug...

[Pedro Guimarães (pguimaraes)]

I've tested the commit and it works on 2.6.3.
Marking this bug as field-critical since it is a blocker for ongoing deployments until the fix is merged.

[Pedro Guimarães (pguimaraes)]

I see this issue on both 2.6.2 and 2.5.7 on amd64. Can we change the title to have both versions and remove s390x?