Juju complains about downgrading a charm when no downgrade requested

[Vern Hart]

When re-deploying the same bundle just a moment later, juju is complaining:

  ERROR cannot deploy bundle: downgrades are not currently supported

This has occurred in complex deployment scenarios for a while but I've been able to narrow it down to two charms:

  juju-lint
  landscape-client

To recreate, here's a simple bundle:

  series: focal
  applications:
    landscape-client:
      bindings:
        ? ''
        : oam-space
      channel: latest/stable
      charm: landscape-client

Deploying works without issue:

  ubuntu@fce:~$ juju deploy ./test.yaml
  Located charm "landscape-client" in charm-hub, channel latest/stable
  Executing changes:
  - upload charm landscape-client from charm-hub for series focal from channel latest/stable with architecture=amd64
  - deploy application landscape-client from charm-hub on focal with latest/stable
  Deploy of bundle completed.
  ubuntu@fce:~$ juju status
  Model Controller Cloud/Region Version SLA Timestamp
  downgrade foundations-maas maas_cloud 2.9.37 unsupported 23:59:30Z

  App Version Status Scale Charm Channel Rev Exposed Message
  landscape-client unknown 0 landscape-client latest/stable 49 no

Immediately re-running the deploy command fails even though the latest/stable channel still has the same revision:

  ubuntu@fce:~$ juju deploy ./test.yaml
  ERROR cannot deploy bundle: downgrades are not currently supported
  ubuntu@fce:~$ juju info landscape-client
  name: landscape-client
  publisher: Landscape Charmers
  summary: The Landscape administration system client
  description: |
    Landscape is a web-based tool for managing Ubuntu systems. This package
    is necessary if you want your machine to be managed in a Landscape
    account. This package provides the Landscape client and requires a
    Landscape account.
  store-url: https://charmhub.io/landscape-client
  charm-id: WsixdI1C4ocYA0eZyXALwrQa8pi5TvDD
  supports: bionic, focal, jammy
  tags: monitoring
  subordinate: true
  relations:
    provides: {}
    requires:
      ceph-client: ceph-client
      container: juju-info
      registration: landscape-registration
  channels: |
    latest/stable: 49 2022-09-07 (49) 121kB
    latest/candidate: ↑
    latest/beta: 49 2022-09-07 (49) 121kB
    latest/edge: 55 2022-11-29 (55) 6MB

I can work-around the issue by hard-coding the current charm revision:

  ubuntu@fce:~$ cat overlay.yaml
  applications:
    landscape-client:
      revision: 49
  ubuntu@fce:~$ juju deploy ./test.yaml --overlay overlay.yaml
  No changes to apply.

It's not desirable to hard-code the charm revision and we'd prefer if the latest/stable channel worked as expected.

I can recreate a similar scenario with the juju-lint charm.

I'm not sure if this is a bug with charmhub or juju but, at the very least, juju should be reporting which charm it thinks a downgrade is being requested for. When you've got a bundle with 50 applications, figuring out which one (or two) is causing the issue is very time consuming.

[Andre Ruiz (andre-ruiz)]

Another weird thing is that if you deploy that juju-lint charm from the command line and from a bundle you get different revisions. One is revision 10 (command line) and other is revision 9 (bundle), which is even weirder because release 9 does not exist anywhere (juju info does not show it and charmhub web page also does not show it).

This may or may not be related to this problem, but it was found while testing the juju bundle for this bug. (If it is a separate thing, a new bug can be filed).

[Loïc Gomez (kotodama)]

We ran into this one today too.
But then, since our bundle includes multiple apps, we don't know which one is causing the issue here.

For starters, I think the error message should be explicit about which app juju thinks is being downgraded.
Thank you

[Harry Pidcock (hpidcock)]

I've improved the error message here https://github.com/juju/juju/pull/15852 hopefully that helps tracking down what is going on, if its just a bundle issue, charmhub issue or a juju issue.

[Loïc Gomez (kotodama)]

In our case, we were deploying a bundle which partially reports currently deployed apps in our model.
Including the other bundles alongside the partial one fixed it for us.

After commenting out all apps but one, one by one, it would seem this is also landscape-client for our case.

I'm looking up into bundles to check if we add more details about this app.

[Loïc Gomez (kotodama)]

Thanks Harry, that was quick!
Sadly as this is production, I cannot update the controllers immediately, but this will be very helpful in future occurrences, thanks again!

[Loïc Gomez (kotodama)]

Okay, our partial bundle declares this app alongside other:
    landscape-client:
        charm: landscape-client

This fails with the error message:
  ERROR cannot deploy bundle: downgrades are not currently supported

Changing that to:
    landscape-client:
        series: focal
        charm: landscape-client

Fixes the bug for us.
01:25:14 INFO cmd bundlehandler.go:624 Deploy of bundle completed.

As a sidenote, there is already a top-level:
series: focal

Hope it helps!

[Harry Pidcock (hpidcock)]

Thanks, kotodama, I'll look into this further soon.

[Haw Loeung (hloeung)]

Seeing this myself, elsewhere:

| ERROR cannot deploy bundle: application "fw-log-aggregator": downgrades are not currently supported: deployed revision 4 is newer than requested revision 3

The bundle doesn't pin revision to anything (not even revision 3). Revision 4 is for Focal and revision 3 Jammy, it wants Jammy:

| $ juju info ...-fw-log-aggregator --series focal | grep 'latest/stable'
| latest/stable: 2cbf407 2023-07-31 (4) 7MB

| $ juju info ...-fw-log-aggregator --series jammy | grep 'latest/stable'
| latest/stable: 2cbf407 2023-07-31 (3) 7MB

This is with Juju 2.9.42 controller and 2.9.44 client.

[Ian Booth (wallyworld)]

The next 2.9.46 candidate release will not include a fix for this bug and we don't plan on any more 2.9 releases. As such it is being removed from its 2.9 milestone.

If the bug is still important to you, let us know and we can consider it for inclusion on a 3.x milestone.