Upgrade local charm fail

[Joakim Nyman]

Upgrading a local charm fails with
`ERROR missing base name or channel not valid`

Same charm upgrade worked on 2 out of 3 models. But on the third one it fails.

Example:
`$ juju upgrade-charm acme --path ./acme_ubuntu-20.04-amd64.charm --debug`

```
11:42:34 INFO juju.cmd supercommand.go:56 running juju [2.9.37 51672c0e4243f0d0e73f13cf1bbf5c5a9a720632 gc go1.18.8]
11:42:34 DEBUG juju.cmd supercommand.go:57 args: []string{"/snap/juju/21315/bin/juju", "upgrade-charm", "acme", "--path", "./acme_ubuntu-20.04-amd64.charm", "--debug"}
11:42:34 INFO juju.juju api.go:86 connecting to API addresses: [192.168.211.241:17070]
11:42:34 DEBUG juju.api apiclient.go:1153 successfully dialed "wss://192.168.211.241:17070/model/8056ecc7-f345-40c3-8491-89b53b8746a1/api"
11:42:34 INFO juju.api apiclient.go:688 connection established to "wss://192.168.211.241:17070/model/8056ecc7-f345-40c3-8491-89b53b8746a1/api"
11:42:34 DEBUG juju.api monitor.go:35 RPC connection died
ERROR missing base name or channel not valid
11:42:34 DEBUG cmd supercommand.go:537 error stack:
missing base name or channel not valid
github.com/juju/juju/api/client/application.(*Client).GetCharmURLOrigin:277:
github.com/juju/juju/cmd/juju/application.(*refreshCommand).Run:339:
```

[Joakim Nyman (joakimnyman)]

Today I got similar error when using `export-bundle`.

```
$ juju export-bundle --filename my-bundle.yaml
ERROR extracting series from application description missing base name or channel not valid (not valid)
```

[Heather Lanigan (hmlanigan)]

I've been looking at the issue in #1, where the same error occurs for export-bundle. This is not reproducible under all circumstances, so it'd be helpful to understand could be different here.

It can happen with charmstore charms as well.

The application data in the charmstore case looks like:
{
        "_id" : "xxxx:rabbitmq-server",
        "name" : "rabbitmq-server",
        "model-uuid" : "xxxx",
        "series" : "focal",
        "subordinate" : false,
        "charmurl" : "cs:rabbitmq-server-118",
        "cs-channel" : "stable",
        "charm-origin" : {
                "source" : "charm-store",
                "id" : "",
                "hash" : "",
                "revision" : 118,
                "channel" : {
                        "risk" : "stable"
                },
                "platform" : {
                        "architecture" : "amd64",
                        "series" : "focal",
                        "os" : "ubuntu"
                }
        },
...
}

There is a workaround floating about for the export-bundle issue which has stopped working:
`db.applications.updateOne({"_id" : "<model-uuid>:<application-name>"},{ $set: {"charm-origin.platform.os":"ubuntu"}})`

FYI: Platform in state uses `series`, while core/charm Platform uses `channel`, they are equivalent and are parsed the same.

[Heather Lanigan (hmlanigan)]

@joakimnyman, would it be possible to get the output of:

JUJU_DEV_FEATURE_FLAGS=developer-mode juju dump-db

Run the command in the model which is experiencing this problem please.

[Joakim Nyman (joakimnyman)]

Regarding the export-bundle problem:

I just found out that it was the prometheus2 charm that was the cause for the export-bundle problem. Upgrading that charm to latest revision 33 solved it. I think it was at rev 25 before. I have several apps in that model and since the ERROR did not tell which app caused the problem it took some time to find it.

[Joakim Nyman (joakimnyman)]

I will do the `JUJU_DEV_FEATURE_FLAGS=developer-mode juju dump-db` thing when I find a model that has the problem. Currently we re-deploy as soon as the problem appears.

[Joakim Nyman (joakimnyman)]

Here is the output for `JUJU_DEV_FEATURE_FLAGS=developer-mode juju dump-db` on a model with the problem. As you see I have tried to upgrade it several times. The app (polkadot) is still on Rev 4 in the model.

[Joakim Nyman (joakimnyman)]

As a reference, here is a model where the Polkadot app was successfully upgraded.

[Mustafa Kemal Gilor (mustafakemalgilor)]

Hello,

One of our customers also stumbled upon this bug. I have the "db.yaml" file from the customer's environment and can confirm that "charm-origin.platform.os" field is missing in several applications. I came up with the following "yq" [1] one-liner to find which particular applications do not have the ".os" field:

-------------------------------------------------------------

cat dump.yaml | yq '.applications[] | select(.["charm-origin"].platform | has("os") == false) | ._id,.["charm-origin"].platform'

Running this command against the dump-db.yaml/dump-db-working.yaml files submitted by @Joakim results in following output:

 12:21 ❯ cat dump-db.yaml | yq '.applications[] | select(.["charm-origin"].platform | has("os") == true) | ._id,.["charm-origin"].platform'
"ba12e8ea-4869-4490-8056-ddd76032d9c1:prometheus"
{
  "architecture": "amd64",
  "os": "ubuntu",
  "series": "focal"
}
 /tmp 109ms
 12:21 ❯ cat dump-db.yaml | yq '.applications[] | select(.["charm-origin"].platform | has("os") == false) | ._id,.["charm-origin"].platform'
"ba12e8ea-4869-4490-8056-ddd76032d9c1:polkadot"
{}

-------------------------------------------------------------

The application "polkadot" lacks the "platform" key completely, as can be seen above. As for the working db.yaml file, this is not the case:

-------------------------------------------------------------

12:28 ❯ cat dump-db-working.yaml | yq '.applications[] | select(.["charm-origin"].platform | has("os") == true) | ._id,.["charm-origin"].platform'
"a68e2aae-e590-494e-8f0f-c193ba07101a:polkadot"
{
  "architecture": "amd64",
  "os": "ubuntu",
  "series": "focal"
}
"a68e2aae-e590-494e-8f0f-c193ba07101a:prometheus"
{
  "architecture": "amd64",
  "os": "ubuntu",
  "series": "focal"
}

12:28 ❯ cat dump-db-working.yaml | yq '.applications[] | select(.["charm-origin"].platform | has("os") == false) | ._id,.["charm-origin"].platform'
12:28 ❯
-------------------------------------------------------------

[1]: sudo pip3 install yq

[Sheila Miguez (codersquid)]

I think I may have encountered this bug, but maybe not since I didn't see this before today and this bug was opened late last year on an earlier version of juju. Since upgrading to juju 2.9.42 I've started getting "missing base name or channel not valid (not valid)" when running the upgrade-charm command for a local charm.

Here's the redacted log of my session

$ juju upgrade-charm --path /path/to/<charm> <app> --debug
15:06:29 INFO juju.cmd supercommand.go:56 running juju [2.9.42 7b871e782195bdac9c90f8a8f01723cc3e08ab92 gc go1.18.10]
15:06:29 DEBUG juju.cmd supercommand.go:57 args: []string{"juju", "upgrade-charm", "--path", "/path/to/<charm>", "<app>", "--debug"}
15:06:29 INFO juju.juju api.go:86 connecting to API addresses: [redacted]
15:06:29 DEBUG juju.api apiclient.go:1152 successfully dialed "wss://redacted"
15:06:29 INFO juju.api apiclient.go:687 connection established to "wss://redacted"
15:06:29 INFO juju.juju api.go:340 API endpoints changed from [redacted] to [redacted]
15:06:29 INFO juju.juju api.go:86 connecting to API addresses: [redacted]
15:06:29 DEBUG juju.api apiclient.go:1152 successfully dialed "wss://redacted"
15:06:29 INFO juju.api apiclient.go:687 connection established to "wss://redacted"
15:06:29 DEBUG juju.core.charm computedseries.go:27 series "xenial, bionic" for charm "<charm>" with format 1, Kubernetes false
15:06:31 INFO cmd refresh.go:447 Added local charm "<charm>", revision 173, to the model
15:06:31 DEBUG juju.api monitor.go:35 RPC connection died
ERROR missing base name or channel not valid (not valid)
15:06:31 DEBUG cmd supercommand.go:537 error stack:
missing base name or channel not valid (not valid)
github.com/juju/juju/rpc.(*Conn).Call:178:
github.com/juju/juju/api.(*state).APICall:1251:

[Sheila Miguez (codersquid)]

> on an earlier version of juju.

I've since been told that before the upgrade to 2.9.42 we were on 2.9.32 so it checks out that it could be this bug.

[Heather Lanigan (hmlanigan)]

Interesting, I was able to reproduce this issue today.

Started a 20.04 instance. Using juju 2.8.13 client, bootstrapped 2.8.5 and deployed the old ubuntu-plus charm with only bionic as a series. Then did an upgrade of juju which went right to 2.9.42 on the controller. Upgraded the model and tried the refresh.

$ juju upgrade-charm ubuntu-plus --path ./ubuntu-plus
Added charm "local:bionic/ubuntu-plus-1" to the model.
ERROR missing base name or channel not valid (not valid)

 "charm-origin" : {
  "source" : "local",
  "type" : "charm",
  "id" : "local:bionic/ubuntu-plus-0",
  "hash" : "",
  "revision" : 0,
  "platform" : {
   "os" : "ubuntu",
   "series" : "bionic"
  }
 }

Using a 2.9.42 client does not reproduce the problem

[Colin Watson (cjwatson)]

We're still running into this and it causes problems such as being unable to properly upgrade the Launchpad production build farm infrastructure environment. Has there been any progress?

[Heather Lanigan (hmlanigan)]

https://github.com/juju/juju/pull/15500 will be fixed in juju 2.9.43. However it's related to a client controller mismatch, which I'm sometimes told is not the case seen.

@cjwatson, do you have a reproducer? I've fixed all I can, but without a reproducer I have no data to fix any outstanding problems. Especially were reports say that a 2.9.42 client and controller are being used. None of the data I've been given so far indicates where a problem may be, other than the client/controller mismatch.

[Colin Watson (cjwatson)]

I don't have a reliable way to get an environment into the bad state, I'm afraid, just at least two environments that I manage that are currently un-upgradeable and that I need to fix one way or another.

[Colin Watson (cjwatson)]

Also, in both environments where I've encountered this (prod-launchpad-vbuilders@is-bastion-ps5, and the dogfood-launchpad-git model in stg-launchpad-git@launchpad-bastion-ps5), both client and controller are 2.9.42 - no mismatch.

[Colin Watson (cjwatson)]

So in all cases I've seen, the relevant bit of dump-db looks like this:

- _id: be99df74-4f14-4b16-841b-502092a160cb:turnip-api
  charm-origin:
    hash: ""
    id: ""
    platform: {}
    revision: 2
    source: local
  charmmodifiedversion: 1
  charmurl: local:bionic/turnip-api-2
  cs-channel: ""
  exposed: false
  forcecharm: false
  life: 0
  metric-credentials: ""
  minunits: 0
  model-uuid: be99df74-4f14-4b16-841b-502092a160cb
  name: turnip-api
  passwordhash: ""
  relationcount: 4
  scale: 0
  series: bionic
  subordinate: false
  txn-queue: []
  txn-revno: 26
  unitcount: 2

i.e. a charm-origin exists, but it has an empty platform. I'm not sure that calling `createCharmOrigin` would be correct in this case since we do know some parts of the origin (in particular the revision), but perhaps it would make sense to create a platform based on the request data, in the same way that `createCharmOrigin` does?

(I know nothing about how things could have got into this state though, beyond observing that it's happened in a couple of different places that I know of.)

[Heather Lanigan (hmlanigan)]

The cases were I can't find a problem, the db does have a platform in the charm-origin for local charms. So this is different.

Is the model running 2.9.42 also?

Do you know if this was a long running model starting before 2.9? If so, do you know if it was migrated to a 2.9 controller? Or just upgraded?

You should be able to update the application charm-origin in the db as a workaround. Though that does will not help fixing it.

Provided the machine running turnip-api is on an AMD platform, updating the charm-origin's platform to:
                "platform" : {
                        "architecture" : "amd64",
                        "series" : "bionic",
                        "os" : "ubuntu"
                }

should work.

[Guillermo Gonzalez (verterok)]

Hi, just passing by as we got hit by this (or something similar) and this might help others.

In our case is a model that was upgraded from pre-2.9, currently running 2.9.42 in both controller and client.
and the actual error from juju deploy was:

12:12:09 O: Resolving charm: filebeat
12:12:09 O: Executing changes:
12:12:09 O: - upload charm cs:filebeat-50 for series focal
12:12:09 O: - upgrade filebeat to use charm cs:filebeat-50 for series focal
12:12:09 O: ERROR cannot deploy bundle: missing base name or channel not valid (not valid)

In our case, the problem seems to be that the bundle was using `charm: filebeat` to refer to a local charm, changing that to the full path to the charm solved the problem

[Kian Parvin (kian-parvin)]

Hi,

Just to provide another scenario on this that's already been mentioned to Juju folks but is helpful on the record here, we're having the same issue with deploying Livepatch server. Interestingly compared to Guillermo's situation above, we're seeing this on `juju upgrade-charm` with an absolute path to the charm. The output was:

juju upgrade-charm --path /srv/mojo/mojo-prod-livepatch-juju2/xenial/production/charms/xenial/livepatch-server livepatch-server
Added local charm "livepatch-server", revision 44, to the model
ERROR missing base name or channel not valid (not valid)

[Heather Lanigan (hmlanigan)]

@verterok,

I'm not surprised, by default if not specified juju assumes that a charm is to be deployed from upstream, not local.
 * What version of the filebeat charm was deployed? on which series?
 * What version of the juju client were you using?
 * Is it possible to attach the bundle you were using or provide a link?

@kain-parvin,

It'd be helpful to have a record of what you're reported to us, the juju folks, around this issue with deploy. Please include the juju client version, the juju controller version and the deploy command used.

[Kian Parvin (kian-parvin)]

@hmlanigan
I don't have access to this production instance directly, but I understand the details are
controller - 2.9.42
client - 2.9.42
deploy command - "juju upgrade-charm --path <path> <application-name>"

[Salman (salmankh)]

I resolved the issue by configuring the application's series, which allowed me to subsequently upgrade the charm.

juju set-series <application> <series>
or
juju set-application-base <application> <series>

[Haw Loeung (hloeung)]

In one such instance, `juju set-application-base haproxy bionic` worked for us. FWIW, This charm was deployed from local rather than charmhub.

I'm wondering if the `juju upgrade-charm` can derive missing series, os, and platform? That's all information that's already available.

[Alan Baghumian (alanbach)]

I hit this bug while upgrading nova-compute units from Focal to Jammy.

$ juju upgrade-series 8 prepare jammy
WARNING: This command will mark machine "8" as being upgraded to series "jammy".
This operation cannot be reverted or canceled once started.
Units running on the machine will also be upgraded. These units include:
  - canonical-livepatch/4
  - ceph-osd-nvme/5
  - lds-client-focal/16
  - nova-compute/5
  - ntp/0
  - ovn-chassis/3
  - telegraf/1

Leadership for the following applications will be pinned and not
subject to change until the "complete" command is run:
  - canonical-livepatch
  - ceph-osd-nvme
  - lds-client-focal
  - nova-compute
  - ntp
  - ovn-chassis
  - telegraf

Continue [y/N]?y
ERROR empty id not valid

Digging deeper, showed ntp is missing the charm-id:

juju:PRIMARY> db.applications.find({"model-uuid" : "98f30d5a-977c-4ce9-80ef-c97152b096f0"}, { "charm-origin.id": 1 });
...
...
{ "_id" : "98f30d5a-977c-4ce9-80ef-c97152b096f0:ntp", "charm-origin" : { "id" : "" } }
...
...

Found the charm id here:

$ juju info ntp
name: ntp
publisher: NTP charm developers
summary: Network Time Protocol
description: |
  NTP, the Network Time Protocol, is used to keep computer clocks accurate
  by synchronizing them over the Internet or a local network, or by
  following an accurate hardware receiver that interprets GPS, DCF-77,
  NIST or similar time signals.
  .
  This charm can be deployed alongside principal charms to enable NTP
  management across deployed services.
store-url: https://charmhub.io/ntp
charm-id: SLOkZHn8HVy160M2oJdvXatzlu7ZBEVx
supports: jammy, focal, bionic, xenial, trusty
subordinate: true
relations:
  provides:
    nrpe-external-master: nrpe-external-master
    ntpmaster: ntp
  requires:
    juju-info: juju-info
    master: ntp
channels: |
  latest/stable: cs-ntp-team-ntp-4-171-g669ff59 2022-08-29 (50) 5MB
  latest/candidate: cs-ntp-team-ntp-4-171-g669ff59 2022-08-29 (50) 5MB
  latest/beta: ↑
  latest/edge: ↑

Then updated the database:

juju:PRIMARY> db.applications.update({"_id" : "98f30d5a-977c-4ce9-80ef-c97152b096f0:ntp"}, { $set: { "charm-origin.id": "SLOkZHn8HVy160M2oJdvXatzlu7ZBEVx" } });
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

That let me proceed with the series upgrade.