Misleading error the second time an upload is tried

[Facundo Batista]

I tried to upload a charm with a problem, first time I got:

{'revisions': [{'errors': [{'code': 'review-error', 'message': "'stuff' is not one of ['file', 'oci-image'] at /resources/0/type"}], 'revision': 4, 'status': 'rejected', 'upload-id': '34a90728-2e30-4f24-b951-024645a0cb53'}]}

That is fine.

I re-tried the upload without touching anything, and I got:

{'revisions': [{'errors': [{'code': 'review-error', 'message': "Cannot insert SHA3-384 '8b5968db2c63e5c850696dcea75b0554133e802a4353a333733045296de6d0c401599732c612452e010cd79e91fd2b19'. A snap revision with that digest already exists in the database."}], 'revision': 5, 'status': 'rejected', 'upload-id': '2668c8ca-452f-4112-9f64-7ff67ac10770'}]}

That is misleading. Also, I'm pushing a charm, not a snap.

Thanks!

[Jonathan Hartley (tartley)]

This is a longstanding constraint of snaprevs: It has never allowed a snap to upload two identical revisions. The same constraint applies to charms, even though this old error message mentions 'snap'.

This problem previously impacted the migration of charms from charmstore to charmhub. Many charms in charmstore do have some identical revisions. (Perhaps some publishers mistakenly thought they had to upload the same revision once for each supportedseries?) The migration currently skips all these clashing revisions but for the one with the highest revision number.

When this was first observed, discussion concluded that this doesn't matter much, it only affects older revisions, and we don't need to fix it for now, unless it proves to be a big problem.

I agree the error message ought to say "{package-type}" instead of "snap". But other than that, when you say 'misleading', what precisely did you mean? What is your expected behavior? A better error message (what?) or to accept the upload?

[Daniel Manrique (roadmr)]

Also, can you detail the charmcraft commands you used for this?

The push process is probably done in two steps:

1- upload charm payload
2- upload resource

so what happened here on your first try:

1- upload the charm, accepted
2- upload the resource, rejected, invalid type

and on the second:
1- upload the charm, rejected, we already have this blob.

Sounds like in this case charmcraft could look at the error, realize the charm blob already exists, and retry only the upload resource operation (with the charm revision given in the error message if needed).

We can fix showing package-type to make the error message clearer, but we cannot change the backend to accept the same binary upload more than once.

[Facundo Batista (facundo)]

Jonathan (and see below 'both'):

I understand that two identical revisions are not allowed (for snaps and charms).

However, from the user POV, this is not the case! The first attempt fails, so I don't expect for the charm to be in Charmhub. But because of whatever reason I miss the real problem.

Then I try again (this time with --verbose, or not sending error to a file, or whatever), to see what is the problem, and the second response is telling me that the second upload fails because the charm is already in Charmhub?

So, the first attempt was a success and the charm is already up?

Daniel (and see below 'both'):

Process to make this happen... create a charm project with a bad metadata, build and upload:

:: charmcraft init
:: (edit metadata.yaml, fix its name for something you own, add a bad resources like...
::
:: resources:
:: foobar:
:: type: asdasd
:: ...)
:: charmcraft build
:: charmcraft upload -v --charm-file your-charm-file.charm
::
:: (that should give you the first error, and then...)
::
:: charmcraft upload -v --charm-file your-charm-file.charm
::
:: (this should give you the second error)

The rest of your comment is not of use here, as I'm not trying to upload a resource at all. Just trying to upload a charm which has a resource key in its metadata.

Both:

Note that if the charm has a wrong metadata because 'series' is missing, the first time it's uploaded the error will be that it's "missing series", and the second time! and the third time!

So, if the metadata is wrong *in that way*, it behaves correctly.

But if the metadata is wrong because an invalid resource key, Charmhub behaves wrongly.

[Facundo Batista (facundo)]

AGGGGGGGGG, Launchpad totally messed up my formating.

Jonathan, Daniel, please look at https://paste.ubuntu.com/p/44QGGhQz67/

[Jonathan Hartley (tartley)]

Facundo, Thank you for explaining, I understand now.

The cause is, our migration code does a silly temporary flow that has never been fixed:

1. Create the charm with minimal metadata (so that it is valid to create revisions)
2. Process revisions, which includes creating in snaprevs, and accumulating charm metadata.
3. Update the now complete charm metadata.

Reordering these parts to be more sensible should be relatively easy, but has never been prioritized. There are a few drivers for it though, this bug is now an extra one. More as it happens.

[Jonathan Hartley (tartley)]

Sorry, ignore my comment above, clearly wasn't thinking straight. You are talking about charmcraft uploads.