Cannot purge a removed ppa

[Stdedos]

Recently, `ppa:jonathonf/ffmpeg-3` was abandoned/removed.

Replicate with:
$ sudo ppa-purge ppa:jonathonf/ffmpeg-3 # After having the PPA added, which is not possible anymore

Expected result:
I'd like to disable it and check if reverting would be straightforward.

Actual result:
Updating packages lists
W: The repository 'http://ppa.launchpad.net/jonathonf/ffmpeg-3/ubuntu xenial Release' does not have a Release file.
E: Failed to fetch http://ppa.launchpad.net/jonathonf/ffmpeg-3/ubuntu/dists/xenial/main/binary-amd64/Packages 403 Forbidden [IP: 91.189.95.83 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
Warning: apt-get update failed for some reason

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Allan W. Macdonald (allan-w-macdonald)]

I believe that this may have happened to me with ppa:oibaf/graphics-drivers

$ sudo ppa-purge ppa:oibaf/graphics-drivers
Updating packages lists
PPA to be removed: oibaf graphics-drivers
Warning: Could not find package list for PPA: oibaf graphics-drivers
$

In this case, the ppa exists for newer releases of ubuntu but the support for bionic has been removed. In any case, I am stuck with packages that were installed via the ppa but I have no way to revert them to official package types. This seems to prevent do-release-upgrade from completing.

See also:
https://answers.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+question/696223
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1920829
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1069133