Charm should not clean older metadata that ubuntu_active points to

[Haw Loeung]

Hi,

On a few occasions, multiple metadata sync succeeds but confused leader election prevents the ubuntu_active symlink to be updated/switched (see LP: #1797297). This results in the symlink pointing at a no longer available metadata directory.

The charm should not remove older metadata directories that the "ubuntu_archive" symlink points at. Maybe it's a bug here?

https://bazaar.launchpad.net/~cloudware-admins/ubuntu-repository-cache/trunk/view/head:/lib/ubuntu_repository_cache/metadata_sync.py#L234

[Alexandre Gomes (alejdg)]

Here is an example of the link pointing to an invalid directory:
https://pastebin.canonical.com/p/BX6TvDdWmK/

And some logs of the sync failing because that directory does not exist:
https://pastebin.canonical.com/p/K4DCpmpyRd/

[Haw Loeung (hloeung)]

Here's one confusing log:

https://pastebin.canonical.com/p/WD6w5N3zvJ/

[Haw Loeung (hloeung)]

This is still happening:

| ubuntu@machine-0:/srv/ubuntu-repository-cache/apache/data$ ls -la
| total 20
| drwxrwxr-x 4 www-sync www-data 4096 Sep 14 22:44 .
| drwxrwxr-x 4 www-data www-data 4096 Jul 8 03:48 ..
| drwxrwxr-x 7 www-sync www-data 4096 Sep 14 21:26 ubuntu_2020-09-14_21:40:02_u0
| -rw-r--r-- 1 root root 0 Sep 14 21:43 ubuntu_2020-09-14_21:40:02_u0_good
| drwxrwxr-x 7 www-sync www-data 4096 Sep 14 21:26 ubuntu_2020-09-14_22:40:02_u0
| -rw-r--r-- 1 root root 0 Sep 14 22:43 ubuntu_2020-09-14_22:40:02_u0_good
| lrwxrwxrwx 1 www-sync www-data 70 Sep 14 22:44 ubuntu_active -> /srv/ubuntu-repository-cache/apache/data/ubuntu_2020-09-14_20:40:02_u0

We should fix this.

[Junien F (axino)]

This is still happening (see portal event 45278). Re-opening.

[Haw Loeung (hloeung)]

Latest LP:1973744