Mismatched version of charm and deb due to failed apt-get update

[Andreas Hasenack]

https://ci.lscape.net/job/landscape-system-tests/2998/

Got this traceback in the gs3 logs:
ERROR * 07-01 07:11:15 [PID:9225] * root * Exception during syncing:
Traceback (most recent call last):
  File "/usr/share/glance-simplestreams-sync/glance-simplestreams-sync.py", line 441, in main
    do_sync(charm_conf, status_exchange)
  File "/usr/share/glance-simplestreams-sync/glance-simplestreams-sync.py", line 175, in do_sync
    mirror_url, policy=policy, user_agent=user_agent)
TypeError: __init__() got an unexpected keyword argument 'user_agent'

The charm was cs:~landscape-charmers/trusty/glance-simplestreams-sync-10

The simplestreams package is python-simplestreams:amd64 (0.1.0~bzr341-0ubuntu2.4)

Our ppa:landscape/glance-simplestreams-sync has 0.1.0~bzr437-0ubuntu1~14.04.440~ubuntu14.04.1. Don't know why we installed an older version:
2016-07-01 06:27:40 INFO install Get:27 http://archive.ubuntu.com/ubuntu/ trusty-updates/main python-simplestreams all 0.1.0~bzr341-0ubuntu2.4 [24.2 kB]

Maybe we failed to add the PPA.

UPDATE: see comment #2 for diagnosis

[Andreas Hasenack (ahasenack)]

ok, I think I know what happened. The logs below show the ppa being added, then an error message about a lock, then an immediate apt-get install, and we install python-simplestreams from trusty, not our ppa. Looks like the lock error message are about apt-get update, which I don't see being run:
2016-07-01 06:27:38 INFO install gpg: keyring `/tmp/tmpnba49qkg/secring.gpg' created
2016-07-01 06:27:38 INFO install gpg: keyring `/tmp/tmpnba49qkg/pubring.gpg' created
2016-07-01 06:27:38 INFO install gpg: requesting key 4652B4E6 from hkp server keyserver.ubuntu.com
2016-07-01 06:27:38 INFO install gpg: /tmp/tmpnba49qkg/trustdb.gpg: trustdb created
2016-07-01 06:27:38 INFO install gpg: key 4652B4E6: public key "Launchpad PPA for Landscape" imported
2016-07-01 06:27:38 INFO install gpg: Total number processed: 1
2016-07-01 06:27:38 INFO install gpg: imported: 1 (RSA: 1)
2016-07-01 06:27:38 INFO install OK
2016-07-01 06:27:38 INFO install Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.xOods5XDk6 --no-auto-check-trustdb --trust-mo
del always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d/landscape-glance-simplestreams-sync.gpg --keyserver hkp
://keyserver.ubuntu.com:80 --recv 4652B4E6
2016-07-01 06:27:38 INFO install gpg: requesting key 4652B4E6 from hkp server keyserver.ubuntu.com
2016-07-01 06:27:39 INFO install gpg: key 4652B4E6: "Launchpad PPA for Landscape" not changed
2016-07-01 06:27:39 INFO install gpg: Total number processed: 1
2016-07-01 06:27:39 INFO install gpg: unchanged: 1
2016-07-01 06:27:39 INFO juju-log creating config dir at /etc/glance-simplestreams-sync
2016-07-01 06:27:39 INFO juju-log creating config dir at /usr/share/glance-simplestreams-sync
2016-07-01 06:27:39 INFO install E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
2016-07-01 06:27:39 INFO install E: Unable to lock directory /var/lib/apt/lists/
2016-07-01 06:27:39 INFO juju-log Installing ['python-simplestreams', 'python-glanceclient', 'python-yaml', 'python-keystoneclient', 'python-kombu', 'python-swiftclient', 'ubuntu-cloudimage-keyring'] with options: ['--option=Dpkg::Options::=--force-confold']
2016-07-01 06:27:40 INFO install Reading package lists...
2016-07-01 06:27:40 INFO install Building dependency tree...
2016-07-01 06:27:40 INFO install Reading state information...
2016-07-01 06:27:40 INFO install python-yaml is already the newest version.

So, bottom line, I think we need to make sure apt-get update retries.

[Andreas Hasenack (ahasenack)]

Added landscape so we do a charm bump once this is fixed in the charm.

[Andreas Hasenack (ahasenack)]

In hooks/hooks.py:
(...)
    add_source('ppa:cloud-installer/simplestreams-testing')
    apt_update()

    apt_install(packages=['python-simplestreams', 'python-glanceclient',
                          'python-yaml', 'python-keystoneclient',
                          'python-kombu',
                          'python-swiftclient', 'ubuntu-cloudimage-keyring'])

Maybe we should call apt_update(fatal=True). It's how configure_sources(), also from charm helpers, calls it after adding a new source.