upgrade-charm can fail at getting the apt/dpkg lock

[Aurelien Lourot]

Happened here [0] on the CI system in a bionic-rocky stable_to_next upgrade test:

2020-05-20 06:32:17 INFO juju-log Installing [] with options: ['--option=Dpkg::Options::=--force-confold']
2020-05-20 06:32:17 DEBUG upgrade-charm E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
2020-05-20 06:32:17 DEBUG upgrade-charm E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
2020-05-20 06:32:17 INFO juju-log Couldn't acquire DPKG lock. Will retry in 10 seconds
2020-05-20 06:32:27 DEBUG upgrade-charm E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
2020-05-20 06:32:27 DEBUG upgrade-charm E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
2020-05-20 06:32:27 INFO juju-log Couldn't acquire DPKG lock. Will retry in 10 seconds
2020-05-20 06:32:37 DEBUG upgrade-charm E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
2020-05-20 06:32:37 DEBUG upgrade-charm E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
2020-05-20 06:32:37 INFO juju-log Couldn't acquire DPKG lock. Will retry in 10 seconds
2020-05-20 06:32:47 DEBUG upgrade-charm E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
2020-05-20 06:32:47 DEBUG upgrade-charm E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
2020-05-20 06:32:47 DEBUG upgrade-charm Traceback (most recent call last):
2020-05-20 06:32:47 DEBUG upgrade-charm File "/var/lib/juju/agents/unit-nova-compute-2/charm/hooks/upgrade-charm.real", line 774, in <module>
2020-05-20 06:32:47 DEBUG upgrade-charm main()
2020-05-20 06:32:47 DEBUG upgrade-charm File "/var/lib/juju/agents/unit-nova-compute-2/charm/hooks/upgrade-charm.real", line 767, in main
2020-05-20 06:32:47 DEBUG upgrade-charm hooks.execute(sys.argv)
2020-05-20 06:32:47 DEBUG upgrade-charm File "/var/lib/juju/agents/unit-nova-compute-2/charm/hooks/charmhelpers/core/hookenv.py", line 943, in execute
2020-05-20 06:32:47 DEBUG upgrade-charm self._hooks[hook_name]()
2020-05-20 06:32:47 DEBUG upgrade-charm File "/var/lib/juju/agents/unit-nova-compute-2/charm/hooks/charmhelpers/contrib/hardening/harden.py", line 93, in _harden_inner2
2020-05-20 06:32:47 DEBUG upgrade-charm return f(*args, **kwargs)
2020-05-20 06:32:47 DEBUG upgrade-charm File "/var/lib/juju/agents/unit-nova-compute-2/charm/hooks/upgrade-charm.real", line 543, in upgrade_charm
2020-05-20 06:32:47 DEBUG upgrade-charm fatal=True)
2020-05-20 06:32:47 DEBUG upgrade-charm File "/var/lib/juju/agents/unit-nova-compute-2/charm/hooks/charmhelpers/fetch/ubuntu.py", line 276, in apt_install
2020-05-20 06:32:47 DEBUG upgrade-charm _run_apt_command(cmd, fatal)
2020-05-20 06:32:47 DEBUG upgrade-charm File "/var/lib/juju/agents/unit-nova-compute-2/charm/hooks/charmhelpers/fetch/ubuntu.py", line 770, in _run_apt_command
2020-05-20 06:32:47 DEBUG upgrade-charm retry_message="Couldn't acquire DPKG lock")
2020-05-20 06:32:47 DEBUG upgrade-charm File "/var/lib/juju/agents/unit-nova-compute-2/charm/hooks/charmhelpers/fetch/ubuntu.py", line 748, in _run_with_retries
2020-05-20 06:32:47 DEBUG upgrade-charm result = subprocess.check_call(cmd, env=env)
2020-05-20 06:32:47 DEBUG upgrade-charm File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
2020-05-20 06:32:47 DEBUG upgrade-charm raise CalledProcessError(retcode, cmd)
2020-05-20 06:32:47 DEBUG upgrade-charm subprocess.CalledProcessError: Command '['apt-get', '--assume-yes', '--option=Dpkg::Options::=--force-confold', 'install']' returned non-zero exit status 100.
2020-05-20 06:32:48 ERROR juju.worker.uniter.operation runhook.go:132 hook "upgrade-charm" failed: exit status 1

Maybe due to some auto-updates [1] enabled in Ubuntu?

0: http://10.245.162.58:8080/view/MojoMatrix/job/mojo_runner/22544/
1: https://help.ubuntu.com/community/AutomaticSecurityUpdates

[Billy Olsen (billy-olsen)]

I suspect this is caused by unattended upgrades, and can always be a problem that another process has the apt/dpkg lock.