Use chdist to avoid modifying the system apt cache

[Haw Loeung]

Hi,

In it's current form, it modifies the system apt cache with -o Dir::Etc::SourceList="/etc/apt-stresstest/54.207.89.157-sources.list". So during and after a run, the system apt cache is hosed and fails:

| root@ip-172-31-2-21:~# apt-cache policy ubuntu-wallpapers
| N: Unable to locate package ubuntu-wallpapers

It is only after an 'apt-get update', without the Dir::Etc::SourceList modifier so it uses /etc/apt/sources.list and sources.d:

| root@ip-172-31-2-21:~# apt-get update
| Get:1 http://sa-east-1.ec2.archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
| ...
| Fetched 28.9 MB in 8s (3,224 kB/s)
| Reading package lists... Done
| root@ip-172-31-2-21:~# apt-cache search wallpapers
| ubuntu-wallpapers - Ubuntu Wallpapers
| ubuntu-wallpapers-xenial - Ubuntu 16.04 Wallpapers

This means security and package updates aren't applied.

I think we should change to using chdist. It shouldn't be that hard to rework to and would allow us to run as a non-root user. With this, we can also check for other series/distros.

~/.chdist structure:

| $ find -type f
| TEMPLATE/var/lib/dpkg/status
| TEMPLATE/etc/apt/sources.list
| TEMPLATE/etc/apt/apt.conf
| TEMPLATE/etc/apt/trusted.gpg.d/ubuntu-archive-keyring.gpg

TEMPLATE/etc/apt/sources.list:

| deb [arch=amd64] TEMPLATE-REPLACE bionic main
| deb-src TEMPLATE-REPLACE bionic main

TEMPLATE./etc/apt/apt.conf:

| Apt {
| Architecture "amd64";
| };
|
| Dir "/home/$USER/.chdist/TEMPLATE-REPLACE";
| Dir::State::status "/home/$USER/.chdist/TEMPLATE-REPLACE/var/lib/dpkg/status";

[Haw Loeung (hloeung)]

We also avoid these locking issues if apt is running (from outside stresstest):

| DEBUG: Sending data to influxdb: 127.0.0.1:8094
| E: Unable to locate package texlive-latex-extra-doc
| E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
| E: Unable to lock directory /var/lib/apt/lists/
apt_transaction_duration_seconds,test=uncached_download_2_texlive-latex-extra-doc,mirror=sa-east-1.ec2.archive.ubuntu.com,ip=52.67.190.94 value=0.065211

[Haw Loeung (hloeung)]

This also causes inaccurate metrics where a system apt-get upgrade (unattended-upgrades or so) would cause the script to block until that one finishes or fail.