ubuntu_docker_smoke_test fails on ppc64el and s390x

Bug #1983786 reported by Francis Ginther
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Invalid
Undecided
Unassigned
Ubuntu on IBM z Systems
Invalid
Undecided
Unassigned
ubuntu-kernel-tests
Invalid
Undecided
Unassigned

Bug Description

Setup for the ubuntu_docker_smoke_test suite fails on both ppc64el and s390x (see console log below). We should disable the test for these architectures if this can't be resolved.

[console log]
11:50:10 ERROR| [stderr] ++ sudo docker run ubuntu:bionic /bin/sh -c 'lsb_release -sc'
11:50:10 ERROR| [stderr] Unable to find image 'ubuntu:bionic' locally
11:50:13 ERROR| [stderr] bionic: Pulling from library/ubuntu
11:50:14 ERROR| [stderr] 1a9ff13e38ca: Pulling fs layer
11:50:17 ERROR| [stderr] 1a9ff13e38ca: Verifying Checksum
11:50:17 ERROR| [stderr] 1a9ff13e38ca: Download complete
11:50:21 ERROR| [stderr] 1a9ff13e38ca: Pull complete
11:50:21 ERROR| [stderr] Digest: sha256:eb1392bbdde63147bc2b4ff1a4053dcfe6d15e4dfd3cce29e9b9f52a4f88bc74
11:50:21 ERROR| [stderr] Status: Downloaded newer image for ubuntu:bionic
11:50:32 ERROR| [stderr] /bin/sh: 1: lsb_release: not found
11:50:33 ERROR| [stderr] + RESULT=
11:50:33 DEBUG| [stdout] ** ERROR: The codename returned from lsb_release -sc was ( ) and we were expecting bionic
11:50:33 ERROR| [stderr] + '[' '' '!=' bionic ']'
11:50:33 ERROR| [stderr] + echo '** ERROR: The codename returned from lsb_release -sc was (' ') and we were expecting bionic'
11:50:33 ERROR| [stderr] + exit 1
11:50:33 ERROR| Exception escaping from test:
Traceback (most recent call last):
  File "/home/ubuntu/autotest/client/shared/test.py", line 411, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/home/ubuntu/autotest/client/shared/test.py", line 823, in _call_test_function
    return func(*args, **dargs)
  File "/home/ubuntu/autotest/client/shared/test.py", line 291, in execute
    postprocess_profiled_run, args, dargs)
  File "/home/ubuntu/autotest/client/shared/test.py", line 212, in _call_run_once
    self.run_once(*args, **dargs)
  File "/home/ubuntu/autotest/client/tests/ubuntu_docker_smoke_test/ubuntu_docker_smoke_test.py", line 39, in run_once
    self.results = utils.system_output(cmd, retain_output=True)
  File "/home/ubuntu/autotest/client/shared/utils.py", line 1268, in system_output
    verbose=verbose, args=args).stdout
  File "/home/ubuntu/autotest/client/shared/utils.py", line 918, in run
    "Command returned non-zero exit status")
CmdError: Command <sudo /home/ubuntu/autotest/client/tests/ubuntu_docker_smoke_test/the-test> failed, rc=1, Command returned non-zero exit status
* Command:
    sudo /home/ubuntu/autotest/client/tests/ubuntu_docker_smoke_test/the-test
Exit status: 1
Duration: 40.8677330017

stdout:
Running in the Canonical CI environment
** ERROR: The codename returned from lsb_release -sc was ( ) and we were expecting bionic
stderr:
+ export http_proxy=
+ http_proxy=
+ echo ''
+ nc -w 2 squid.internal 3128
+ echo 'Running in the Canonical CI environment'
+ export http_proxy=http://squid.internal:3128
+ http_proxy=http://squid.internal:3128
+ '[' -n http://squid.internal:3128 ']'
+ command -v systemctl
+ '[' 0 -eq 0 ']'
+ '[' '!' -d /etc/systemd/system/docker.service.d ']'
+ mkdir /etc/systemd/system/docker.service.d
+ echo -e '[Service]\nEnvironment=HTTP_PROXY=http://squid.internal:3128\n'
+ systemctl daemon-reload
+ systemctl restart docker
+ release=bionic
+ image=ubuntu:bionic
++ uname -m
+ '[' ppc64le = i686 ']'
++ sudo docker run ubuntu:bionic /bin/sh -c 'lsb_release -sc'
Unable to find image 'ubuntu:bionic' locally
bionic: Pulling from library/ubuntu
1a9ff13e38ca: Pulling fs layer
1a9ff13e38ca: Verifying Checksum
1a9ff13e38ca: Download complete
1a9ff13e38ca: Pull complete
Digest: sha256:eb1392bbdde63147bc2b4ff1a4053dcfe6d15e4dfd3cce29e9b9f52a4f88bc74
Status: Downloaded newer image for ubuntu:bionic
/bin/sh: 1: lsb_release: not found
+ RESULT=
+ '[' '' '!=' bionic ']'
+ echo '** ERROR: The codename returned from lsb_release -sc was (' ') and we were expecting bionic'
+ exit 1

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Hey Francis,
On which kernel / release / cloud you're seeing this failure?

tags: added: ubuntu-docker-smoke-test
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

To comment #1
this issue was found on c2022.08.09 Bionic 4.15.0-191.202

Revision history for this message
Patricia Domingues (patriciasd) wrote (last edit ):

hey, I was testing this on a ppc64le, not running the autopkgtest or kernel testing, just downloaded the image and I was also not able to run the `lsb_release -sd` command because theres'no lsb_release in the docker image.

If I install `lsb-core` in the docker instance, so the command will work:

root@9a6824dc9e4e:/# apt update

root@9a6824dc9e4e:/# lsb_release -a
bash: lsb_release: command not found

root@9a6824dc9e4e:/# apt install lsb-core
...
root@9a6824dc9e4e:/# lsb_release -a
LSB Version: core-9.20170808ubuntu1-noarch:security-9.20170808ubuntu1-noarch
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic

Revision history for this message
Patricia Domingues (patriciasd) wrote :

but if you can add instructions on how to reproduce your testing... I can run that.

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Hi Patricia,

It's:
git clone --depth=1 git://git.launchpad.net/~canonical-kernel-team/+git/autotest-client-tests
git clone --depth=1 git://git.launchpad.net/~canonical-kernel-team/+git/autotest
rm -fr autotest/client/tests
ln -sf ~/autotest-client-tests autotest/client/tests
sudo apt-get install git python-minimal -y || sudo apt install python2 -y
AUTOTEST_PATH=/home/ubuntu/autotest sudo -E autotest/client/autotest-local --verbose autotest/client/tests/ubuntu_docker_smoke_test/control

So base on your tests, I think it's caused by the image from docker.io. Not sure why this lsb-core got dropped from it.

But I think we can change our test to make it less-dependent to a certain package.

Revision history for this message
Patricia Domingues (patriciasd) wrote :

Actually I do not see `lsb-core` in ppc64le, but
the output of running on power bare metal with Bionic:

```
ubuntu@tiselius:~$ which lsb_release
/usr/bin/lsb_release
```

ubuntu@tiselius:~$ dpkg -l |grep lsb
ii lsb-base 9.20170808ubuntu1 all Linux Standard Base init script functionality
ii lsb-release 9.20170808ubuntu1 all Linux Standard Base version reporting utility

It seems it's missing the `lsb-release` pkg (not lsb-core). lsb-core is just another option ...

:~$ sudo docker run -it ubuntu:bionic bash

root@426a4cf9623a:/# lsb_release -a
bash: lsb_release: command not found

root@426a4cf9623a:/# which lsb_release (NO output)
root@426a4cf9623a:/#

root@426a4cf9623a:/# dpkg -l |grep lsb
ii lsb-base 9.20170808ubuntu1 all Linux Standard Base init script functionality

root@426a4cf9623a:/# apt update && apt install lsb-release -y
0% [Connecting to ports.ubuntu.com]
...
root@426a4cf9623a:/# lsb_release -sd
Ubuntu 18.04.6 LTS

root@426a4cf9623a:/# dpkg -l |grep lsb
ii lsb-base 9.20170808ubuntu1 all Linux Standard Base init script functionality
ii lsb-release 9.20170808ubuntu1 all Linux Standard Base version reporting utility

root@426a4cf9623a:/# which lsb_release
/usr/bin/lsb_release (same output from a non cloud image)

...

I was checking an LXD instance:
it has `lsb-release`
( focal host:

 ubuntu@thiel:~$ snap list
 Name Version Rev Tracking Publisher Notes
 core20 20220805 1612 latest/stable canonical✓ base
 lxd 4.0.9-8e2046b 22757 4.0/stable/… canonical✓ -
 snapd 2.56.2 16298 latest/stable canonical✓ snapd
)

ubuntu@thiel:~$ lxc launch ubuntu:18.04 bionic
Creating bionic
Starting bionic
ubuntu@thiel:~$ lxc exec bionic bash

root@bionic:~# which lsb_release
/usr/bin/lsb_release

root@bionic:~# lsb_release -sd
Ubuntu 18.04.6 LTS

root@bionic:~# dpkg -l |grep lsb
ii lsb-base 9.20170808ubuntu1 all Linux Standard Base init script functionality
ii lsb-release 9.20170808ubuntu1 all Linux Standard Base version reporting utility

Revision history for this message
Patricia Domingues (patriciasd) wrote :

Po-Hsu Lin, I've ran with success the ubuntu_docker_smoke_test, but I saw you have changed it

`-c 'cat /etc/lsb-release ` :

14:51:33 ERROR| [stderr] ++ sudo docker run ubuntu:bionic /bin/sh -c 'cat /etc/lsb-release | grep CODENAME | cut -d '\''='\'' -f 2'
14:51:35 ERROR| [stderr] + RESULT=bionic
14:51:35 ERROR| [stderr] + '[' bionic '!=' bionic ']'
14:51:35 INFO | GOOD ubuntu_docker_smoke_test.docker_smoke_test ubuntu_docker_smoke_test.docker_smoke_test timestamp=1660834295 localtime=Aug 18 14:51:35 completed successfully
14:51:35 INFO | END GOOD ubuntu_docker_smoke_test.docker_smoke_test ubuntu_docker_smoke_test.docker_smoke_test timestamp=1660834295 localtime=Aug 18 14:51:35
14:51:35 DEBUG| Persistent state client._record_indent now set to 1
14:51:35 DEBUG| Persistent state client.unexpected_reboot deleted
14:51:35 INFO | END GOOD ---- ---- timestamp=1660834295 localtime=Aug 18 14:51:35
14:51:35 DEBUG| Persistent state client._record_indent now set to 0
14:51:35 INFO | Report successfully generated at /home/ubuntu/autotest/client/results/default/job_report.html

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

No I haven't change anything yet haha.
Perhaps the image got updated on their (docker.io) side?

Revision history for this message
Patricia Domingues (patriciasd) wrote :

the bug description shows it is running `- c lsb_release -sc` command: and then it fails

"
 11:50:10 ERROR| [stderr] ++ sudo docker run ubuntu:bionic /bin/sh -c 'lsb_release -sc'
"

from my comment #7: it is not anymore the `lsb_release -sc` command and Instead it is getting the `lsb-release`-file content:::

 `-c 'cat /etc/lsb-release`

"
 14:51:33 ERROR| [stderr] ++ sudo docker run ubuntu:bionic /bin/sh -c 'cat /etc/lsb-release | grep CODENAME | cut -d '\''='\'' -f 2'
"

Revision history for this message
Alexandre Erwin Ittner (aittner) wrote :

This also happens in s390x and amd64. That's expected as these images stopping including command lsb_release since at least image ubuntu:bionic-20220315.

The latest image for s390x listed in https://github.com/docker-library/official-images/blob/master/library/ubuntu is built from commit ff6467ff309d1e8c1834f45dc088ab9bf71d019b in https://github.com/tianon/docker-brew-ubuntu-core/ and the manifest file for this commit ( https://github.com/tianon/docker-brew-ubuntu-core/blob/ff6467ff309d1e8c1834f45dc088ab9bf71d019b/bionic/ubuntu-bionic-oci-s390x-root.manifest ) does not list package lsb-release.

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

I look more closely into the test case, and I found why it's a bit confusing while reading the test output.

The test script is calling the following command to the the running version:
RESULT=`sudo docker run $image /bin/sh -c "cat /etc/lsb-release | grep CODENAME | cut -d '=' -f 2"`

However, when it fails it will print:
  echo "** ERROR: The codename returned from lsb_release -c was (" $RESULT ") and we were expecting $release"

I am not sure why in the bug description it's trying to run lsb_release:
11:50:10 ERROR| [stderr] ++ sudo docker run ubuntu:bionic /bin/sh -c 'lsb_release -sc'

grep -r 'lsb_release -sc' * in autotest-client-tests didn't return anything.
Will confirm with Francis about this.

Revision history for this message
Patricia Domingues (patriciasd) wrote :

hi Po-Hsu Lin, I'm marking as invalid for now, but feel free to re-open if you see any other issue, thanks

Changed in ubuntu-power-systems:
status: New → Invalid
Changed in ubuntu-z-systems:
status: New → Invalid
Changed in ubuntu-kernel-tests:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.