avocado tests not running on aarch64 host

[Philippe Mathieu-Daudé]

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal

$ make check-venv
  VENV /home/phil/qemu/build/tests/venv
  PIP /home/phil/qemu/tests/requirements.txt
  ERROR: Command errored out with exit status 1:
   command: /home/phil/qemu/build/tests/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-w1h2bh4a/pycdlib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-w1h2bh4a/pycdlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ic25ctcg
       cwd: /tmp/pip-install-w1h2bh4a/pycdlib/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for pycdlib
$

[Philippe Mathieu-Daudé (philmd)]

Eric Auger suggested the fix: 'pip install wheel'.

[Cleber Rosa (cleber-gnu)]

Looks like an installation problem. On Ubuntu 20.04, python3-pip depends on python3-wheel:

$ apt-cache show python3-pip | grep Depends
Depends: ca-certificates, python3-distutils, python3-setuptools, python3-wheel, python-pip-whl (= 20.0.2-5ubuntu1), python3:any

And it gets pulled during an installation attempt:

$ apt install python3-pip
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  binutils binutils-aarch64-linux-gnu binutils-common build-essential ca-certificates cpp cpp-9 dirmngr dpkg-dev fakeroot file g++ g++-9 gcc
  gcc-9 gcc-9-base gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libalgorithm-diff-perl
  libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libasn1-8-heimdal libassuan0 libatomic1 libbinutils libc-dev-bin libc6-dev
  libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libexpat1 libexpat1-dev libfakeroot libfile-fcntllock-perl libgcc-9-dev
  libgdbm-compat4 libgdbm6 libgomp1 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal
  libisl22 libitm1 libkrb5-26-heimdal libksba8 libldap-2.4-2 libldap-common liblocale-gettext-perl liblsan0 libmagic-mgc libmagic1 libmpc3
  libmpdec2 libmpfr6 libnpth0 libperl5.30 libpython3-dev libpython3-stdlib libpython3.8 libpython3.8-dev libpython3.8-minimal
  libpython3.8-stdlib libreadline8 libroken18-heimdal libsasl2-2 libsasl2-modules libsasl2-modules-db libsqlite3-0 libssl1.1 libstdc++-9-dev
  libtsan0 libubsan1 libwind0-heimdal linux-libc-dev make manpages manpages-dev mime-support netbase openssl patch perl perl-modules-5.30
  pinentry-curses python-pip-whl python3 python3-dev python3-distutils python3-lib2to3 python3-minimal python3-pkg-resources python3-setuptools
  python3-wheel python3.8 python3.8-dev python3.8-minimal readline-common xz-utils zlib1g-dev

This is on:

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal

[Cleber Rosa (cleber-gnu)]

This seems to happen on this specific machine, not because it's an aarch64 machine, but because it has python3-minimal installed. The docs on the acceptance test state that:

---
Note: the build environment must be using a Python 3 stack, and have
the ``venv`` and ``pip`` packages installed. If necessary, make sure
``configure`` is called with ``--python=`` and that those modules are
available. On Debian and Ubuntu based systems, depending on the
specific version, they may be on packages named ``python3-venv`` and
``python3-pip``.
---

[Cleber Rosa (cleber-gnu)]

As a mitigation, I'm asking the pycdlib maintainer the possibility of also shipping wheels:

https://github.com/clalancette/pycdlib/issues/53

[Philippe Mathieu-Daudé (philmd)]

On 10/9/20 10:55 PM, Cleber Rosa wrote:
> On with certain versions of "pip", package installations will attempt
> to create wheels. And, on environments without a "complete" Python
> installation (as described in the acceptance tests requirements docs),
> that will fail.
>
> pycdlib, starting with version 1.11.0, is now being made available
> as wheels, so its instalation on those constrained environments is
> now possible.
>
> Cc: Bug 1897783 <email address hidden>
> Buglink: https://bugs.launchpad.net/qemu/+bug/1880189

This BZ is different. The correct URL is:
https://bugs.launchpad.net/qemu/+bug/1897783

Reviewed-by: Philippe Mathieu-Daudé <email address hidden>
Tested-by: Philippe Mathieu-Daudé <email address hidden>

> Reported-by: Philippe Mathieu-Daudé <email address hidden>
> Signed-off-by: Cleber Rosa <email address hidden>
> ---
> tests/requirements.txt | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> index 036691c922..a1c631fa59 100644
> --- a/tests/requirements.txt
> +++ b/tests/requirements.txt
> @@ -2,4 +2,4 @@
> # in the tests/venv Python virtual environment. For more info,
> # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> avocado-framework==81.0
> -pycdlib==1.9.0
> +pycdlib==1.11.0
>

[Willian Rampazzo (wrampazz)]

On Fri, Oct 9, 2020 at 5:55 PM Cleber Rosa <email address hidden> wrote:
>
> On with certain versions of "pip", package installations will attempt
> to create wheels. And, on environments without a "complete" Python
> installation (as described in the acceptance tests requirements docs),
> that will fail.
>
> pycdlib, starting with version 1.11.0, is now being made available
> as wheels, so its instalation on those constrained environments is
> now possible.
>
> Cc: Bug 1897783 <email address hidden>
> Buglink: https://bugs.launchpad.net/qemu/+bug/1880189
> Reported-by: Philippe Mathieu-Daudé <email address hidden>
> Signed-off-by: Cleber Rosa <email address hidden>
> ---
> tests/requirements.txt | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> index 036691c922..a1c631fa59 100644
> --- a/tests/requirements.txt
> +++ b/tests/requirements.txt
> @@ -2,4 +2,4 @@
> # in the tests/venv Python virtual environment. For more info,
> # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> avocado-framework==81.0
> -pycdlib==1.9.0
> +pycdlib==1.11.0
> --
> 2.25.4
>

Reviewed-by: Willian Rampazzo <email address hidden>

[Thomas Huth (th-huth)]

Released with QEMU v5.2.0.