apt does not download dep11 files for foreign architectures and appstream cannot find applications for these archs.

[Alan Pope 🍺🐧🐱 🦄]

I noticed the following community member had a problem installing steam. I reproduced it on 16.04.3 clean install i386 and amd64.

https://www.youtube.com/watch?v=24NH3Ry22Dg

Install 16.04 amd64 or i386
Install all updates via software updater
Open Ubuntu Software
Search for steam.

No results for Steam found.
Yet steam is in the repository as an i386 deb - but it doesn't show up in Ubuntu Software on either amd64 or i386 installs.

Also discussed at https://ubuntu.labix.org/t/ubuntu-software-deb-install-issues/81

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: gnome-software 3.20.5-0ubuntu0.16.04.5
ProcVersionSignature: Ubuntu 4.10.0-35.39~16.04.1-generic 4.10.17
Uname: Linux 4.10.0-35-generic i686
ApportVersion: 2.20.1-0ubuntu2.10
Architecture: i386
CurrentDesktop: Unity
Date: Wed Sep 20 15:46:02 2017
InstallationDate: Installed on 2017-09-20 (0 days ago)
InstallationMedia: Ubuntu 16.04.3 LTS "Xenial Xerus" - Release i386 (20170801)
SourcePackage: gnome-software
UpgradeStatus: No upgrade log present (probably fresh install)

[Robert Ancell (robert-ancell)]

Linking in appstream as the reason is there is no appstream data:

$ appstreamcli search steam
Identifier: ri-li.desktop [desktop-application]
Name: Ri-li
Summary: a toy simulator game
Package: ri-li
Icon: ri-li_ri-li.png
----

[Matthias Klumpp (ximion)]

It is there though, on i386: http://appstream.ubuntu.com/artful/multiverse/metainfo/steam.html
Maybe the multiverse sources aren't active here?

[Robert Ancell (robert-ancell)]

Looking in /var/lib/apt/lists I have:

archive.ubuntu.com_ubuntu_dists_xenial_multiverse_binary-amd64_Packages
archive.ubuntu.com_ubuntu_dists_xenial_multiverse_binary-i386_Packages
archive.ubuntu.com_ubuntu_dists_xenial_multiverse_dep11_Components-amd64.yml.gz
archive.ubuntu.com_ubuntu_dists_xenial_multiverse_dep11_icons-64x64.tar.gz
archive.ubuntu.com_ubuntu_dists_xenial_multiverse_i18n_Translation-en
archive.ubuntu.com_ubuntu_dists_xenial_multiverse_source_Sources

So it appears the i386 dep11 data is not there. Perhaps it is an APT issue that is not pulling it down?

[Robert Ancell (robert-ancell)]

Explicitly, it looks like I am missing:
archive.ubuntu.com_ubuntu_dists_xenial_multiverse_dep11_Components-i386.yml.gz

[Matthias Klumpp (ximion)]

Are you on a real i386 system, or on an amd64 system with i386 as foreign architecture? The latter case is not supported, which is a known issue (affects Steam and Skype).

[Jeremy Bícha (jbicha)]

This bug is likely a duplicate of https://launchpad.net/bugs/1564570

[Jeremy Bícha (jbicha)]

Oh, I missed Robert's comments. #1564570 is for non-i386 architectures.

[Alan Pope 🍺🐧🐱 🦄 (popey)]

I tested on both amd64 and i386 as detailed in the linked post in the original description. I installed both amd64 and i386 variants of Ubuntu 16.04.3 in parallel in separate VMs and experienced (from a user perspective) the same issue. The difference being what appstreamcli returns on each...

On amd64:-

alan@cleanamd64:~$ appstreamcli search steam
Identifier: ri-li.desktop [desktop-application]
Name: Ri-li
Summary: a toy simulator game
Package: ri-li
Icon: ri-li_ri-li.png
----

On i386:-

alan@cleani386:~$ appstreamcli search steam
Identifier: steam.desktop [desktop-application]
Name: Steam
Summary: Application for managing and playing games on Steam
Package: steam
----
Identifier: ri-li.desktop [desktop-application]
Name: Ri-li
Summary: a toy simulator game
Package: ri-li
Icon: ri-li_ri-li.png
----
alan@cleani386:~$

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[DANIEL CARON (carondaniel1957)]

After trying to install it gives me:
Running Steam on ubuntu 16.04 64-bit
STEAM_RUNTIME is enabled automatically
Installing breakpad exception handler for appid(steam)/version(0)
libGL error: unable to load driver: r600_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: r600
libGL error: unable to load driver: r600_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: r600
libGL error: unable to load driver: swrast_dri.so
libGL error: failed to load driver: swrast

in terminal

[Jean-Baptiste Lallement (jibel)]

I'm removing gnome-software task because it is not a problem with this component.

[Jean-Baptiste Lallement (jibel)]

There is no component files for i386 on amd64 in /var/lib/apt/lists/
If you download the corresponding dep11 files and refresh appstream cache then a search returns the right results.

Adding a task for apt which is responsible for downloading these files.

[David Kalnischkies (donkult)]

apt does what it is told – appstream configures apt to download only the files for the native architecture, so there is no sensible action to be taken by apt and hence this task invalid.

If "$(NATIVE_ARCHITECTURE)" in the apt.conf file shipped by appstream is changed to "$(ARCHITECTURE)" apt will download the files for all configured architectures – if that is really desired is what appstream developers have to figure out. I will add that it might be also a good idea to add support for Components-all first to avoid at least a bit of duplication (yes, apt supports downloading those files, too, it just wont by default – but that default can be switched via Release file metadata).

[Matthias Klumpp (ximion)]

@donkult: The downloads for all architectures are not allowed at time, because doing so would lead to AppStream-ID collisions, and AppStream itself has no notion of architectures (yet - I played around with that a bit, but there is no definitive good solution yet).

For downloading Components-all, if we would generate such a file, could APT be configured to download that one unconditionally, no matter what the Releases file says? Because in that case, we could do quite a few optimizations to save disk space.

[Matthias Klumpp (ximion)]

At this point, I think it's safe to say that implementing full multiarch support for AppStream simply isn't going to happen, at least not anytime soon. It's simply easier for app authors to provide native implementations for amd64 instead of relying on i386 compatibility.
If support was implemented fully, we would download lots of additional big metadata and run into all kinds of troubles of which app gets selected when there are ID conflicts.
Resolving all of these issues is possible, but the huge effort IMHO doesn't justify the gains. If someone else works on this, I'd assist the work, but I will unlikely work on this myself. I originally intended to, but TBH, this issue has really low priority nowadays.

There is a workaround for high-profile packages that are only available on one architecture: *If* i386 support is enabled by default, the appstream-generator can be instructed to inject metadata for a specific package directly into the final metadata collection. See https://github.com/ximion/appstream-generator/blob/master/docs/asgen-config.md#injecting-extra-metainfo--removing-components for details. This is intended to be used for adding web applications, but may work for the multiarch case as well, at least for a select few of high-profile application packages.