gpg return 'No dirmngr' error when handling add_extra_ppas phase

[Laider Lai]

Hi,

We have an issue starting from 3.2+snap6 rev#742 when building an image with PPA.
The 3.2+snap4 rev#737 version is workable without this issue.

The building yaml at here: The working yaml: https://git.launchpad.net/~erlangen-team/erlangen/+git/iot-image-builds/tree/yaml/s32g-jammy-server-arm64.yaml

Could you help to check this issue? Tks.

Err msg:
[6] add_extra_ppas
gpg: WARNING: unsafe permissions on homedir '/home/ubuntu/workspace/iot-erlangen-classic-server-2204/iot-image-builds/work/chroot/tmp/ubuntu-image-gpg'
gpg: keybox '/home/ubuntu/workspace/iot-erlangen-classic-server-2204/iot-image-builds/work/chroot/tmp/ubuntu-image-gpg/pubring.kbx' created
gpg: error running '/usr/bin/dirmngr': exit status 1
gpg: failed to start the dirmngr '/usr/bin/dirmngr': General error
gpg: connecting dirmngr at '/home/ubuntu/workspace/iot-erlangen-classic-server-2204/iot-image-builds/work/chroot/tmp/ubuntu-image-gpg/S.dirmngr' failed: General error
gpg: keyserver receive failed: No dirmngr
duration: 503.675444ms
Error: Error retrieving signing key for ppa "erlangen-team/nxp-s32g-bsp": Error running gpg command "/snap/ubuntu-image/776/usr/bin/gpg --no-default-keyring --no-options --batch --homedir work/chroot/tmp/ubuntu-image-gpg --secret-keyring work/chroot/tmp/ubuntu-image-gpg/tempring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 880A6D439C9C8920D1101AF97591D2F24E6BCB60". Error is "exit status 2". Full output below:
gpg: WARNING: unsafe permissions on homedir '/home/ubuntu/workspace/iot-erlangen-classic-server-2204/iot-image-builds/work/chroot/tmp/ubuntu-image-gpg'
gpg: keybox '/home/ubuntu/workspace/iot-erlangen-classic-server-2204/iot-image-builds/work/chroot/tmp/ubuntu-image-gpg/pubring.kbx' created
gpg: error running '/usr/bin/dirmngr': exit status 1
gpg: failed to start the dirmngr '/usr/bin/dirmngr': General error
gpg: connecting dirmngr at '/home/ubuntu/workspace/iot-erlangen-classic-server-2204/iot-image-builds/work/chroot/tmp/ubuntu-image-gpg/S.dirmngr' failed: General error
gpg: keyserver receive failed: No dirmngr

[Paul Mars (upils)]

I was unable to reproduce for now. laiderlai will investigate more to understand why this is happening in the build server.

[Laider Lai (laiderlai)]

Hi Paul,

After a long experiment, we found the key point to reproduce this issue (100%).
The key point is the number of characters of the full path for "<working directory>/work/chroot/tmp/ubuntu-image-gpg/<pubring.kbx | trustdb.gpg"

If the total number of characters >= 100, the issue is reproduced.
Ex. /home/ubuntu/workspace/iot-murcia-classic-2204/iot-image-builds/work/chroot/tmp/ubuntu-image-gpg/pubring.kbx

If the path is shorter than 100 characters, the issue is gone.
Ex. /home/ubuntu/iot-image-builds/work/chroot/tmp/ubuntu-image-gpg

Looks like there is a string array declared with 100 length.
Could you try to reproduce this issue on your side and check the root cause? Tks.

[Paul Mars (upils)]

Since you showed versions of gpg/dirmngr did not change between the working and the buggy versions of ubuntu-image, I suspect this limitation (length of the path of the gpg temp dir) is not new in gnupg.

However, after looking into it I noticed that previously this directory was created in the /tmp folder of the building machine, so outside the chroot. When reworking the PPA handling I moved this path in the chroot to make the build process more self contained and avoid creating files/directories outside the workdir, and thus the path can be arbitrarily long depending on the workdir location. This is also safer in case several ubuntu-image builds run concurrently (but we could fix this by adding a random string in the temp dir name).

I will investigate more to understand if this was done by design in dirmngr or if we could raise this 100 char length limit.

As a temporary/half solution I can also rename the /tmp/ubuntu-image-gpg as /tmp/ui-gpg to be under 100 chars in your case. We loose a bit in readability but we do not really expect users to look into this dir anyway. We could also check the length of the path early and display an explicit error to avoid confusing users with this cryptic error.

In the end if no other solution is possible we could also revert this change and be sure it will always work.

[Paul Mars (upils)]

See https://github.com/canonical/ubuntu-image/pull/215 for the workaround implementation.

[Paul Mars (upils)]

The workaround is now merged and even available in latest/stable.

Let me know if this enough for you for now. I shall investigate further after the 24.04 release.

[Laider Lai (laiderlai)]

Thanks! We saw the gpg name is changed to "u-i-gpg".
This workaround helps to work with a shorter full path.