Comment 6 for bug 1844504

On Tue, Sep 24, 2019 at 03:00:16PM -0000, Mauricio Faria de Oliveira wrote:
> The same upload on Xenial passed on Sep 3rd and failed on Sep 17th,
> due to openpty() failing in tests of util-linux/script.

We switched over to the new xenial chroots on 2019-09-05.

> The strace of the failing test-case shows no permission on /dev/ptmx:
>
> open("/dev/ptmx", O_RDWR) = -1 EACCES (Permission
> denied)
>
> That's because /dev/ptmx is a symlink to /dev/pts/ptmx, which has
> no permissions for non-root users on the livecd-rootfs chroot [1],
> used with sbuild-launchpad-chroot locally.

Grepping for dev/pt in the old (pre-2019-09-05) xenial chroots:

  drwxr-xr-x root/root 0 2011-11-14 17:30 chroot-autobuild/dev/pts/
  crw-rw-rw- root/tty 5,2 2011-05-23 11:44 chroot-autobuild/dev/ptmx

In the ones used from 2019-09-05:

  lrwxrwxrwx 0/0 0 2019-09-05 08:13 chroot-autobuild/dev/ptmx -> pts/ptmx
  drwxr-xr-x 0/0 0 2019-09-05 08:13 chroot-autobuild/dev/pts/

In a chroot built with my changes:

  crw-rw-rw- 0/0 5,2 2019-09-19 12:30 chroot-autobuild/dev/ptmx
  drwxr-xr-x 0/0 0 2019-09-19 12:30 chroot-autobuild/dev/pts/

(That said, I was building my test on bionic, so the different
debootstrap version could affect things.)

> I'm not exactly sure how makedev would affect it specifically
> (maybe I didnt find/run the right hooks that gate on it, possibly?)
>
> You mentioned you're 'fairly sure' about this, so would sincerely
> appreciate a comment about it, if you have the time/chance. :- )

It was an educated guess given that makedev obviously does stuff with
device nodes and is a glaring difference between old and new chroots,
but I'm certainly prepared to believe that it might be wrong. If things
still don't work after it's got all the way through the normal process
then I'll debug some more.

> Other than that, for further debugging on that other problem..
>
> Is there a way to find the specific chroot_url [1] used in a PPA for a particular build?
> (buildlog only mentions 'in-target unpack-chroot' for a /home/buildd/file-cache-default/<hash>)

I don't think so, sorry.

> Otherwise, is there a way to find out when/on which release the
> livecd-rootfs image was built?

You'd have to hunt it down from
https://launchpad.net/~cloud-images/+livefs/ubuntu/xenial/cpc-buildd
(possibly via the API, as the web UI there doesn't show older builds).
We don't have great tools for this yet.

At the moment my ad-hoc workaround is to mention chroot changes in
#launchpad-ops on irc.canonical.com, so they can be tracked down via
correlation with logs there. I know this needs improvement.

> Specifically looking for the deboostrap version used, as there's an
> (old) change to deboostrap that changed /dev/ptmx from static node
> to this symlink, but it's been a while ago, so not very sure of it.
> (Debian bugs 817236 for discussion/issue, and 571136 for change)

It'll be debootstrap 1.0.78+nmu1ubuntu1.9, since that's what's in
xenial-updates and has been for some months.

> Steps used to build livecd.ubuntu-base.rootfs.tar.gz
> (according to python-lpbuildd's build_livefs.py,
> then patched sbuild-launchpad-chroot to use that.)

What release were you building the rootfs on? Was makedev present in
the resulting rootfs? What did /dev/pt* look like in it?