The following error shows as passed: ERROR: testbed failure: unexpected eof from the testbed

[Mitchell Dzurick]

I have the logs at https://autopkgtest.ubuntu.com/results/autopkgtest-jammy-mitchdz-rabbitmq-server-mre-2204/jammy/armhf/r/rabbitmq-server/20240513_125051_441be@/log.gz

And the results from ppa-dev-tools looks like:

  -
    + ✅ rabbitmq-server on jammy for armhf @ 13.05.24 12:50:51
      • Log: https://autopkgtest.ubuntu.com/results/autopkgtest-jammy-mitchdz-rabbitmq-server-mre-2204/jammy/armhf/r/rabbitmq-server/20240513_125051_441be@/log.gz

I plan to rerun the test, so I will paste the contents in this bug description since it is so short:

  0s autopkgtest [12:50:43]: starting date and time: 2024-05-13 12:50:43+0000
  0s autopkgtest [12:50:43]: git checkout: 43bc6cdf gitlab-ci: do not include the salsa pipeline
  0s autopkgtest [12:50:43]: host juju-7f2275-prod-proposed-migration-environment-4; command line: /home/ubuntu/autopkgtest/runner/autopkgtest --output-dir /tmp/autopkgtest-work.okrgyofl/out --timeout-copy=6000 --setup-commands 'ln -s /dev/null /etc/systemd/system/bluetooth.service; printf "http_proxy=http://squid.internal:3128\nhttps_proxy=http://squid.internal:3128\nno_proxy=127.0.0.1,127.0.1.1,login.ubuntu.com,localhost,localdomain,novalocal,internal,archive.ubuntu.com,ports.ubuntu.com,security.ubuntu.com,ddebs.ubuntu.com,changelogs.ubuntu.com,keyserver.ubuntu.com,launchpadlibrarian.net,launchpadcontent.net,launchpad.net,10.24.0.0/24,keystone.ps5.canonical.com,objectstorage.prodstack5.canonical.com\n" >> /etc/environment' --setup-commands 'wget -O- '"'"'http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&v=1&search=0x0F1FB299A1ECD860B7376336C9FD332D2F4EBC9C'"'"' | gpg --dearmor > '"'"'/etc/apt/trusted.gpg.d/autopkgtest-mitchdz-rabbitmq-server-mre-2204.gpg'"'"'' --setup-commands 'REL=$(lsb_release -cs); echo "deb http://ppa.launchpad.net/mitchdz/rabbitmq-server-mre-2204/ubuntu $REL main" > /etc/apt/sources.list.d/autopkgtest-mitchdz-rabbitmq-server-mre-2204.list; echo "deb-src http://ppa.launchpad.net/mitchdz/rabbitmq-server-mre-2204/ubuntu $REL main" >> /etc/apt/sources.list.d/autopkgtest-mitchdz-rabbitmq-server-mre-2204.list;' --apt-upgrade rabbitmq-server --timeout-short=300 --timeout-copy=20000 --timeout-build=20000 '--env=ADT_TEST_TRIGGERS=rabbitmq-server/3.9.27-0ubuntu0.1~jammy8' -- lxd -r lxd-armhf-10.145.243.202 lxd-armhf-10.145.243.202:autopkgtest/ubuntu/jammy/armhf
  8s <VirtSubproc>: failure: ['lxc', 'launch', '--ephemeral', 'lxd-armhf-10.145.243.202:autopkgtest/ubuntu/jammy/armhf', 'lxd-armhf-10.145.243.202:autopkgtest-lxd-jnivnc'] failed (exit status 1, stderr 'Error: Failed pre-start check for device "eth0": Network "lxdbr0" unavailable on this server\n')
  8s autopkgtest [12:50:51]: ERROR: testbed failure: unexpected eof from the testbed

[Bryce Harrington (bryce)]

$ ppa tests https://launchpad.net/~mitchdz/+archive/ubuntu/rabbitmq-server-mre-2204/
[...]
  - rabbitmq-server/3.9.27-0ubuntu0.1~jammy8
    + ✅ rabbitmq-server on jammy for arm64 @ 13.05.24 19:10:51 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for i386 @ 13.05.24 19:16:38 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for s390x @ 15.05.24 01:04:39 Log️ 🗒️
  -
    + ✅ rabbitmq-server on jammy for armhf @ 13.05.24 12:50:51 Log️ 🗒️

It looks like the failed test results have a blank trigger for some reason.
I suspect the reason this result got a green check is because the text in the log didn't match a recognized test 'fail'. Looking in further...

[Bryce Harrington (bryce)]

ppa-dev-tools looks for the '@@@@@@@@@' separator to identify the header for the test log, however when the testbed setup fails, there is no separator to detect. That's why the blank trigger occurs; ppa-dev-tools didn't recognize a header so didn't try to parse out the trigger.

A second problem is that when reviewing the subtest status it did not recognize BAD as a broken state, so lists the overall test result as passed.

With these issues fixed, the output looks more like you'd expect:

* Results:
  - rabbitmq-server/3.8.3-0ubuntu0.1~focal5
    + ⛔ rabbitmq-server on focal for arm64 @ 12.05.24 03:07:20 Log️ 🗒️
      • testbed BAD ⛔
    + ⛔ rabbitmq-server on focal for armhf @ 11.05.24 08:01:13 Log️ 🗒️
      • testbed BAD ⛔
    + ✅ rabbitmq-server on focal for i386 @ 11.05.24 13:35:11 Log️ 🗒️
    + ✅ rabbitmq-server on focal for ppc64el @ 14.05.24 23:10:14 Log️ 🗒️
    + ✅ rabbitmq-server on focal for s390x @ 14.05.24 03:20:42 Log️ 🗒️
  - rabbitmq-server/3.9.27-0ubuntu0.1~jammy4
    + ✅ rabbitmq-server on jammy for amd64 @ 06.05.24 23:39:20 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for arm64 @ 06.05.24 23:21:35 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for armhf @ 06.05.24 23:22:57 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for i386 @ 06.05.24 23:39:34 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for ppc64el @ 06.05.24 23:23:10 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for s390x @ 06.05.24 23:23:28 Log️ 🗒️
  - rabbitmq-server/3.9.27-0ubuntu0.1~jammy8
    + ✅ rabbitmq-server on jammy for arm64 @ 13.05.24 19:10:51 Log️ 🗒️
    + ⛔ rabbitmq-server on jammy for armhf @ 13.05.24 12:50:51 Log️ 🗒️
      • testbed BAD ⛔
    + ✅ rabbitmq-server on jammy for i386 @ 13.05.24 19:16:38 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for s390x @ 15.05.24 01:04:39 Log️ 🗒️

[Bryce Harrington (bryce)]

Launchpad seems buggy unfortunately. I tried pushing up my branch to LP but it gave me an error message. I've filed LP: #2065766.

I'll try again later.

[Mitchell Dzurick (mitchdz)]

yeah Launchpad seems unhappy right now :(

[Bryce Harrington (bryce)]

I've updated the MP with test cases for this change. To run them, do:

    $ make check

I built the snap and tried to reproduce Mitchell's finding but the snap seems to show the fixed behavior okay for me:

stirling: ~/src/PpaDevTools/ppa-dev-tools$ /snap/bin/ppa-dev-tools.ppa tests https://launchpad.net/~mitchdz/+archive/ubuntu/rabbitmq-server-mre-2204/
The authorization page:
 (https://launchpad.net/+authorize-token?oauth_token=SWQnVhtZWkLF7QPVhLBz&allow_permission=DESKTOP_INTEGRATION)
should be opening in your browser. Use your browser to authorize
this program to access Launchpad on your behalf.
Waiting to hear from Launchpad about your decision...
* Triggers:
  - Source rabbitmq-server/3.9.27-0ubuntu0.1~jammy8: Published
    + Trigger basic @amd64♻️ Trigger all-proposed @amd64💍
    + Trigger basic @arm64♻️ Trigger all-proposed @arm64💍
    + Trigger basic @armhf♻️ Trigger all-proposed @armhf💍
    + Trigger basic @i386♻️ Trigger all-proposed @i386💍
    + Trigger basic @ppc64el♻️ Trigger all-proposed @ppc64el💍
    + Trigger basic @s390x♻️ Trigger all-proposed @s390x💍
  - Source rabbitmq-server/3.8.3-0ubuntu0.1~focal5: Published
    + Trigger basic @amd64♻️ Trigger all-proposed @amd64💍
    + Trigger basic @arm64♻️ Trigger all-proposed @arm64💍
    + Trigger basic @armhf♻️ Trigger all-proposed @armhf💍
    + Trigger basic @i386♻️ Trigger all-proposed @i386💍
    + Trigger basic @ppc64el♻️ Trigger all-proposed @ppc64el💍
    + Trigger basic @s390x♻️ Trigger all-proposed @s390x💍
* Results:
  - rabbitmq-server/3.8.3-0ubuntu0.1~focal5
    + ⛔ rabbitmq-server on focal for amd64 @ 16.05.24 01:43:46 Log️ 🗒️
      • testbed BAD ⛔
    + ✅ rabbitmq-server on focal for amd64 @ 21.05.24 14:50:15 Log️ 🗒️
    + ⛔ rabbitmq-server on focal for arm64 @ 12.05.24 03:07:20 Log️ 🗒️
      • testbed BAD ⛔
    + ✅ rabbitmq-server on focal for arm64 @ 21.05.24 14:50:48 Log️ 🗒️
    + ⛔ rabbitmq-server on focal for armhf @ 11.05.24 08:01:13 Log️ 🗒️
      • testbed BAD ⛔
    + ✅ rabbitmq-server on focal for armhf @ 21.05.24 14:53:34 Log️ 🗒️
    + ✅ rabbitmq-server on focal for i386 @ 11.05.24 13:35:11 Log️ 🗒️
    + ✅ rabbitmq-server on focal for ppc64el @ 14.05.24 23:10:14 Log️ 🗒️
    + ✅ rabbitmq-server on focal for s390x @ 14.05.24 03:20:42 Log️ 🗒️
  - rabbitmq-server/3.9.27-0ubuntu0.1~jammy4
    + ✅ rabbitmq-server on jammy for amd64 @ 06.05.24 23:39:20 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for arm64 @ 06.05.24 23:21:35 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for armhf @ 06.05.24 23:22:57 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for i386 @ 06.05.24 23:39:34 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for ppc64el @ 06.05.24 23:23:10 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for s390x @ 06.05.24 23:23:28 Log️ 🗒️
  - rabbitmq-server/3.9.27-0ubuntu0.1~jammy8
    + ⛔ rabbitmq-server on jammy for amd64 @ 17.05.24 10:35:04 Log️ 🗒️
      • testbed BAD ⛔
    + ❌ rabbitmq-server on jammy for amd64 @ 17.05.24 20:50:40 Log️ 🗒️
      • smoke-test PASS 🟩
      • hello-world ...

[Bryce Harrington (bryce)]

Athos, I've landed the branch but would still appreciate your once-over on the tests in case I missed anything or you spot improvements to be made.

$ git push origin
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To git+ssh://git.launchpad.net/ppa-dev-tools
   dbb03ea..074f622 main -> main

[Mitchell Dzurick (mitchdz)]

Yeah this is strange, not sure why my system is not matching yours.

$ snap list ppa-dev-tools
Name Version Rev Tracking Publisher Notes
ppa-dev-tools v0.5.0+git43.074f622 x1 - - -
$ /snap/bin/ppa-dev-tools.ppa tests https://launchpad.net/\~mitchdz/+archive/ubuntu/rabbitmq-server-mre-2204/
The authorization page:
 (https://launchpad.net/+authorize-token?oauth_token=bt14mzjgJMW0N3m9G9FR&allow_permission=DESKTOP_INTEGRATION)
should be opening in your browser. Use your browser to authorize
this program to access Launchpad on your behalf.
Waiting to hear from Launchpad about your decision...
* Triggers:
  - Source rabbitmq-server/3.9.27-0ubuntu0.1~jammy8: Published
    + Trigger basic @amd64♻️ Trigger all-proposed @amd64💍
    + Trigger basic @arm64♻️ Trigger all-proposed @arm64💍
    + Trigger basic @armhf♻️ Trigger all-proposed @armhf💍
    + Trigger basic @i386♻️ Trigger all-proposed @i386💍
    + Trigger basic @ppc64el♻️ Trigger all-proposed @ppc64el💍
    + Trigger basic @s390x♻️ Trigger all-proposed @s390x💍
  - Source rabbitmq-server/3.8.3-0ubuntu0.1~focal5: Published
    + Trigger basic @amd64♻️ Trigger all-proposed @amd64💍
    + Trigger basic @arm64♻️ Trigger all-proposed @arm64💍
    + Trigger basic @armhf♻️ Trigger all-proposed @armhf💍
    + Trigger basic @i386♻️ Trigger all-proposed @i386💍
    + Trigger basic @ppc64el♻️ Trigger all-proposed @ppc64el💍
    + Trigger basic @s390x♻️ Trigger all-proposed @s390x💍
* Results:
  -
    + ✅ rabbitmq-server on focal for amd64 @ 16.05.24 01:43:46 Log️ 🗒️
    + ✅ rabbitmq-server on focal for arm64 @ 12.05.24 03:07:20 Log️ 🗒️
    + ✅ rabbitmq-server on focal for armhf @ 11.05.24 08:01:13 Log️ 🗒️
  - rabbitmq-server/3.8.3-0ubuntu0.1~focal5
    + ✅ rabbitmq-server on focal for amd64 @ 21.05.24 14:50:15 Log️ 🗒️
    + ✅ rabbitmq-server on focal for arm64 @ 21.05.24 14:50:48 Log️ 🗒️
    + ✅ rabbitmq-server on focal for armhf @ 21.05.24 14:53:34 Log️ 🗒️
    + ✅ rabbitmq-server on focal for i386 @ 11.05.24 13:35:11 Log️ 🗒️
    + ✅ rabbitmq-server on focal for ppc64el @ 14.05.24 23:10:14 Log️ 🗒️
    + ✅ rabbitmq-server on focal for s390x @ 14.05.24 03:20:42 Log️ 🗒️
  - rabbitmq-server/3.9.27-0ubuntu0.1~jammy4
    + ✅ rabbitmq-server on jammy for amd64 @ 06.05.24 23:39:20 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for arm64 @ 06.05.24 23:21:35 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for armhf @ 06.05.24 23:22:57 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for i386 @ 06.05.24 23:39:34 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for ppc64el @ 06.05.24 23:23:10 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for s390x @ 06.05.24 23:23:28 Log️ 🗒️
  -
    + ✅ rabbitmq-server on jammy for amd64 @ 17.05.24 10:35:04 Log️ 🗒️
    + ✅ rabbitmq-server on jammy for armhf @ 13.05.24 12:50:51 Log️ 🗒️
  - rabbitmq-server/3.9.27-0ubuntu0.1~jammy8
    + ❌ rabbitmq-server on jammy for amd64 @ 17.05.24 20:50:40 Log️ 🗒️
      • smoke-test PASS 🟩
      • hello-world PASS 🟩
      • work-queue PASS 🟩
      • publish-subscribe PASS 🟩
      • rpc ...

[Bryce Harrington (bryce)]

Yes, that is weird. 074f622 is the right commit, so presumably the snap is getting built correctly, but the behavior definitely matches the unfixed version.

Is it possible you have another copy of ppa-dev-tools installed somewhere? I've seen some situations where having the ppa, and the snap, and/or pip co-installed can cause confusion, with the cli script from one install loading the python modules from another.

[Bryce Harrington (bryce)]

Fix Released in 0.6.0: https://launchpad.net/ppa-dev-tools/trunk/0.6