autopkgtests run too often, too much and don't skip enough

Bug #1823056 reported by Dimitri John Ledkov on 2019-04-03
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Undecided
Unassigned
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned
Cosmic
Undecided
Unassigned

Bug Description

[Impact]

 * linux autopkgtest should only execute either rebuild tests, when triggered by toolchain.
   or execute regression suite when triggered by meta
   but never both. As otherwise, it results in false negative results for the kernel

[Test Case]

 * Trigger adt test of linux with a matching linux-meta. Check that rebuild test is skipped, and that the regression suite test runs.

 * trigger adt test of linux with triggered by gcc-6/7/8 (as appropriate) and observe that rebuild test runs, and regression suite test is skipped.

 * (when this is applied to flavours) trigger adt test of linux-* with a matching flavour meta, and check that regression test-suite is skipped on kernels that cannot boot in scaling stack (e.g. gcp, azure, aws, etc)

[Fix]

 * debian/tests/* are modified to pay more attention as to what they are triggered by, and raise appropriate skipped error codes

[Regression Potential]

 * incorrect tests may run at incorrect time is the regression potential here, hence the two test cases verify that the right tests are executed when expected.

 * care was taken to take into account all linux kernel flavours, hence the third test case need to be reverified on flavoured kernels.

[Other Info]

 * affects all stable series xenial and up, across all kernel flavours

Seth Forshee (sforshee) on 2019-04-03
Changed in linux (Ubuntu):
status: New → Fix Committed
Launchpad Janitor (janitor) wrote :
Download full text (50.5 KiB)

This bug was fixed in the package linux - 5.0.0-11.12

---------------
linux (5.0.0-11.12) disco; urgency=medium

  * linux: 5.0.0-11.12 -proposed tracker (LP: #1824383)

  * hns3: PPU_PF_ABNORMAL_INT_ST over_8bd_no_fe found [error status=0x1]
    (LP: #1824194)
    - net: hns3: fix for not calculating tx bd num correctly

  * disco: unable to use iptables/enable ufw under -virtual kernel
    (LP: #1823862)
    - [Packaging] add bpfilter to linux-modules

  * Make shiftfs a module rather than built-in (LP: #1824354)
    - [Config] CONFIG_SHIFT_FS=m

  * shiftfs: chown sets untranslated ids in lower fs (LP: #1824350)
    - SAUCE: shiftfs: use translated ids when chaning lower fs attrs

  * [Hyper-V] KVP daemon fails to start on first boot of disco VM (LP: #1820063)
    - [Packaging] bind hv_kvp_daemon startup to hv_kvp device

linux (5.0.0-10.11) disco; urgency=medium

  * linux: 5.0.0-10.11 -proposed tracker (LP: #1823936)

  * Apparmor enforcement failure in lxc selftests (LP: #1823379)
    - SAUCE: apparmor: Restore Y/N in /sys for apparmor's "enabled"

  * systemd cause kernel trace "BUG: unable to handle kernel paging request at
    6db23a14" on Cosmic i386 (LP: #1813244)
    - openvswitch: fix flow actions reallocation

linux (5.0.0-9.10) disco; urgency=medium

  * linux: 5.0.0-9.10 -proposed tracker (LP: #1823228)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log
    - [Packaging] update helper scripts
    - [Packaging] resync retpoline extraction

  * Huawei Hi1822 NIC has poor performance (LP: #1820187)
    - net-next/hinic: replace disable_irq_nosync/enable_irq

  * Add uid shifting overlay filesystem (shiftfs) (LP: #1823186)
    - shiftfs: uid/gid shifting bind mount
    - shiftfs: rework and extend
    - shiftfs: support some btrfs ioctls
    - [Config] enable shiftfs

  * Cannot boot or install - have to use nomodeset (LP: #1821820)
    - Revert "drm/i915/fbdev: Actually configure untiled displays"

  * Disco update: v5.0.6 upstream stable release (LP: #1823060)
    - netfilter: nf_tables: fix set double-free in abort path
    - dccp: do not use ipv6 header for ipv4 flow
    - genetlink: Fix a memory leak on error path
    - gtp: change NET_UDP_TUNNEL dependency to select
    - ipv6: make ip6_create_rt_rcu return ip6_null_entry instead of NULL
    - mac8390: Fix mmio access size probe
    - mISDN: hfcpci: Test both vendor & device ID for Digium HFC4S
    - net: aquantia: fix rx checksum offload for UDP/TCP over IPv6
    - net: datagram: fix unbounded loop in __skb_try_recv_datagram()
    - net/packet: Set __GFP_NOWARN upon allocation in alloc_pg_vec
    - net: phy: meson-gxl: fix interrupt support
    - net: rose: fix a possible stack overflow
    - net: stmmac: fix memory corruption with large MTUs
    - net-sysfs: call dev_hold if kobject_init_and_add success
    - net: usb: aqc111: Extend HWID table by QNAP device
    - packets: Always register packet sk in the same order
    - rhashtable: Still do rehash when we get EEXIST
    - sctp: get sctphdr by offset in sctp_compute_cksum
    - sctp: use memdup_user instead of vmemdup_user
    - tcp: do not use ipv6 header for ipv4 flow
    - tipc: allow servic...

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Changed in linux (Ubuntu Xenial):
status: New → In Progress
Changed in linux (Ubuntu Bionic):
status: New → In Progress
Changed in linux (Ubuntu Cosmic):
status: New → In Progress
Changed in linux (Ubuntu Bionic):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Cosmic):
status: In Progress → Fix Committed

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-cosmic' to 'verification-done-cosmic'. If the problem still exists, change the tag 'verification-needed-cosmic' to 'verification-failed-cosmic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-cosmic
Dimitri John Ledkov (xnox) wrote :

Requesting to rerun linux test cases, as triggered by gcc-8 to confirm the fix.
As currently they pass rebuild test, but end up as an overall FAIL due to regression suite.
they should turn green under:
http://people.canonical.com/~ubuntu-archive/proposed-migration/cosmic/update_excuses.html#gcc-8

Or look for gree pass, as triggered by gcc-8 on https://autopkgtest.ubuntu.com/packages/linux/cosmic/s390x

Dimitri John Ledkov (xnox) wrote :

https://people.canonical.com/~ubuntu-archive/proposed-migration/cosmic/update_excuses.html#gcc-8 all is now green, and checking the results one can see that rebuild test passed, and the regression test suite was skipped, correctly.

Great stuff.

tags: added: verification-done-cosmic
removed: verification-needed-cosmic

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
Dimitri John Ledkov (xnox) wrote :

Triggering for example linux adt test in bionic, as triggered by openssl to verify the fix.

Dimitri John Ledkov (xnox) wrote :
tags: added: verification-done-bionic
removed: verification-needed-bionic
Changed in linux (Ubuntu Xenial):
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :
Download full text (12.6 KiB)

This bug was fixed in the package linux - 4.15.0-50.54

---------------
linux (4.15.0-50.54) bionic; urgency=medium

  * CVE-2018-12126 // CVE-2018-12127 // CVE-2018-12130
    - Documentation/l1tf: Fix small spelling typo
    - x86/cpu: Sanitize FAM6_ATOM naming
    - kvm: x86: Report STIBP on GET_SUPPORTED_CPUID
    - locking/atomics, asm-generic: Move some macros from <linux/bitops.h> to a
      new <linux/bits.h> file
    - tools include: Adopt linux/bits.h
    - x86/msr-index: Cleanup bit defines
    - x86/speculation: Consolidate CPU whitelists
    - x86/speculation/mds: Add basic bug infrastructure for MDS
    - x86/speculation/mds: Add BUG_MSBDS_ONLY
    - x86/kvm: Expose X86_FEATURE_MD_CLEAR to guests
    - x86/speculation/mds: Add mds_clear_cpu_buffers()
    - x86/speculation/mds: Clear CPU buffers on exit to user
    - x86/kvm/vmx: Add MDS protection when L1D Flush is not active
    - x86/speculation/mds: Conditionally clear CPU buffers on idle entry
    - x86/speculation/mds: Add mitigation control for MDS
    - x86/speculation/mds: Add sysfs reporting for MDS
    - x86/speculation/mds: Add mitigation mode VMWERV
    - Documentation: Move L1TF to separate directory
    - Documentation: Add MDS vulnerability documentation
    - x86/speculation/mds: Add mds=full,nosmt cmdline option
    - x86/speculation: Move arch_smt_update() call to after mitigation decisions
    - x86/speculation/mds: Add SMT warning message
    - x86/speculation/mds: Fix comment
    - x86/speculation/mds: Print SMT vulnerable on MSBDS with mitigations off
    - x86/speculation/mds: Add 'mitigations=' support for MDS

  * CVE-2017-5715 // CVE-2017-5753
    - s390/speculation: Support 'mitigations=' cmdline option

  * CVE-2017-5715 // CVE-2017-5753 // CVE-2017-5754 // CVE-2018-3639
    - powerpc/speculation: Support 'mitigations=' cmdline option

  * CVE-2017-5715 // CVE-2017-5754 // CVE-2018-3620 // CVE-2018-3639 //
    CVE-2018-3646
    - cpu/speculation: Add 'mitigations=' cmdline option
    - x86/speculation: Support 'mitigations=' cmdline option

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log

linux (4.15.0-49.53) bionic; urgency=medium

  * linux: 4.15.0-49.53 -proposed tracker (LP: #1826358)

  * Backport support for software count cache flush Spectre v2 mitigation. (CVE)
    (required for POWER9 DD2.3) (LP: #1822870)
    - powerpc/64s: Add support for ori barrier_nospec patching
    - powerpc/64s: Patch barrier_nospec in modules
    - powerpc/64s: Enable barrier_nospec based on firmware settings
    - powerpc: Use barrier_nospec in copy_from_user()
    - powerpc/64: Use barrier_nospec in syscall entry
    - powerpc/64s: Enhance the information in cpu_show_spectre_v1()
    - powerpc/64: Disable the speculation barrier from the command line
    - powerpc/64: Make stf barrier PPC_BOOK3S_64 specific.
    - powerpc/64: Add CONFIG_PPC_BARRIER_NOSPEC
    - powerpc/64: Call setup_barrier_nospec() from setup_arch()
    - powerpc/64: Make meltdown reporting Book3S 64 specific
    - powerpc/lib/code-patching: refactor patch_instruction()
    - powerpc/lib/feature-fixups: use raw_patch_instruction()
    - powerpc/asm: Add a patch_site mac...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :
Download full text (7.0 KiB)

This bug was fixed in the package linux - 4.18.0-20.21

---------------
linux (4.18.0-20.21) cosmic; urgency=medium

  * CVE-2018-12126 // CVE-2018-12127 // CVE-2018-12130
    - Documentation/l1tf: Fix small spelling typo
    - x86/cpu: Sanitize FAM6_ATOM naming
    - kvm: x86: Report STIBP on GET_SUPPORTED_CPUID
    - locking/atomics, asm-generic: Move some macros from <linux/bitops.h> to a
      new <linux/bits.h> file
    - tools include: Adopt linux/bits.h
    - x86/msr-index: Cleanup bit defines
    - x86/speculation: Consolidate CPU whitelists
    - x86/speculation/mds: Add basic bug infrastructure for MDS
    - x86/speculation/mds: Add BUG_MSBDS_ONLY
    - x86/kvm: Expose X86_FEATURE_MD_CLEAR to guests
    - x86/speculation/mds: Add mds_clear_cpu_buffers()
    - x86/speculation/mds: Clear CPU buffers on exit to user
    - x86/kvm/vmx: Add MDS protection when L1D Flush is not active
    - x86/speculation/mds: Conditionally clear CPU buffers on idle entry
    - x86/speculation/mds: Add mitigation control for MDS
    - x86/speculation/mds: Add sysfs reporting for MDS
    - x86/speculation/mds: Add mitigation mode VMWERV
    - Documentation: Move L1TF to separate directory
    - Documentation: Add MDS vulnerability documentation
    - x86/speculation/mds: Add mds=full,nosmt cmdline option
    - x86/speculation: Move arch_smt_update() call to after mitigation decisions
    - x86/speculation/mds: Add SMT warning message
    - x86/speculation/mds: Fix comment
    - x86/speculation/mds: Print SMT vulnerable on MSBDS with mitigations off
    - x86/speculation/mds: Add 'mitigations=' support for MDS

  * CVE-2017-5715 // CVE-2017-5753
    - s390/speculation: Support 'mitigations=' cmdline option

  * CVE-2017-5715 // CVE-2017-5753 // CVE-2017-5754 // CVE-2018-3639
    - powerpc/speculation: Support 'mitigations=' cmdline option

  * CVE-2017-5715 // CVE-2017-5754 // CVE-2018-3620 // CVE-2018-3639 //
    CVE-2018-3646
    - cpu/speculation: Add 'mitigations=' cmdline option
    - x86/speculation: Support 'mitigations=' cmdline option

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log

linux (4.18.0-19.20) cosmic; urgency=medium

  * linux: 4.18.0-19.20 -proposed tracker (LP: #1826171)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log

  * autopkgtests run too often, too much and don't skip enough (LP: #1823056)
    - [Debian] Set +x on rebuild testcase.
    - [Debian] Skip rebuild test, for regression-suite deps.
    - [Debian] Make ubuntu-regression-suite skippable on unbootable kernels.
    - [Debian] make rebuild use skippable error codes when skipping.
    - [Debian] Only run regression-suite, if requested to.

  * CVE-2017-5753
    - s390/keyboard: sanitize array index in do_kdsk_ioctl
    - drm/bufs: Fix Spectre v1 vulnerability
    - drivers/misc/sgi-gru: fix Spectre v1 vulnerability
    - ipv4: Fix potential Spectre v1 vulnerability
    - aio: fix spectre gadget in lookup_ioctx
    - ALSA: emux: Fix potential Spectre v1 vulnerabilities
    - ALSA: pcm: Fix potential Spectre v1 vulnerability
    - ip6mr: Fix potential Spectre v1 vulnerability
    - ALSA: rme9652: Fix potential Spectre v1...

Read more...

Changed in linux (Ubuntu Cosmic):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for linux-aws has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-xenial

Confirmed that with the fix on xenial a autopkg test triggered by a package other then linux-meta triggers only the rebuild test and not the regression tests:

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/l/linux/20190531_164624_fe9c2@/log.gz

tags: added: verification-done-xenial
removed: verification-needed-xenial
Launchpad Janitor (janitor) wrote :
Download full text (19.2 KiB)

This bug was fixed in the package linux - 4.4.0-150.176

---------------
linux (4.4.0-150.176) xenial; urgency=medium

  * linux: 4.4.0-150.176 -proposed tracker (LP: #1830941)

  * glibc 2.23-0ubuntu11 ADT test failure with linux 4.4.0-149.175
    (LP: #1830890)
    - x86/vdso: Pass --eh-frame-hdr to the linker

linux (4.4.0-149.175) xenial; urgency=medium

  * linux: 4.4.0-149.175 -proposed tracker (LP: #1829209)

  * disable a.out support (LP: #1818552)
    - [Config] Disable a.out support

  * autopkgtests run too often, too much and don't skip enough (LP: #1823056)
    - [Debian] Set +x on rebuild testcase.
    - [Debian] Skip rebuild test, for regression-suite deps.
    - [Debian] Make ubuntu-regression-suite skippable on unbootable kernels.
    - [Debian] make rebuild use skippable error codes when skipping.
    - [Debian] Only run regression-suite, if requested to.

  * Xenial update: 4.4.179 upstream stable release (LP: #1828420)
    - arm64: debug: Don't propagate UNKNOWN FAR into si_code for debug signals
    - arm64: debug: Ensure debug handlers check triggering exception level
    - ext4: cleanup bh release code in ext4_ind_remove_space()
    - lib/int_sqrt: optimize initial value compute
    - tty/serial: atmel: Add is_half_duplex helper
    - mm: mempolicy: make mbind() return -EIO when MPOL_MF_STRICT is specified
    - i2c: core-smbus: prevent stack corruption on read I2C_BLOCK_DATA
    - Bluetooth: Fix decrementing reference count twice in releasing socket
    - tty/serial: atmel: RS485 HD w/DMA: enable RX after TX is stopped
    - CIFS: fix POSIX lock leak and invalid ptr deref
    - h8300: use cc-cross-prefix instead of hardcoding h8300-unknown-linux-
    - tracing: kdb: Fix ftdump to not sleep
    - gpio: gpio-omap: fix level interrupt idling
    - sysctl: handle overflow for file-max
    - enic: fix build warning without CONFIG_CPUMASK_OFFSTACK
    - mm/cma.c: cma_declare_contiguous: correct err handling
    - mm/page_ext.c: fix an imbalance with kmemleak
    - mm/vmalloc.c: fix kernel BUG at mm/vmalloc.c:512!
    - mm/slab.c: kmemleak no scan alien caches
    - ocfs2: fix a panic problem caused by o2cb_ctl
    - f2fs: do not use mutex lock in atomic context
    - fs/file.c: initialize init_files.resize_wait
    - cifs: use correct format characters
    - dm thin: add sanity checks to thin-pool and external snapshot creation
    - cifs: Fix NULL pointer dereference of devname
    - fs: fix guard_bio_eod to check for real EOD errors
    - tools lib traceevent: Fix buffer overflow in arg_eval
    - usb: chipidea: Grab the (legacy) USB PHY by phandle first
    - scsi: core: replace GFP_ATOMIC with GFP_KERNEL in scsi_scan.c
    - coresight: etm4x: Add support to enable ETMv4.2
    - ARM: 8840/1: use a raw_spinlock_t in unwind
    - mmc: omap: fix the maximum timeout setting
    - e1000e: Fix -Wformat-truncation warnings
    - IB/mlx4: Increase the timeout for CM cache
    - scsi: megaraid_sas: return error when create DMA pool failed
    - perf test: Fix failure of 'evsel-tp-sched' test on s390
    - SoC: imx-sgtl5000: add missing put_device()
    - media: sh_veu: Correct return type for mem2mem buffer helpers
    - media: s5...

Changed in linux (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers