openscap 1.3.4 rebase+merge from sid

[Alexander Scheel]

In the interest of long-term maintainability ahead of Ubuntu 22.04 release, the Security team would like to propose rebasing to upstream OpenSCAP 1.3.4 release as has presently landed in Debian.

Upstream, OpenSCAP is a Red Hat maintained project. Version 1.2.x (as currently present in Ubuntu releases) aligns with RHEL 7. Version 1.3.x has shipped in RHEL 8 and is currently in Fedora ELN (slated for RHEL 9). Since RHEL 7 has left active feature development, it makes sense for new Ubuntu releases to move to the active 1.3.x version. Additionally, 1.3.x features several enhancements and bug fixes over the 1.2.x branch.

Debian has picked up OpenSCAP 1.3.4 prior to the recent upstream 1.3.5 release. While these changes are helpful (including SCAP 1.3 which brings in mandatory OVAL 5.11.x support), we feel it is more important to follow Debian's lead in this instance. Additionally, even older 1.2.x versions of OpenSCAP support OVAL 5.11.x content in SCAP 1.2 content, making this a lower concern.

Changes over Debian's 1.3.4 include:

 - Shipping autotailor, a utility to tailor XCCDF files (changing variables and selecting/deselecting rules in an XCCDF profile), including manpage.
 - Shipping helper function oscap-run-sce-script that was missed in packaging. This utility helps when SCE content is shipped without executable permissions.
 - Pulling in the dpkg verison comparison patches from Hirsute.

Note that the dpkg version comparison patches have landed upstream in the 1.3.5 release and so should eventually be dropped were we to rebase in the future.

All other hirsute patches have been dropped as they have been picked up by this release.

This rebase has been sanity-tested against building ComplianceAsCode/content and no errors reported. At this time, the Security team does not have any Impish content and thus cannot test scanner functionality against this release.

A PPA containing this build can be found here: https://launchpad.net/~cipherboy/+archive/ubuntu/scap/+build/21682204

Thanks,
Alex

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "rev1 rebase debdiff over sid" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Alex Murray (alexmurray)]

The debdiff appears to be missing a lot of changes from 1.2.17 in impish to 1.3.4 - it should be against the current version in impish, not the 1.3.4-1 version in Debian.

Also the debian/changelog then should include entries for the intervening releases in Ubuntu.

See attached for a version of this merge that I did myself which should hopefully give you more of an idea of what is normally expected.

[Alexander Scheel (cipherboy)]

Thanks Alex for your comments! :-)

I've attached the impish-to-impish debdiff as rev3. This aligns closer with the changelog format suggested by Seth but otherwise contains no new deltas.

Should I also provide the sid-to-impish debdiff? A 15MiB impish-to-impish debdiff seems much harder to review than the smaller 15KiB sid-to-impish one.

- A

[Alexander Scheel (cipherboy)]

Per discussion with Alex on MM, attached same rev3 debdiff just based against sid instead of impish (making it more reviewable). No changes were made and same .dsc file was used.

[Alex Murray (alexmurray)]

Thanks, you seem to have dropped the changelog entry for 1.3.4-1 - can this please be added back in?

Also the changelog entry for your merge should include an entry like:

  * Dropped changes, included upstream:
    - debian/patches/5e5bc61c1fc6a6556665aa5689a62d6bc6487c74.patch: Fix
      dangling '*' in dpkginfo_free_reply declaration.

As this patch has now been dropped which we previously carried as a delta.

Thanks

[Alexander Scheel (cipherboy)]

Changelog updated.

[Alexander Scheel (cipherboy)]

Sorry about the delay. The merge tool was not working for me:

dpkg-mergechangelogs ../../debian/openscap/debian/changelog ../../ubuntu-impish-original/openscap-1.2.17/debian/changelog ./debian/changelog.bak > debian/changelog

This was giving me a file with only the 1.2.17 changelog and no entries from debian and/or my new changelog entry. I ended up manually pulling the latest Debian changelog entry over.

[Alex Murray (alexmurray)]

The sid to impish debdiff missing seems to have dropped a heap of changelog entries (unless I am reading it wrong...)

Also I can't seem to cleanly apply the impish-to-impish debdiff - most of it is fine but for a few of the files it complains that the patch is reversed plus some bits don't apply properly at all:

[amurray:~/ubuntu/sbuild] $ umt download -r impish openscap
Downloading 'openscap' version '1.2.17-0.1ubuntu5' for release 'impish'.
[amurray:~/ubuntu/sbuild] 7s $ cd openscap/impish/openscap-1.2.17/
[amurray:~/ubuntu … h/openscap-1.2.17] $ patch -p1 --dry-run < ~/Downloads/openscap-1.3.4.impish-to-impish-rev4.debdiff
checking file acinclude.m4
checking file ac_probes/ac_probes.sh
checking file ac_probes/configure.ac.tpl
checking file ac_probes/libs/acl
checking file ac_probes/libs/apt_pkg
checking file ac_probes/libs/blkid
checking file ac_probes/libs/cap
checking file ac_probes/libs/dbus1
checking file ac_probes/libs/gconf2
checking file ac_probes/libs/lber
checking file ac_probes/libs/ldap
checking file ac_probes/libs/opendbx
checking file ac_probes/libs/pcre
checking file ac_probes/libs/procps
checking file ac_probes/libs/rpm
checking file ac_probes/libs/selinux
checking file ac_probes/libs/xml2
checking file ac_probes/libs/xslt
checking file ac_probes/README
checking file appveyor.yml
checking file AUTHORS
checking file autogen.sh
checking file cmake/Copyright.txt
checking file cmake/FindACL.cmake
checking file cmake/FindAptPkg.cmake
checking file cmake/FindBlkid.cmake
checking file cmake/FindCap.cmake
checking file cmake/FindDBus.cmake
checking file cmake/FindGConf.cmake
checking file cmake/FindGCrypt.cmake
checking file cmake/FindGLib.cmake
checking file cmake/FindGObject.cmake
checking file cmake/FindLdap.cmake
checking file cmake/FindLibyaml.cmake
checking file cmake/FindNSS.cmake
checking file cmake/FindOpenDbx.cmake
checking file cmake/FindPCRE.cmake
checking file cmake/FindPopt.cmake
checking file cmake/FindProcps.cmake
checking file cmake/FindRPM.cmake
checking file cmake/FindSELinux.cmake
checking file cmake/LibFindMacros.cmake
checking file CMakeLists.txt
checking file compat/CMakeLists.txt
checking file compat/compat.h
checking file compat/dev_to_tty.c
checking file compat/oscap_platforms.h
checking file compat/strptime.c
checking file compat/strsep.c
checking file confgen.sh
checking file config/snippet/arg-nonnull.h
checking file config/snippet/c++defs.h
checking file config/snippet/_Noreturn.h
checking file config/snippet/warn-on-use.h
checking file config.h.in
checking file configure.ac
checking file cpe/CMakeLists.txt
checking file cpe/Makefile.am
checking file cpe/openscap-cpe-dict.xml
checking file cpe/openscap-cpe-oval.xml
checking file debian/changelog
Hunk #1 FAILED at 1.
Hunk #2 succeeded at 101 (offset 6 lines).
1 out of 2 hunks FAILED
checking file debian/compat
checking file debian/control
checking file debian/gbp.conf
checking file debian/libopenscap8.install
checking file debian/libopenscap8.lintian-overrides
checking file debian/libopenscap-dev.install
checking file debian/missing-sources/bootstrap.js
checking file debian/missing-sources/jquery.js
checking file debian/missing-sources/README
checking...

[Alexander Scheel (cipherboy)]

Rebase off of 1ubuntu5 presently in Impish.

[Alexander Scheel (cipherboy)]

Hmmm maybe the changelog failing to was caused by the 1ubuntu5 update I didn't see. I've redone the patchset off of 1ubuntu5.

I think I've fixed the missing Ubuntu entries, but note that the existing Ubuntu 1ubuntu5 changelog drops a lot of older Debian entries.

[Alex Murray (alexmurray)]

Thanks - looking at the sid-to-impish debdiff I see a heap of changes to yaml-filter/ which aren't documented in the changelog entry - were these intentional? Ideally the debdiff to impish from sid would only contain changes to the debian/ directory. Can you check the debdiff and see that the various changes there are all intentional?

[Alexander Scheel (cipherboy)]

Note that due to the 1ubuntu5 change I decided to start with a fresh tarball: https://github.com/OpenSCAP/openscap/releases/download/1.3.4/openscap-1.3.4.tar.gz

yaml-filter isn't tracked in Debian's src-git: it is a git submodule in the upstream repo and it appears that Debian hasn't included it and hasn't documented why AFAICT; it was missing from the big 1.3.4 update commit. Using the upstream cmake-generated tarball aligns us better with what OpenSCAP upstream uses and supports (note that the same team does the fedora packaging off of this tarball as well: https://src.fedoraproject.org/rpms/openscap/blob/rawhide/f/openscap.spec#_8 -- and messages here https://gitter.im/OpenSCAP/openscap?at=6103f97323956a5aa464209c). yaml-filter (https://github.com/OpenSCAP/yaml-filter/) isn't packaged in Ubuntu and thus would still be required if we wish to support the yamlfilepath probes (with support for auditing YAML config files, which upstream CaC is beginning to use more).

It seems like, if you'd like to minimize Debian-Ubuntu delta, we should open a bug with Debian to get yaml-filter packaged and/or included in their srcgit? I'm guessing it is a simple oversight as the upstream OpenSCAP team hasn't used submodules before in their repo.

[Alex Murray (alexmurray)]

I don't love the idea of vendoring in yaml-filter here - there is no way to easily update / maintain this, plus by diverging from debian's orig.tar.gz this will make doing future merges / updates from Debian for openscap a lot harder. For now, to get this bug closed, can we simply merge in the ubuntu changes onto Debian's openscap-1.3.4 and then as a future step look to getting yaml-filter into Debian's openscap package?

[Eduardo Barretto (ebarretto)]

openscap 1.3.7 now available in lunar, and yaml-filter also present on it.
I'm closing this bug then.