Openscap can report false positives

[Eduardo Barretto]

[Impact]

Openscap didn't implement Debian package version comparison algorithm.
This can cause a user/client to get false positive results when running
oscap.

For example, we have a system running Bionic, with package "foo" version
1.2.3-4ubuntu1~18.04.1 installed. Ubuntu fixed CVE-2020-XXXX on Bionic for
"foo" on version 1.2.3-4ubuntu1. If oscap compares both version it would
would return "false", meaning that "foo" is not vulnerable, which is not
correct as 1.2.3-4ubuntu1 is greater than the installed version
1.2.3-4ubuntu1~18.04.1.

$ dpkg --compare-versions 1.2.3-4ubuntu1 gt 1.2.3-4ubuntu1~18.04.1 && echo TRUE || echo FALSE
TRUE

If a client relies on software like openscap to decide when to upgrade their system (especially for clients that need to have a downtime to upgrade packages), openscap could be giving the wrong information and causing unnecessary downtimes, or even showing the system as vulnerable, when it isn't or vice-versa.

[Test Case]

Attached to this bug is a zip file that contains oval data for 3 different
packages (gdcm, gnutls28 and openssl) with specific CVE data for each (
CVE-2016-5300, CVE-2018-10845 and CVE-2020-1968). This data* is for Bionic
only.

The test consists of comparing the installed version of the mentioned
packages, to different versions where the CVE could have been fixed.

For more info on the test data see:
https://pastebin.ubuntu.com/p/cVp2xcq9fs/

Testing procedure (Bionic):
$ sudo apt update
$ sudo apt install libopenscap8
$ sudo apt install libgdcm2.8 openssl libgnutls30
$ tar -xzf test-data.tar.gz
$ cd test-data/
$ ./run.sh

Here is a diff between the results of the test, between current openscap
and the openscap with the algorithm fix:
https://pastebin.ubuntu.com/p/38N8GsgZnf/

*PS: This data doesn't reflect the reality of those vulnerabilities and it
should only be used for test purposes.

[Where problems could occur]

The patches only touch the comparison algorithm, so any regressions that it might have, might impact the comparison, generating false positives too.

[Other Info]

This affects all releases of Ubuntu, from Xenial to Hirsute.

The versioning algorithm implemented is based on dpkg's algorithm.

Upstream accepted and merged the Debian version comparison algorithm to
its maint-1.3 branch and it should make it to 1.3.5 version whenever it
gets released.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "xenial.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Marc Deslauriers (mdeslaur)]

Please modify debian/changelog to include:
- bug reference
- patch names

Thanks!

[Eduardo Barretto (ebarretto)]

The following debdiffs added the requests from Marc.

[Marc Deslauriers (mdeslaur)]

ACK on the recent debdiffs. I've uploaded the package to Hirsute, and to the stable releases for processing by the SRU team. Thanks!

[Launchpad Janitor (janitor)]

This bug was fixed in the package openscap - 1.2.17-0.1ubuntu4

---------------
openscap (1.2.17-0.1ubuntu4) hirsute; urgency=medium

  * Add dpkg version comparison algorithm to avoid false positives.
    (LP: #1911791)
    - debian/patches/dpkg-version-comparison-1.patch: Add dpkg version
      comparison algorithm.
    - debian/patches/dpkg-version-comparison-2.patch: Free a_copy and b_copy
      in case of failure and code format.
    - debian/patches/dpkg-version-comparison-3.patch: Fix
      oval_debian_evr_string_cmp.

 -- Eduardo Barretto <email address hidden> Tue, 05 Jan 2021 17:19:13 -0300

[Brian Murray (brian-murray)]

Hello Eduardo, or anyone else affected,

Accepted openscap into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openscap/1.2.17-0.1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Eduardo, or anyone else affected,

Accepted openscap into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openscap/1.2.16-2ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Eduardo, or anyone else affected,

Accepted openscap into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openscap/1.2.15-1ubuntu0.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Eduardo, or anyone else affected,

Accepted openscap into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openscap/1.2.8-1ubuntu0.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Eduardo Barretto (ebarretto)]

Hey Brian,

I ran the following test on Xenial, Bionic, Focal and Groovy with archive openscap and openscap from -proposed and compared the results:
$ wget https://people.canonical.com/~ubuntu-security/oval/com.ubuntu.$(lsb_release -cs).cve.oval.xml.bz2
$ bunzip2 com.ubuntu.$(lsb_release -cs).cve.oval.xml.bz2
$ oscap oval eval --report report.htm com.ubuntu.$(lsb_release -cs).cve.oval.xml

For Xenial the results are the same with both versions of openscap, which means the changes didn't introduce a regression so far. Same is true for Focal.

For Bionic the results differ:
 - With the archive openscap I get 607 vulnerabilities still needing a fix, while the -proposed version returns 606 vulnerabilities still needs a fix. The difference is CVE-2017-9763 and I could check that this is a false positive with archive openscap, which means that -proposed version fixed it.

For Groovy the results also differ:
 - With archive openscap I get 220 vulnerabilities still needing a fix, while the -proposed version returns 211 vulnerabilities still needs a fix. The differences are:
         CVE-2020-14803
         CVE-2020-14798
         CVE-2020-14797
         CVE-2020-14796
         CVE-2020-14792
         CVE-2020-14782
         CVE-2020-14781
         CVE-2020-14779
         CVE-2019-18348
   And I could check that those were all false positives with archive openscap, which means that -proposed version fixed it.

Hope this helps, let me know in case of doubts.

[Launchpad Janitor (janitor)]

This bug was fixed in the package openscap - 1.2.17-0.1ubuntu1.1

---------------
openscap (1.2.17-0.1ubuntu1.1) groovy; urgency=medium

  * Add dpkg version comparison algorithm to avoid false positives.
    (LP: #1911791)
    - debian/patches/dpkg-version-comparison-1.patch: Add dpkg version
      comparison algorithm.
    - debian/patches/dpkg-version-comparison-2.patch: Free a_copy and b_copy
      in case of failure and code format.
    - debian/patches/dpkg-version-comparison-3.patch: Fix
      oval_debian_evr_string_cmp.

 -- Eduardo Barretto <email address hidden> Wed, 06 Jan 2021 08:21:49 -0300

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for openscap has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package openscap - 1.2.16-2ubuntu3.2

---------------
openscap (1.2.16-2ubuntu3.2) focal; urgency=medium

  * Add dpkg version comparison algorithm to avoid false positives.
    (LP: #1911791)
    - debian/patches/dpkg-version-comparison-1.patch: Add dpkg version
      comparison algorithm.
    - debian/patches/dpkg-version-comparison-2.patch: Free a_copy and b_copy
      in case of failure and code format.
    - debian/patches/dpkg-version-comparison-3.patch: Fix
      oval_debian_evr_string_cmp.

 -- Eduardo Barretto <email address hidden> Wed, 06 Jan 2021 10:21:49 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package openscap - 1.2.15-1ubuntu0.3

---------------
openscap (1.2.15-1ubuntu0.3) bionic; urgency=medium

  * Add dpkg version comparison algorithm to avoid false positives.
    (LP: #1911791)
    - debian/patches/dpkg-version-comparison-1.patch: Add dpkg version
      comparison algorithm.
    - debian/patches/dpkg-version-comparison-2.patch: Free a_copy and b_copy
      in case of failure and code format.
    - debian/patches/dpkg-version-comparison-3.patch: Fix
      oval_debian_evr_string_cmp.

 -- Eduardo Barretto <email address hidden> Wed, 06 Jan 2021 10:23:12 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package openscap - 1.2.8-1ubuntu0.3

---------------
openscap (1.2.8-1ubuntu0.3) xenial; urgency=medium

  * Add dpkg version comparison algorithm to avoid false positives.
    (LP: #1911791)
    - debian/patches/dpkg-version-comparison-1.patch: Add dpkg version
      comparison algorithm.
    - debian/patches/dpkg-version-comparison-2.patch: Free a_copy and b_copy
      in case of failure and code format.
    - debian/patches/dpkg-version-comparison-3.patch: Fix
      oval_debian_evr_string_cmp.

 -- Eduardo Barretto <email address hidden> Wed, 06 Jan 2021 11:21:46 -0300