[ffe][sru]: Update sosreport [last] plugin

[Eric Desrochers]

[IMPACT]

Canonical support is about to offer SOSCleaner project for UA customer who wants to obfuscate sensible data from an existing sosreport tarball such as user, hostname, network information obfuscation.

For user obfuscation, currently SOSCleaner relies on what sosreport offers which is "lastlog", reporting all user regardless of their UID.

Unfortunately, this generate a lot of false positives (and require a significant ignored_users list) inside SOSCleaner especially for user in the UID range 0-999 (e.g.'sys', 'bin', 'syslog', and much more)

I have submitted a PR in sosreport upstream in order to split lastlog output by UID ranges, in order to help SOSCleaner to obfuscate what we really want to obfuscate and ignore the rest.

https://github.com/sosreport/sos/issues/1743
https://github.com/sosreport/sos/pull/1770

Once approved upstream, I'd like to push that change in E/D/B/X in order to unblock the development of the tool "SOSCleaner" which unfortunately won't land in the archive for now, will be offered in a Private PPA. Note that the plan is for the package to land in the Ubuntu archive in the near future (few months still) and this change will highly impact the delay if this fix doesn't land in sosreport (Ubuntu archive).

[TEST CASE]

* Install sosreport
 ** Run sosreport:
 - sosreport -a
* Install soscleaner
 ** Run soscleaner:
 - soscleaner /tmp/sosreport-<TARBALL>
* Make sure only user in the 1000-60000 UID range are obfuscated.

[REGRESSION POTENTIAL]
- None, sosreport will still collect the entire lastlog (current behavior), we are not removing that behaviour, we simply add an alternative which is not costly for typical systems.

[OTHER INFO]

* Upstream fix not merged yet, but +1 by maintainers:
https://github.com/sosreport/sos/issues/1743
https://github.com/sosreport/sos/pull/1770

* Build logs:
https://launchpad.net/~slashd/+archive/ubuntu/lp1842489/+build/17517716
https://launchpad.net/~slashd/+archive/ubuntu/lp1842489/+build/17517717

[Łukasz Zemczak (sil2100)]

Ok, this seems fine. Be sure to test-build the changes before uploading to eoan to protect against it FTBFS in the archive. +1 on the FFe from eoan's POV.

[Eric Desrochers (slashd)]

[EOAN VERIFICATION PRE-UPLOAD]

$ ls -altr sosreport-eoansponsor-20190904170850/sos_commands/last/
total 40
-rw-r--r-- 1 root root 113 Sep 4 17:08 lastlog_-u_65537-4294967295
-rw-r--r-- 1 root root 113 Sep 4 17:08 lastlog_-u_60001-65536
-rw-r--r-- 1 root root 113 Sep 4 17:08 lastlog_-u_1000-60000
-rw-r--r-- 1 root root 1877 Sep 4 17:08 lastlog_-u_0-999
-rw-r--r-- 1 root root 2066 Sep 4 17:08 lastlog
-rw-r--r-- 1 root root 38 Sep 4 17:08 last_shutdown
-rw-r--r-- 1 root root 183 Sep 4 17:08 last_reboot
-rw-r--r-- 1 root root 183 Sep 4 17:08 last
drwx------ 2 root root 4096 Sep 4 17:08 .
drwx------ 47 root root 4096 Sep 4 17:08 ..

$ more sosreport-eoansponsor-20190904170850/sos_commands/last/lastlog_-u_*
sosreport-eoansponsor-20190904170850/sos_commands/last/lastlog_-u_0-999
::::::::::::::
Username Port From Latest
root **Never logged in**
daemon **Never logged in**
bin **Never logged in**
sys **Never logged in**
sync **Never logged in**
games **Never logged in**
man **Never logged in**
lp **Never logged in**
mail **Never logged in**
news **Never logged in**
uucp **Never logged in**
proxy **Never logged in**
www-data **Never logged in**
backup **Never logged in**
list **Never logged in**
irc **Never logged in**
gnats **Never logged in**
systemd-timesync **Never logged in**
systemd-network **Never logged in**
systemd-resolve **Never logged in**
messagebus **Never logged in**
syslog **Never logged in**
_apt **Never logged in**
uuidd **Never logged in**
sshd **Never logged in**
landscape **Never logged in**
pollinate **Never logged in**
systemd-coredump **Never logged in**
lxd **Never logged in**
::::::::::::::
sosreport-eoansponsor-20190904170850/sos_commands/last/lastlog_-u_1000-60000
::::::::::::::
Username Port From Latest
ubuntu **Never logged in**
::::::::::::::
sosreport-eoansponsor-20190904170850/sos_commands/last/lastlog_-u_60001-65536
::::::::::::::
Username Port From Latest
nobody **Never logged in**
::::::::::::::
sosreport-eoansponsor-20190904170850/sos_c...

[Launchpad Janitor (janitor)]

This bug was fixed in the package sosreport - 3.6-1ubuntu3

---------------
sosreport (3.6-1ubuntu3) eoan; urgency=medium

  * d/p/split-lastlog-per-uid-ranges.patch: (LP: #1842489)
    - Split lastlog per UID ranges.

 -- Eric Desrochers <email address hidden> Wed, 04 Sep 2019 16:47:09 +0000

[Łukasz Zemczak (sil2100)]

Even though this is more of a new feature than an SRU-typical bugfix, I do see merits in getting this backported to the stable series.

[Łukasz Zemczak (sil2100)]

Hello Eric, or anyone else affected,

Accepted sosreport into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.6-1ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Eric, or anyone else affected,

Accepted sosreport into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.6-1ubuntu0.18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Eric, or anyone else affected,

Accepted sosreport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.6-1ubuntu0.16.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Eric Desrochers (slashd)]

[VERIFICATION DISCO]

proposed package version: 3.6-1ubuntu2.1

# sosreport -a
# tar Jxvf /tmp/sreport-<REDACTED_HOST>-20190910212551.tar.xz

# ls -altr sosreport-<REDACTED_HOST>-20190910212551/sos_commands/last/
-rw-r--r-- 1 root root 0 Sep 10 21:25 lastlog_-u_65537-4294967295 # New behaviour to split by UID Range work as expected.
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_60001-65536 # ...
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_1000-60000 # ...
-rw-r--r-- 1 root root 1877 Sep 10 21:25 lastlog_-u_0-999 # End
-rw-r--r-- 1 root root 2003 Sep 10 21:25 lastlog
-rw-r--r-- 1 root root 38 Sep 10 21:25 last_shutdown
-rw-r--r-- 1 root root 549 Sep 10 21:25 last_reboot
-rw-r--r-- 1 root root 549 Sep 10 21:25 last # Old behaviour remain intact

I also looked at the content of each file, and everything look good.

[Eric Desrochers (slashd)]

[VERIFICATION BIONIC]

-proposed package version: 3.6-1ubuntu0.18.04.3

# sosreport -a

# tar Jxvf /tmp/sosreport-<REDACTED_HOST>-20190911113800.tar.xz

# ls -altr sosreport-<REDACTED_HOST>-20190911113800/sos_commands/last/
-rw-r--r-- 1 root root 0 Sep 10 21:25 lastlog_-u_65537-4294967295 # New behaviour to split by UID Range work as expected.
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_60001-65536 # ...
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_1000-60000 # ...
-rw-r--r-- 1 root root 1877 Sep 10 21:25 lastlog_-u_0-999 # End
-rw-r--r-- 1 root root 2003 Sep 10 21:25 lastlog
-rw-r--r-- 1 root root 38 Sep 10 21:25 last_shutdown
-rw-r--r-- 1 root root 549 Sep 10 21:25 last_reboot
-rw-r--r-- 1 root root 549 Sep 10 21:25 last # Old behaviour remain intact

I also looked at the content of each file, and everything look good.

[Eric Desrochers (slashd)]

[VERIFICATION XENIAL]

-proposed package version: 3.6-1ubuntu0.16.04.3

# sosreport -a

# tar Jxvf /tmp/sosreport-<REDACTED_HOST>-20190911114236.tar.xz

# ls -altr sosreport-<REDACTED_HOST>-20190911114236/sos_commands/last/
-rw-r--r-- 1 root root 0 Sep 10 21:25 lastlog_-u_65537-4294967295 # New behaviour to split by UID Range work as expected.
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_60001-65536 # ...
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_1000-60000 # ...
-rw-r--r-- 1 root root 1877 Sep 10 21:25 lastlog_-u_0-999 # End
-rw-r--r-- 1 root root 2003 Sep 10 21:25 lastlog
-rw-r--r-- 1 root root 38 Sep 10 21:25 last_shutdown
-rw-r--r-- 1 root root 549 Sep 10 21:25 last_reboot
-rw-r--r-- 1 root root 549 Sep 10 21:25 last # Old behaviour remain intact

I also looked at the content of each file, and everything look good.

[Łukasz Zemczak (sil2100)]

And I see upstream actually approved the PR 5 hours ago, so we're good to go. Thanks for validating!

[Launchpad Janitor (janitor)]

This bug was fixed in the package sosreport - 3.6-1ubuntu2.1

---------------
sosreport (3.6-1ubuntu2.1) disco; urgency=medium

  * d/p/split-lastlog-per-uid-ranges.patch: (LP: #1842489)
    - Split lastlog per UID ranges.

 -- Eric Desrochers <email address hidden> Wed, 04 Sep 2019 20:38:52 +0000

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for sosreport has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package sosreport - 3.6-1ubuntu0.18.04.3

---------------
sosreport (3.6-1ubuntu0.18.04.3) bionic; urgency=medium

  * d/p/split-lastlog-per-uid-ranges.patch: (LP: #1842489)
    - Split lastlog per UID ranges.

 -- Eric Desrochers <email address hidden> Wed, 04 Sep 2019 20:51:59 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package sosreport - 3.6-1ubuntu0.16.04.3

---------------
sosreport (3.6-1ubuntu0.16.04.3) xenial; urgency=medium

  * d/p/split-lastlog-per-uid-ranges.patch: (LP: #1842489)
    - Split lastlog per UID ranges.

 -- Eric Desrochers <email address hidden> Wed, 04 Sep 2019 21:32:28 +0000