Bug #2038648 “package sosreport 4.4-1ubuntu0.18.04.1 failed to i...” : Bugs : sosreport package : Ubuntu

Revision history for this message

Branimir Radovic (bradovic) wrote on 2023-10-06:

#1

Dependencies.txt Edit (2.4 KiB, text/plain; charset="utf-8")
Df.txt Edit (466 bytes, text/plain; charset="utf-8")
Dmesg.txt Edit (37.0 KiB, text/plain; charset="utf-8")
DpkgHistoryLog.txt Edit (10.0 KiB, text/plain; charset="utf-8")
DpkgTerminalLog.txt Edit (403 bytes, text/plain; charset="utf-8")
DuplicateSignature.txt Edit (443 bytes, text/plain; charset="utf-8")
ProcCpuinfoMinimal.txt Edit (900 bytes, text/plain; charset="utf-8")

Revision history for this message

Athos Ribeiro (athos-ribeiro) wrote on 2023-10-18:

#2

Thanks for reporting this bug.

This is an interesting one. The user is upgrading from xenial to bionic.

xenial has python 3.5, bionic has python 3.6.

During the postinst (included by dh_python), sosreport runs a python script containing an f-string. However f-strings were only introduced in python 3.6. In this case, the postinst ran with python 3.5, which results in a syntax error when the f-string is parsed.

tags:

added: server-triage-discuss

Revision history for this message

Robie Basak (racb) wrote on 2023-10-25:

#3

This sounds like it needs a versioned depends on a higher version of Python, but this might mess with the ordering of package upgrades on release upgrade, so needs to be considered carefully.

Changed in sosreport (Ubuntu):
status:	New → Triaged
Changed in sosreport (Ubuntu Bionic):
status:	New → Triaged

Revision history for this message

Robie Basak (racb) wrote on 2023-10-25:

#4

Probably a fix isn't needed in series after Bionic? Needs checking.

Revision history for this message

Robie Basak (racb) wrote on 2023-10-25:

#5

High for Bionic because this affects release upgrades from Xenial to Bionic when sosreport is installed, which is something that Canonical recommends to its customers.

Revision history for this message

Robie Basak (racb) wrote on 2023-10-25:

#6

And indeed it's installed by default on server.

Robie Basak (racb) on 2023-10-25

Changed in sosreport (Ubuntu Bionic):
importance:	Undecided → High

Revision history for this message

nikhil kshirsagar (nkshirsagar) wrote on 2023-10-25 (last edit on 2023-10-26):

#7

Download full text (3.4 KiB)

I looked into this issue, and also discussed with mfo.

The sos code that fails compilation due to the python 3.6 package not yet being installed is,

    @staticmethod
    def _query_address(url: str) -> HTTPResponse:
        """
        Query the given url address with headers required by Google Metadata
        Server.
        """
        try:
            req = request.Request(url, headers={'Metadata-Flavor': 'Google'})
            response = request.urlopen(req)
        except URLError as err:
            raise RuntimeError(
                "Failed to communicate with Metadata Server: " + str(err))
        if response.code != 200:
            raise RuntimeError(
                f"Failed to communicate with Metadata Server " <---
                f"(code: {response.code}): " + response.read().decode()) <----
        return response

We have decided to hold off the 4.5.6 release on bionic since 4.5.6 has many more plugins using f strings.

For this version, it's just the gcp plugin that has f strings in the bionic sos. But 4.5.6 source has more than hundred instances (https://pastebin.canonical.com/p/8DpdjwWKM4/)

Looking further,

The generated postinst tries to build py3compile sosreport, (Thanks mfo for this info!)

$ pull-lp-debs sosreport bionic 4.4-1ubuntu0.18.04.1
Found sosreport 4.4-1ubuntu0.18.04.1 in bionic
Downloading sosreport_4.4-1ubuntu0.18.04.1_amd64.deb from archive.ubuntu.com (0.291 MiB)
[=====================================================>]100%
:~/Downloads/dpkgtest$ mkdir ext
:~/Downloads/dpkgtest$ dpkg-deb -e
ext/ sosreport_4.4-1ubuntu0.18.04.1_amd64.deb
:~/Downloads/dpkgtest$ dpkg-deb -e sosreport_4.4-1ubuntu0.18.04.1_amd64.deb ext/
:~/Downloads/dpkgtest$ cd ext/
:~/Downloads/dpkgtest/ext$ ls
conffiles control md5sums postinst prerm
:~/Downloads/dpkgtest/ext$ cat postinst
#!/bin/sh
set -e

# Automatically added by dh_python3:
if which py3compile >/dev/null 2>&1; then
py3compile -p sosreport <----
fi

# End automatically added section

There's a couple of things we can do for a fix.

Either have the depends in the control file using something like python3 (>= 3.6.5-3) which might be an easier fix, or try to generate a check additionally in the postinst that dh_python generates, that does not compile sos if the version is bionic and python 3.6 isn't available.

The series after bionic seem fine,

I looked into this issue, and also discussed with mfo.

The sos code that fails compilation due to the python 3.6 package not yet being installed is,

@staticmethod
    def _query_address(url: str) -> HTTPResponse:
        """
        Query the given url address with headers required by Google Metadata
        Server.
        """
        try:
            req = request.Request(url, headers={'Metadata-Flavor': 'Google'})
            response = request.urlopen(req)
        except URLError as err:
            raise RuntimeError(
                "Failed to communicate with Metadata Server: " + str(err))
        if response.code != 200:
            raise RuntimeError(
                f"Failed to communicate with Metadata Server " <---
                f"(code: {response.code}): " + response.read().decode()) <----
        return response

We have decided to hold off the 4.5.6 release on bionic since 4.5.6 has many more plugins using f strings.

For this version, it's just the gcp plugin that has f strings in the bionic sos. But 4.5.6 source has more than hundred instances (https://pastebin.canonical.com/p/8DpdjwWKM4/)

Looking further,

The generated postinst tries to build py3compile sosreport, (Thanks mfo for this info!)

$ pull-lp-debs sosreport bionic 4.4-1ubuntu0.18.04.1
Found sosreport 4.4-1ubuntu0.18.04.1 in bionic
Downloading sosreport_4.4-1ubuntu0.18.04.1_amd64.deb from archive.ubuntu.com (0.291 MiB)
[=====================================================>]100%
:~/Downloads/dpkgtest$ mkdir ext
:~/Downloads/dpkgtest$ dpkg-deb -e  
ext/                                      sosreport_4.4-1ubuntu0.18.04.1_amd64.deb  
:~/Downloads/dpkgtest$ dpkg-deb -e  sosreport_4.4-1ubuntu0.18.04.1_amd64.deb ext/
:~/Downloads/dpkgtest$ cd ext/
:~/Downloads/dpkgtest/ext$ ls
conffiles  control  md5sums  postinst  prerm
:~/Downloads/dpkgtest/ext$ cat postinst 
#!/bin/sh
set -e

# Automatically added by dh_python3:
if which py3compile >/dev/null 2>&1; then
	py3compile -p sosreport  <----
fi

# End automatically added section

There's a couple of things we can do for a fix.

Either have the depends in the control file using something like python3 (>= 3.6.5-3) which might be an easier fix, or try to generate a check additionally in the postinst that dh_python generates, that does not compile sos if the version is bionic and python 3.6 isn't available.

The series after bionic seem fine,

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2023-10-25:

#8

I could not reproduce this in a GCP VM or LXD VM.
(launch 16.04, apt update && apt -y upgrade, reboot, do-release-upgrade).

# grep -A1 'Setting up sosreport' /var/log/dist-upgrade/apt-term.log
Setting up sosreport (4.4-1ubuntu0.18.04.1) ...
Setting up xfsprogs (4.9.0+nmu1ubuntu2) ...

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2023-10-25:

#9

Hi Branimir,

Could you please upload /var/log/dist-upgrade/screenlog.0
(or a tarball with the full directory contents) ?

Thanks!

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2023-10-25:

#10

lp2038648-sosreport-bionic.debdiff Edit (2.7 KiB, text/plain)

If it turns out to be needed, we could consider a minimal change to bionic-updates for now (details in .patch comment).

Changed in sosreport (Ubuntu Bionic):
status:	Triaged → Incomplete

Ubuntu Foundations Team Bug Bot (crichton) on 2023-10-26

tags:

added: patch

Revision history for this message

nikhil kshirsagar (nkshirsagar) wrote on 2023-10-26:

#11

Hi Mauricio,

I could not reproduce this either,

root@dixie:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic
root@dixie:~# dpkg -l | grep sos
ii sosreport 4.4-1ubuntu0.18.04.1 amd64 Set of tools to gather troubleshooting data from a system

Looking at /var/log/dist-upgrade/apt-term.log, python3 got installed and set up earlier than sos,

1704 Setting up python3.6 (3.6.9-1~18.04ubuntu1.12) ...^M
1705 Setting up libpython3-stdlib:amd64 (3.6.7-1~18.04) ...^M
1706 Setting up python3 (3.6.7-1~18.04) ...^M
1707 running python rtupdate hooks for python3.6...^M

2003 Setting up python3-pexpect (4.2.1-1) ...^M
2004 Setting up sosreport (4.4-1ubuntu0.18.04.1) ...^M
2005 Setting up xfsprogs (4.9.0+nmu1ubuntu2) ...^M

For bionic 4.5.6 to be able to release on bionic (https://warthogs.atlassian.net/browse/SEC-2624), we would need a solution for the more than hundred instances of formatted strings used in 4.5.6. (https://pastebin.canonical.com/p/8DpdjwWKM4/)

I noted "The optional X-Python3-Version field specifies the versions of Python 3 supported. ", but I am not sure if this would apply for the compilation in the postinst.

The Depends clause currently is,

Depends: ${python3:Depends}, ${misc:Depends}, python3-pexpect

Does adding python3 (>= 3.6.5-3) to Depends sound like a reasonable solution?

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2023-10-26:

#12

Hi Nikhil,

Thanks for checking!

> I noted "The optional X-Python3-Version field specifies the versions of Python 3 supported. ", but I am not sure if this would apply for the compilation in the postinst.

The version of the package providing py3compile at postinst time should be the key factor, IIUIC?

> Does adding python3 (>= 3.6.5-3) to Depends sound like a reasonable solution?

IMHO we first should understand the issue/criteria for reproducing the problem;
so I'd wait for Branimir to reply, or we try proactively to reproduce the issue,
e.g., should pro not be enabled? (there are no 'esm' versions in attached logs.)

When that is clear, we should be in a better position to assess the impact/pros/cons of different approaches to address it.

And as Robie mentioned in comment #3, if we change that, we should be clear on the changes/impacts to do-release-upgrade
(that's why I suggested in our chat to run do-release-upgrade before/after such a change, and compare the logs to look for differences, and also understand the steps done by do-release-ugprade in general w.r.t. python, if any).

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2023-10-26 (last edit on 2023-10-26):

#13

Additionally, for the future, we could use this opportunity to further improve the sosreport SRU exception/test coverage [1,2] with release upgrade testing too (after understanding the details here; and it should be easy to automate).

Apparently this issue got in the SRU of sosreport 4.3 [3], but only got a bug report at the 4.4 timeframe.
(Ok, this means it isn't apparently widespread, which is good news, but we can always improve on catching issues early! :)

--- sosreport-4.1/sos/report/plugins/gcp.py 1970-01-01 00:00:00.000000000 +0000
+++ sosreport-4.3/sos/report/plugins/gcp.py 2022-02-15 04:20:20.000000000 +0000
...
+ f"Failed to communicate with Metadata Server "
+ f"(code: {response.code}): " + response.read().decode())
...

...

Thanks again for all your work on this!

[1] https://wiki.ubuntu.com/StableReleaseUpdates#sosreport
[2] https://wiki.ubuntu.com/SosreportUpdates
[3] https://launchpadlibrarian.net/588301517/sosreport_4.1-1ubuntu0.18.04.3_4.3-1ubuntu0.18.04.1.diff.gz

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2023-10-27:

#14

(comment #12)
> IMHO we first should understand the issue/criteria for reproducing the problem;
> so I'd wait for Branimir to reply, or we try proactively to reproduce the issue,
> e.g., should pro not be enabled? (there are no 'esm' versions in attached logs.)

I tried to reproduce this scenario too (or close to it): launched 16.04 Pro VM in GCP and downgraded the packages with 'esm' version suffix to the version in xenial-updates (this changes package status, so might not reflect the reporter's environment), then do-release-upgrade.

The issue didn't reproduce.

$ grep -A1 'Setting up sosreport' /var/log/dist-upgrade/apt-term.log
Setting up sosreport (4.4-1ubuntu0.18.04.1) ...
Setting up xfsprogs (4.9.0+nmu1ubuntu2) ...

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2023-10-27:

#15

I also tried to reproduce in a 16.04 LXD VM, which does not have any esm packages installed by default (so the package status change above wasn't an issue), and the issue didn't reproduce either.

# grep -A1 'Setting up sosreport' /var/log/dist-upgrade/apt-term.log
Setting up sosreport (4.4-1ubuntu0.18.04.1) ...
Setting up xfsprogs (4.9.0+nmu1ubuntu2) ...

So, after all these attempts to reproduce the issue, and failing to do so, it apparently indicates the issue might be related to something particular in this VM/system packages status?

Hopefully Branimir's logs requested in comment #9 may help understand this better.

Changed in sosreport (Ubuntu):
status:	Triaged → Invalid

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2023-10-27:

#16

(Updating sosreport/devel release to Invalid as this issue only affects Xenial -> Bionic upgrades.)

Bryce Harrington (bryce) on 2023-11-01

tags:

removed: server-triage-discuss

Mauricio Faria de Oliveira (mfo) on 2023-11-16

Changed in sosreport (Ubuntu Bionic):
status:	Incomplete → Triaged

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-13:

#17

As we have not been able to reproduce the problem, but still conscious of the issue. We have implemented an update with the debian control file for the next SRU where we depend on python 3.6. By default, based on testing the dependency was for python 3.3, and as xenial already had this, we could have gone into a situation where the python3 was then not upgraded for sosreport to be functional.

Ref: https://github.com/sosreport/sos/pull/3567

Changed in sosreport (Ubuntu Bionic):
status:	Triaged → Fix Committed
assignee:	nobody → Arif Ali (arif-ali)

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-03-14 (last edit on 2024-03-14):

#18

Hi Arif,

I'll revert the status in 'sosreport (Ubuntu) / Bionic' to In Progress
from Fix Committed, as this means available in bionic-proposed in the
'Ubuntu' project.

The PR you mentioned being committed in sosreport upstream refers
to another project in Launchpad terms; you may find 'sosreport'
in 'Also affects project' to better reflect that if needed! :)

cheers,

Changed in sosreport (Ubuntu Bionic):
status:	Fix Committed → In Progress

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-14:

#19

ack, no worries.

As per our discussions yesterday, I've run 2 different dist-upgrades, and did d dpkg -l in both scenarios. One with my amended PPA [1] with X-Python3-Version: >= 3.6 and one that is default from the bionic repos, and the only difference was the package available in the PPA, as shown below

--- xenial01-post-upgrade_comma.txt 2024-03-14 15:14:51.421651041 +0000
+++ xenial02-post-upgrade_comma.txt 2024-03-14 15:14:58.693678265 +0000
@@ -502,7 +502,7 @@
ii,shared-mime-info,1.9-2
ii,snapd,2.58+18.04.1
ii,software-properties-common,0.96.24.32.22
-ii,sosreport,4.7.0-0ubuntu1~18.04.7
+ii,sosreport,4.4-1ubuntu0.18.04.1
ii,squashfs-tools,1:4.3-6ubuntu0.18.04.4
ii,ssh-import-id,5.7-0ubuntu1.1
ii,strace,4.21-1ubuntu1

I hope that this helps and avoids any concerns on the extra package dependancies.

[1] https://launchpad.net/~arif-ali/+archive/ubuntu/sosreport-dev

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-14:

#20

apt.log Edit (53.9 KiB, text/plain)

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-14:

#21

apt-term.log Edit (210.5 KiB, text/plain)

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-14:

#22

history.log Edit (58.8 KiB, text/plain)

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-14:

#23

main.log Edit (45.9 KiB, text/plain)

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-14:

#24

Excerpts from the logs with the upgrade using the PPA

apt.log
100 MarkInstall python3 [ amd64 ] < 3.5.1-3 -> 3.6.7-1~18.04 > ( python ) FU=0
101 Installing python3.6 as Depends of python3
102 MarkInstall python3.6 [ amd64 ] < none -> 3.6.9-1~18.04ubuntu1.12 > ( python ) FU=0
103 Installing python3.6-minimal as Depends of python3.6
104 MarkInstall python3.6-minimal [ amd64 ] < none -> 3.6.9-1~18.04ubuntu1.12 > ( python ) FU=0
105 Installing libpython3.6-minimal as Depends of python3.6-minimal
106 MarkInstall libpython3.6-minimal [ amd64 ] < none -> 3.6.9-1~18.04ubuntu1.12 > ( python ) FU=0
107 Installing libpython3.6-stdlib as Recommends of libpython3.6-minimal
108 MarkInstall libpython3.6-stdlib [ amd64 ] < none -> 3.6.9-1~18.04ubuntu1.12 > ( python ) FU=0
<snip>
548 MarkInstall sosreport [ amd64 ] < 3.9.1-1ubuntu0.16.04.3 -> 4.7.0-0ubuntu1~18.04.7 > ( admin ) FU=0
549 Installing python3-pexpect as Depends of sosreport
550 MarkInstall python3-pexpect [ amd64 ] < none -> 4.2.1-1 > ( python ) FU=0
551 Installing python3-ptyprocess as Depends of python3-pexpect
552 MarkInstall python3-ptyprocess [ amd64 ] < none -> 0.5.2-1 > ( python ) FU=0

apt-term.log
1719 Setting up python3 (3.6.7-1~18.04) ...^M
2019 Setting up sosreport (4.7.0-0ubuntu1~18.04.7) ...^M

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-03-15:

#25

Hi Arif,

Cool, thanks for the tests and logs!

Did you also have a chance to compare apt history.log (or another log for this purpose),
as we discussed, to be able to address/consider the point from Robie in comment #3?

Thanks again,

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-03-15:

#26

By the way, if at all possible, could you please test the changes on top of sosreport 4.4 in Bionic (for a minimal change, instead of 4.7) so that we can SRU it to the Ubuntu Archive, to address non-Pro/ESM users?
(Bionic in Ubuntu Pro/ESM can have the newer version).

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-20:

#27

bionic-4_4-sosreport.debdiff Edit (1.0 KiB, text/plain)

The 4.4 minimal change debdiff attached. tested this in lab, and works as expected

tags:

added: sts-sru-needed

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-03-21:

#28

Hi Arif,

Thanks for the debdiff!

AFAICT, comment #25 (ref to #3) is not yet addressed, could you please check/confirm?

Changed in sosreport (Ubuntu Bionic):
status:	In Progress → Incomplete

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-21:

#29

So, did the tests again to get more data

Below is the diff between the 2 apt-term.log, which shows the ordering of installation of the packages. The main differences are dates, the version of sosreport being installed and loads of data around "Reading database"

https://paste.ubuntu.com/p/8wZ5bsXcbh/

history.log was not worth looking at, as it was only 25 lines

On another note, I was looking for a corner case where I could re-produce the problem.

Installed a xenial machine, changed /etc/apt/sources.ist to point to bionic, and then just install sosreport. This reports the same problem as reported by OP.

Now, adding my test PPA with the "X-Python3-version: >= 3.6" doesn't see this problem, and installs python3.6 from the bionic repos.

So, on that basis, I think this should work as expected.

Hopefully that suffices the tests, and let me know if you need anything further

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-03-25:

#30

diff_apt-term_log.diff Edit (216.0 KiB, text/plain)

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-03-25:

#31

Hi Arif,

Thank you very much!

I'm attaching the pastebin contents for archival purposes (pastebins may disappear).

I can confirm there are no changes on the install order of python or other packages.
(filtered diff [1])

AFAICT, previous concerns were addressed (ordering of package upgrades due to python
versioned dependency, in comment #3), so it looks good, and this is a proper fix vs.
the workaround in comment #10, as the python language features in the package should
be reflected in the dependencies.

So, let's proceed with the SRU!

One of the requirements is the newer releases (supported & devel) are fixed.
Can you please provide the debdiffs for them, for us to proceed?

$ for release in noble mantic jammy focal bionic; do git show pkg/ubuntu/${release}-devel:debian/control | grep 'X-Python-Version:'; done
$

I think that the strictly minimal Python version (ie, based on language features)
is not necessarily what has to be figured out and included, but just the lowest
Python version supported per release (e.g., `py3versions --min-supported`),
so as to ensure that it (or a later version) is always installed on that release
or on upgrade to it when sosreport is installed.
(And of course, if a language feature of a Python version newer than the lowest
supported is needed, than that should be used instead. :)

Thanks again for all the work and help on fixing this issue!

[1]

$ grep '^[+-]' /tmp/diff_apt-term_log.diff | grep -v 'Reading database'
--- xenial01_logs/log/dist-upgrade/apt-term.log 2024-03-21 14:16:41.306113621 +0000
+++ xenial02_logs/log/dist-upgrade/apt-term.log 2024-03-21 14:16:58.374173593 +0000
-Log started: 2024-03-21 12:48:08
-Log ended: 2024-03-21 12:48:11
+Log started: 2024-03-21 13:31:32
+Log ended: 2024-03-21 13:31:34
-Log started: 2024-03-21 12:48:16
+Log started: 2024-03-21 13:31:37
-Log ended: 2024-03-21 12:48:27
+Log ended: 2024-03-21 13:31:44
-Log started: 2024-03-21 12:49:44
+Log started: 2024-03-21 13:32:29
-Preparing to unpack .../sosreport_4.4-1ubuntu0.18.04.1_amd64.deb ...
-Unpacking sosreport (4.4-1ubuntu0.18.04.1) over (3.9.1-1ubuntu0.16.04.2) ...
+Preparing to unpack .../sosreport_4.4-1ubuntu0.18.04.2_amd64.deb ...
+Unpacking sosreport (4.4-1ubuntu0.18.04.2) over (3.9.1-1ubuntu0.16.04.3) ...
-File descriptor 3 (pipe:[100507]) leaked on lvs invocation. Parent PID 9320: /bin/sh
+File descriptor 3 (pipe:[104562]) leaked on lvs invocation. Parent PID 9969: /bin/sh
-Setting up sosreport (4.4-1ubuntu0.18.04.1) ...
+Setting up sosreport (4.4-1ubuntu0.18.04.2) ...
-Replacing config file /etc/ssh/sshd_config with new version
+Replacing config file /etc/ssh/sshd_config with new version
-Local time is now: Thu Mar 21 13:03:00 UTC 2024.
-Universal Time is now: Thu Mar 21 13:03:00 UTC 2024.
+Local time is now: Thu Mar 21 13:43:40 UTC 2024.
+Universal Time is now: Thu Mar 21 13:43:40 UTC 2024.
-Log ended: 2024-03-21 13:06:10
+Log ended: 2024-03-21 13:45:49
-Log started: 2024-03-21 13:09:03
+Log started: 2024-03-21 14:07:58
-Log ended: 2024-03-21 13:09:27
+Log ended: 2024-03-21 14:08:14

Hi Arif,

Thank you very much!

I'm attaching the pastebin contents for archival purposes (pastebins may disappear).

I can confirm there are no changes on the install order of python or other packages.
(filtered diff [1])

AFAICT, previous concerns were addressed (ordering of package upgrades due to python
versioned dependency, in comment #3), so it looks good, and this is a proper fix vs.
the workaround in comment #10, as the python language features in the package should
be reflected in the dependencies.

So, let's proceed with the SRU!

One of the requirements is the newer releases (supported & devel) are fixed.
Can you please provide the debdiffs for them, for us to proceed?

$ for release in noble mantic jammy focal bionic; do git show pkg/ubuntu/${release}-devel:debian/control | grep 'X-Python-Version:'; done
  $

I think that the strictly minimal Python version (ie, based on language features)
is not necessarily what has to be figured out and included, but just the lowest
Python version supported per release (e.g., `py3versions --min-supported`),
so as to ensure that it (or a later version) is always installed on that release
or on upgrade to it when sosreport is installed.
(And of course, if a language feature of a Python version newer than the lowest
supported is needed, than that should be used instead. :)

Thanks again for all the work and help on fixing this issue!

[1]

$ grep '^[+-]' /tmp/diff_apt-term_log.diff | grep -v 'Reading database'
--- xenial01_logs/log/dist-upgrade/apt-term.log	2024-03-21 14:16:41.306113621 +0000
+++ xenial02_logs/log/dist-upgrade/apt-term.log	2024-03-21 14:16:58.374173593 +0000
-Log started: 2024-03-21  12:48:08
-Log ended: 2024-03-21  12:48:11
+Log started: 2024-03-21  13:31:32
+Log ended: 2024-03-21  13:31:34
-Log started: 2024-03-21  12:48:16
+Log started: 2024-03-21  13:31:37
-Log ended: 2024-03-21  12:48:27
+Log ended: 2024-03-21  13:31:44
-Log started: 2024-03-21  12:49:44
+Log started: 2024-03-21  13:32:29
-Preparing to unpack .../sosreport_4.4-1ubuntu0.18.04.1_amd64.deb ...
-Unpacking sosreport (4.4-1ubuntu0.18.04.1) over (3.9.1-1ubuntu0.16.04.2) ...
+Preparing to unpack .../sosreport_4.4-1ubuntu0.18.04.2_amd64.deb ...
+Unpacking sosreport (4.4-1ubuntu0.18.04.2) over (3.9.1-1ubuntu0.16.04.3) ...
-File descriptor 3 (pipe:[100507]) leaked on lvs invocation. Parent PID 9320: /bin/sh
+File descriptor 3 (pipe:[104562]) leaked on lvs invocation. Parent PID 9969: /bin/sh
-Setting up sosreport (4.4-1ubuntu0.18.04.1) ...
+Setting up sosreport (4.4-1ubuntu0.18.04.2) ...
-Replacing config file /etc/ssh/sshd_config with new version
+Replacing config file /etc/ssh/sshd_config with new version
-Local time is now:      Thu Mar 21 13:03:00 UTC 2024.
-Universal Time is now:  Thu Mar 21 13:03:00 UTC 2024.
+Local time is now:      Thu Mar 21 13:43:40 UTC 2024.
+Universal Time is now:  Thu Mar 21 13:43:40 UTC 2024.
-Log ended: 2024-03-21  13:06:10
+Log ended: 2024-03-21  13:45:49
-Log started: 2024-03-21  13:09:03
+Log started: 2024-03-21  14:07:58
-Log ended: 2024-03-21  13:09:27
+Log ended: 2024-03-21  14:08:14

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-03-25 (last edit on 2024-03-25):

#32

> Installed a xenial machine, changed /etc/apt/sources.[l]ist to point to
> bionic, and then just install sosreport. This reports the same problem
> as reported by OP.

Clarifying, that is the same symptom; the problem is on do-release-upgrade
(per description), but this could not be reproducible by any means by many
of us in this bug (as you are aware and helping with).

Upgrading a release (or individual packages across releases) like that is
not supported, as do-release-upgrade does a bunch of work for the upgrade
(including changing apt sources list at the right time), but it looks like
the best reproducer, and apparently related to what the bug reporter might
have done (thanks!), but we could never get a confirmation or more info.

So, that will definitely be helpful to verify the SRU in -proposed.

Thanks again!

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-26:

#33

> One of the requirements is the newer releases (supported & devel) are fixed.
> Can you please provide the debdiffs for them, for us to proceed?
>
> $ for release in noble mantic jammy focal bionic; do git show pkg/ubuntu/${release}-devel:debian/control | grep 'X-Python-Version:'; done
> $

The noble, mantic, jammy and focal builds will have the X-Python3-version sorted in the 4.7.0 SRU which is currently in progress in LP #2054395. Once that is done, all packages will have this there. I hope that suffices

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-03-26:

#34

Hi Arif,

That's good news, thanks!

Yes, but I see the debdiffs for all releases have `X-Python3-Version: >= 3.6`.

Could you please confirm that has been verified with `py3compile`, so as to
make sure this bug is not hit again in newer releases?

(And to be honest, I'm afraid that if it's set like a hard-coded value, this
bug may just be hit again in the future, when just meeting that version dep
is not going to be enough again -- so something dynamic would be preferred.)

I had in mind something like comment #31, to ensure python3 from the newer
release (upgraded to) is installed before sosreport on release upgrade, so
it does not `py3compile` with python from the older release (upgraded from).

But, as mentioned, if the strictly right version is python 3.6, and you want
to keep that, it is fine. *However*, we have to cover this issue in testing
for sosreport SRUs (i.e., whether the new package can install fine/py3compile
with the python version from the previous/upgraded-from stable release _and_
previous LTS release).

Please let us know which route is preferred by sosreport in Ubuntu.

Thanks!

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-03-26:

#35

Hi Mauricio,

The version that sosreport currently supports is 3.6 and there are plans to
potentially move on from 3.6 for the next major release of sosreport. My current
idea and plan was, when upstream make that change we would then update the
X-Python3-Version based on the upstream python compatibility.

So, when 4.8.0 comes out in August, we plan to update that value to 3.8, as th
upstream project is planning to move on. This will ensure that the compatibility
of this future package will ensure that we have the right version of python, and
we are unlikely to be SRU'ing back to bionic, but will ensure that any future
upgrades, i.e. bionic to focal would work as the new package will depend on the
newer python version when we get there and hence we will not see this particular
issue.

I hope that makes sense.

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-04-04:

#36

Hi Arif,

Cool. So, the python version is a coordinated decision at the upstream project level -- that looks good.

Thanks for clarifying!

Arif Ali (arif-ali) on 2024-04-26

Changed in sosreport (Ubuntu Bionic):
status:	Incomplete → In Progress

Arif Ali (arif-ali) on 2024-05-14

description:

updated

Mauricio Faria de Oliveira (mfo) on 2024-06-19

description:	updated
description:	updated

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-06-19:

#37

Hi Arif,

I've reviewed, updated SRU template, and built packages in a PPA [1].

It looks good, but there's an apparently unrelated change: some locale
files are now in the sosreport deb package (pasted below).

Could you please take a look?

Thanks!
Mauricio

[1] https://launchpad.net/~mfo/+archive/ubuntu/lp2038648

$ pull-lp-debs sosreport bionic
Found sosreport 4.4-1ubuntu0.18.04.1 in bionic
...

$ pull-ppa-debs --ppa mfo/lp2038648 sosreport bionic
Found sosreport 4.4-1ubuntu0.18.04.2 in bionic
...

$ debdiff sosreport_4.4-1ubuntu0.18.04.{1,2}_amd64.deb
[The following lists of changes regard files as different if they have
different names, permissions or owners.]

Files in second .deb but not in first
-------------------------------------
-rw-r--r-- root/root /usr/share/locale/af/LC_MESSAGES/sos.mo
...
-rw-r--r-- root/root /usr/share/locale/zu/LC_MESSAGES/sos.mo

Control files: lines which differ (wdiff format)
------------------------------------------------
Depends: python3-pexpect, python3-yaml, python3:any (>= [-3.3.2-2~)-] {+3.6~)+}
Installed-Size: [-2764-] {+2095+}
Maintainer: [-Nikhil Kshirsagar <email address hidden>-] {+Ubuntu Developers <email address hidden>+}
Version: [-4.4-1ubuntu0.18.04.1-] {+4.4-1ubuntu0.18.04.2+}

Changed in sosreport (Ubuntu Bionic):
status:	In Progress → Incomplete

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-06-19:

#38

Hi Mauricio,

I recreated the deb from the original control file as well as with the new Version option. Both debs now produced have the /usr/share/locale/*/LC_MESSAGES/sos.mo files. So I suspect this may be something that ma have been introduced as part of a different package.

Here is a debdiff of 2 packages that I created locally

root@bionic01:~/bionic# debdiff sosreport_4.4-1ubuntu0.18.04.2_amd64.deb sosreport_4.4-1ubuntu0.18.04.3_amd64.deb
File lists identical (after any substitutions)

Control files: lines which differ (wdiff format)
------------------------------------------------
Depends: python3-pexpect, python3-yaml, python3:any (>= [-3.3.2-2~)-] {+3.6~)+}
Version: [-4.4-1ubuntu0.18.04.2-] {+4.4-1ubuntu0.18.04.3+}

Doing `dpkg-deb --contents <deb-file>` to both packages now show the locale files

I think this is unrelated to the change I have made here in my debdiff

Revision history for this message

Lukas Märdian (slyon) wrote on 2024-06-25 (last edit on 2024-06-25):

#39

After working my way through bug #2054395, I'm also sponsoring this, even tough Bionic is end-of-standard-support already. I'll leave it to the SRU team to accept or reject it.

The change generally LGTM. I applied some cosmetic whitespace fixes to d/changelog.

Unsubscribing ~ubuntu-sponsors.

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-06-25:

#40

Hi Lukas,

Thanks for the assistance in sponsoring!

I had other changes to make in the upload (in the PPA),
so I'll reject this one and continue with that; sorry,
I should have made that clearer in my previous comment.

cheers,

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-06-25:

#41

Hi Arif,

Thanks for checking and confirming the locale changes are unrelated!

I suspected the locale difference were unrelated, but wanted a closer
look and clear understanding of why, before uploading, to ensure our
changes to Bionic demonstrate the required care to request an upload
in the exception criteria (to the Ubuntu archive for an ESM release).

I looked into this a bit more, and tracked these "new" locale files
to a behavior change in pkgstriptranslations (src:pkgbinarymangler)
if running on a PPA (do not strip), so the locale files are kept:
`INFO: Disabling pkgstriptranslations for PPA build`

So, for the regular/archive build (non-PPA), we expect the locale
files not to be present (as their strip should happen), therefore
their delta should drop -- and only expected changes would remain.

I confirmed this on a local build, by setting 'enable: true' in
`/etc/pkgbinarymangler/striptranslations.conf` and building the
package. The 'Disabling ... for PPA build' line is not present,
nor are the locale files:

$ dpkg-buildpackage -b -uc 2>&1 | tee ../build.log
$ grep pkgstriptranslations ../build.log
INFO: pkgstriptranslations version 138.18.04.2
pkgstriptranslations: processing sosreport (in debian/sosreport); do_strip: 1, oemstrip:
pkgstriptranslations: preparing translation tarball sosreport_4.4-1ubuntu0.18.04.2_amd64_translations.tar.gz...done
$ dpkg-deb -c ../sosreport_4.4-1ubuntu0.18.04.2_amd64.deb | grep locale
$

Uploading to Bionic.

cheers,
Mauricio

...

https://manpages.ubuntu.com/manpages/bionic/en/man1/pkgstriptranslations.1.html

This script also removes all gettext *.mo files below <pkg-build-dir>/usr/share/locale from all built binary packages.

...

$ curl -sL 'https://launchpadlibrarian.net/627169149/buildlog_ubuntu-bionic-amd64.sosreport_4.4-1ubuntu0.18.04.1_BUILDING.txt.gz' -o - | zcat >old.log
$ curl -sL 'https://launchpadlibrarian.net/735903734/buildlog_ubuntu-bionic-amd64.sosreport_4.4-1ubuntu0.18.04.2_BUILDING.txt.gz' -o - | zcat >new.log

$ grep pkgstriptranslations old.log new.log

old.log:INFO: pkgstriptranslations version 138.18.04.1
old.log:pkgstriptranslations: processing sosreport (in debian/sosreport); do_strip: 1, oemstrip:
old.log:pkgstriptranslations: preparing translation tarball sosreport_4.4-1ubuntu0.18.04.1_amd64_translations.tar.gz...done

new.log:INFO: pkgstriptranslations version 138.18.04.2
new.log:INFO: Disabling pkgstriptranslations for PPA build

Hi Arif,

Thanks for checking and confirming the locale changes are unrelated!

I suspected the locale difference were unrelated, but wanted a closer
look and clear understanding of why, before uploading, to ensure our
changes to Bionic demonstrate the required care to request an upload
in the exception criteria (to the Ubuntu archive for an ESM release).

I looked into this a bit more, and tracked these "new" locale files
to a behavior change in pkgstriptranslations (src:pkgbinarymangler)
if running on a PPA (do not strip), so the locale files are kept:
`INFO: Disabling pkgstriptranslations for PPA build`

So, for the regular/archive build (non-PPA), we expect the locale
files not to be present (as their strip should happen), therefore
their delta should drop -- and only expected changes would remain.

I confirmed this on a local build, by setting 'enable: true' in
`/etc/pkgbinarymangler/striptranslations.conf` and building the
package. The 'Disabling ... for PPA build' line is not present,
nor are the locale files:

$ dpkg-buildpackage -b -uc 2>&1 | tee ../build.log
$ grep pkgstriptranslations ../build.log
INFO: pkgstriptranslations version 138.18.04.2
pkgstriptranslations: processing sosreport (in debian/sosreport); do_strip: 1, oemstrip:
pkgstriptranslations: preparing translation tarball sosreport_4.4-1ubuntu0.18.04.2_amd64_translations.tar.gz...done
$ dpkg-deb -c ../sosreport_4.4-1ubuntu0.18.04.2_amd64.deb | grep locale
$

Uploading to Bionic.

cheers,
Mauricio

...

https://manpages.ubuntu.com/manpages/bionic/en/man1/pkgstriptranslations.1.html

This  script  also  removes  all gettext *.mo files below <pkg-build-dir>/usr/share/locale from all built binary packages.

...

$ curl -sL 'https://launchpadlibrarian.net/627169149/buildlog_ubuntu-bionic-amd64.sosreport_4.4-1ubuntu0.18.04.1_BUILDING.txt.gz' -o - | zcat >old.log
$ curl -sL 'https://launchpadlibrarian.net/735903734/buildlog_ubuntu-bionic-amd64.sosreport_4.4-1ubuntu0.18.04.2_BUILDING.txt.gz' -o - | zcat >new.log

$ grep pkgstriptranslations old.log new.log

old.log:INFO: pkgstriptranslations version 138.18.04.1
old.log:pkgstriptranslations: processing sosreport (in debian/sosreport); do_strip: 1, oemstrip:
old.log:pkgstriptranslations: preparing translation tarball sosreport_4.4-1ubuntu0.18.04.1_amd64_translations.tar.gz...done

new.log:INFO: pkgstriptranslations version 138.18.04.2
new.log:INFO: Disabling pkgstriptranslations for PPA build

Mauricio Faria de Oliveira (mfo) on 2024-06-26

Changed in sosreport (Ubuntu Bionic):
status:	Incomplete → In Progress

Revision history for this message

Brian Murray (brian-murray) wrote on 2024-07-02: Please test proposed package

#42

Hello Branimir, or anyone else affected,

Accepted sosreport into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.4-1ubuntu0.18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Bionic):
status:	In Progress → Fix Committed
tags:	added: verification-needed verification-needed-bionic

Revision history for this message

Arif Ali (arif-ali) wrote on 2024-07-03:

#43

xenial-bionic-sos-update-test.txt Edit (194.0 KiB, text/plain)

tested the package in the same as before, and compared to the PPA I created as well

1. First test

normal do-release-upgrade with the -proposed pocket enabled, no issues

2. Second Test

* Add proposed
* Change all xenial to bionic in all apt sources
* Upgrade sosreport

Encountered no issues; this was the only test case that was failing; so all good from my perspective.

I have attached my full log of what I did as well for reference

tags:

added: verification-done-bionic
removed: verification-needed-bionic

Revision history for this message

Mauricio Faria de Oliveira (mfo) wrote on 2024-07-03:

#44

Hi Arif,

Thanks for testing!

Could you please also check/confirm the steps 3 and 4 of the Test Plan,
as we did in previous comments for testing, per the point in comment 3?

tags:

added: verification-needed-bionic
removed: verification-done-bionic

Ubuntu
sosreport package

package sosreport 4.4-1ubuntu0.18.04.1 failed to install/upgrade: installed sosreport package post-installation script subprocess returned error exit status 1

Bug Description

Other bug subscribers

Patches

Bug attachments

Remote bug watches

Affects		Status	Importance	Assigned to	Milestone
	sosreport (Ubuntu)	Invalid	Undecided	Unassigned
	Bionic	Fix Committed	High	Arif Ali

Ubuntusosreport package

package sosreport 4.4-1ubuntu0.18.04.1 failed to install/upgrade: installed sosreport package post-installation script subprocess returned error exit status 1

Bug Description

Other bug subscribers

Patches

Bug attachments

Remote bug watches

Ubuntu
sosreport package