duplicate signature for Package problems can be too short

Bug #1692127 reported by Brian Murray on 2017-05-19
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
apport (Ubuntu)
Undecided
Unassigned
Xenial
Undecided
Brian Murray
Yakkety
Undecided
Brian Murray
Zesty
High
Brian Murray
Artful
Undecided
Unassigned

Bug Description

[Impact]
apport is creating a shorter than appropriate duplicate signature in apport-package bug reports which can lead to misconsolidation and confused developers.

[Test Case]
1) Boot a system with an old kernel installed or a freshly installed system
2) edit /etc/default/grub and replace a ` with a '
3) Upgrade the kernel, watch it crash
4) Discover a crash report with a DuplicateSignature whose 2nd line starts with "Examining"

With the version of the package from -proposed the DuplicateSignature in this case should really have a 2nd line that starts with 'Setting up linux-image* ...'. While in this particular case not much is lost in other cases valuable information will be missing.

[Regression Potential]
The code change is correct and has been tested in artful, additionally I'm using the same regular expression to clean up apport-package reports as they come in to LP. The only risk here is a typo in the patch that would cause the hook (the change is in the ubuntu general hook) to error out which apport would handle gracefully and we just wouldn't have a DuplicateSignature.

[The Original Description]
Steve brought up bug 1691983 and how it's duplicate signature is missing information. It contains:

package:shim-signed:1.27~16.04.1+0.9+1474479173.6c180c6-1ubuntu1
Installing for x86_64-efi platform.
Installation finished. No error reported.
Running in non-interactive mode, doing nothing.
dpkg: error processing package shim-signed (--configure):
 subprocess installed post-installation script returned error exit status 1

But is missing:

Setting up shim-signed (1.27~16.04.1+0.9+1474479173.6c180c6-1ubuntu1) ...
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
debconf: unable to initialize frontend: Passthrough
debconf: (Cannot connect to /tmp/aptdaemon-7jrqb_du/debconf.socket: Connection refused at (eval 18) line 3.)
debconf: falling back to frontend: Noninteractive

This is because of the following code in data/general-hooks/ubuntu.py:

145 PKG_MSGS = ('Authenticating', 'De-configuring', 'Examining',
146 'Installing ', 'Preparing', 'Processing triggers', 'Purging',
147 'Removing', 'Replaced', 'Replacing', 'Setting up',
148 'Unpacking', 'Would remove')
149 for line in termlog.split('\n'):
150 if line.startswith(PKG_MSGS):
151 dupe_sig = '%s\n' % line
152 continue

The shim-signed package prints a line starts with "Installing " and that matches PKG_MSGS so we reset the dupe_sig because that's a message we'd expect from a package manager.

Brian Murray (brian-murray) wrote :

We could continue if the line equals "Installing for x86_64-efi platform." but that would only work for shim-signed and seems fragile.

tags: added: artful xenial yakkety zesty
Steve Langasek (vorlon) wrote :

When each of these messages are output by dpkg, they also match the pattern '\.\.\.[[:space:]]*$'. So perhaps that should be included as part of the filter?

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.5-0ubuntu4

---------------
apport (2.20.5-0ubuntu4) artful; urgency=medium

  * data/general-hooks/ubuntu.py: Modify how a duplicate signature is created
    for package installation failures. (LP: #1692127)

 -- Brian Murray <email address hidden> Mon, 22 May 2017 16:37:50 -0700

Changed in apport (Ubuntu Artful):
status: New → Fix Released
Brian Murray (brian-murray) wrote :

This should be SRU'ed to stable releases but let's make sure it is working properly first.

Changed in apport (Ubuntu Zesty):
status: New → Triaged
importance: Undecided → High
assignee: nobody → Brian Murray (brian-murray)
Brian Murray (brian-murray) wrote :

Trusty uses the old method of creating a duplicate signature so does not require this fix.

104 # build a duplicate signature tag for package reports
105 if report.get('ProblemType') == 'Package' and 'ErrorMessage' in report and 'Package' in report:
106 (package, version) = report['Package'].split(None, 1)
107 dupe_sig = 'package:%s:%s:%s' % (package, version, report['ErrorMessage'])
108 report['DuplicateSignature'] = dupe_sig

Changed in apport (Ubuntu Yakkety):
status: New → Triaged
Changed in apport (Ubuntu Xenial):
status: New → Triaged
description: updated
Changed in apport (Ubuntu Yakkety):
assignee: nobody → Brian Murray (brian-murray)
Changed in apport (Ubuntu Xenial):
assignee: nobody → Brian Murray (brian-murray)
status: Triaged → In Progress
Changed in apport (Ubuntu Yakkety):
status: Triaged → In Progress
Changed in apport (Ubuntu Zesty):
status: Triaged → In Progress

Hello Brian, or anyone else affected,

Accepted apport into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.4-0ubuntu4.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

description: updated
Changed in apport (Ubuntu Zesty):
status: In Progress → Fix Committed
tags: added: verification-needed
Changed in apport (Ubuntu Yakkety):
status: In Progress → Fix Committed
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.3-0ubuntu8.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apport (Ubuntu Xenial):
status: In Progress → Fix Committed
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.1-0ubuntu2.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Brian Murray (brian-murray) wrote :

The upload of apport which fixed this bug did not cause but revealed an autopkgtest failure with test_signal_crashes.py in apport. The tests test_core_dump_packaged and test_core_dump_unpackaged are failing because the core file being produced is greater than the ulimit set for core files. This limit is arbitrary and I've bumped it from 1 million to 10 million bytes to allow the test pass.

As part of a recent change in the Stable Release Update verification policy we would like to inform that for a bug to be considered verified for a given release a verification-done-$RELEASE tag needs to be added to the bug where $RELEASE is the name of the series the package that was tested (e.g. verification-done-xenial). Please note that the global 'verification-done' tag can no longer be used for this purpose.

Thank you!

Hello Brian, or anyone else affected,

Accepted apport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.1-0ubuntu2.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Adam Conrad (adconrad) wrote :

Hello Brian, or anyone else affected,

Accepted apport into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.3-0ubuntu8.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Adam Conrad (adconrad) wrote :

Hello Brian, or anyone else affected,

Accepted apport into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.4-0ubuntu4.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.4-0ubuntu4.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed-zesty
tags: added: verification-needed-yakkety
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.3-0ubuntu8.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-yakkety to verification-done-yakkety.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-yakkety. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed-xenial
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.1-0ubuntu2.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Brian Murray (brian-murray) wrote :

This worked for me in zesty with apport version 2.20.4-0ubuntu4.4:

DuplicateSignature:
 package:linux-image-4.10.0-22-generic:4.10.0-22.24
 Setting up linux-image-4.10.0-22-generic (4.10.0-22.24) ...
 Running depmod.
 update-initramfs: deferring update (hook will be called later)
 Not updating initrd symbolic links since we are being updated/reinstalled
 (4.10.0-22.24 was configured last, according to dpkg)
 Not updating image symbolic links since we are being updated/reinstalled
 (4.10.0-22.24 was configured last, according to dpkg)
 Examining /etc/kernel/postinst.d.
 run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/dkms 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 update-initramfs: Generating /boot/initrd.img-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/pm-utils 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/update-notifier 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 /usr/sbin/grub-mkconfig: 15: /etc/default/grub: Syntax error: Unterminated quoted string
 run-parts: /etc/kernel/postinst.d/zz-update-grub exited with return code 2
 Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.10.0-22-generic.postinst line 1052.
 dpkg: error processing package linux-image-4.10.0-22-generic (--configure):
  subprocess installed post-installation script returned error exit status 2

tags: added: verification-done-zesty
removed: verification-needed-zesty
Brian Murray (brian-murray) wrote :

This also works for me on yakkety with apport version 2.20.3-0ubuntu8.6:

Setting up linux-image-4.8.0-58-generic (4.8.0-58.63) ...
Running depmod.
update-initramfs: deferring update (hook will be called later)
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
update-initramfs: Generating /boot/initrd.img-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/pm-utils 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
/usr/sbin/grub-mkconfig: 15: /etc/default/grub: Syntax error: Unterminated quoted string
run-parts: /etc/kernel/postinst.d/zz-update-grub exited with return code 2
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.8.0-58-generic.postinst line 1052.
dpkg: error processing package linux-image-4.8.0-58-generic (--configure):
 subprocess installed post-installation script returned error exit status 2

tags: added: verification-done-yakkety
removed: verification-needed-yakkety
Brian Murray (brian-murray) wrote :

This also works for me on xenial with apport version 2.20.1-0ubuntu2.9:

== DuplicateSignature =================================
package:linux-image-4.4.0-84-generic:4.4.0-84.107
Setting up linux-image-4.4.0-84-generic (4.4.0-84.107) ...
Running depmod.
update-initramfs: deferring update (hook will be called later)
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/pm-utils 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
/usr/sbin/grub-mkconfig: 15: /etc/default/grub: Syntax error: Unterminated quoted string
run-parts: /etc/kernel/postinst.d/zz-update-grub exited with return code 2
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.4.0-84-generic.postinst line 1052.
dpkg: error processing package linux-image-4.4.0-84-generic (--configure):
 subprocess installed post-installation script returned error exit status 2

tags: added: verification-done-xenial
removed: verification-needed verification-needed-xenial
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.4-0ubuntu4.4

---------------
apport (2.20.4-0ubuntu4.4) zesty; urgency=medium

  * test/test_signal_crashes.py: delete the test which uses an arbitrary
    unpredictable core file size.

apport (2.20.4-0ubuntu4.3) zesty; urgency=medium

  * test/test_signal_crashes.py: a ulimit of 1M bytes isn't enough to produce
    a core file anymore so bump it to 10M.

apport (2.20.4-0ubuntu4.2) zesty; urgency=medium

  * data/general-hooks/ubuntu.py: Modify how a duplicate signature is created
    for package installation failures. (LP: #1692127)

 -- Brian Murray <email address hidden> Thu, 29 Jun 2017 11:41:12 -0700

Changed in apport (Ubuntu Zesty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for apport has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.3-0ubuntu8.6

---------------
apport (2.20.3-0ubuntu8.6) yakkety; urgency=medium

  * test/test_signal_crashes.py: delete the test which uses an arbitrary
    unpredictable core file size.

apport (2.20.3-0ubuntu8.5) yakkety; urgency=medium

  * test/test_signal_crashes.py: a ulimit of 1M bytes isn't enough to produce
    a core file anymore so bump it to 10M.

apport (2.20.3-0ubuntu8.4) yakkety; urgency=medium

  * data/general-hooks/ubuntu.py: Modify how a duplicate signature is created
    for package installation failures. (LP: #1692127)

 -- Brian Murray <email address hidden> Thu, 29 Jun 2017 13:30:39 -0700

Changed in apport (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.1-0ubuntu2.9

---------------
apport (2.20.1-0ubuntu2.9) xenial; urgency=medium

  * test/test_signal_crashes.py: delete the test which uses an arbitrary
    unpredictable core file size.

apport (2.20.1-0ubuntu2.8) xenial; urgency=medium

  * test/test_signal_crashes.py: a ulimit of 1M bytes isn't enough to produce
    a core file anymore so bump it to 10M.

apport (2.20.1-0ubuntu2.7) xenial; urgency=medium

  * data/general-hooks/ubuntu.py: Modify how a duplicate signature is created
    for package installation failures. (LP: #1692127)

 -- Brian Murray <email address hidden> Thu, 29 Jun 2017 13:33:50 -0700

Changed in apport (Ubuntu Xenial):
status: Fix Committed → Fix Released
tags: added: id-5956aefd1fbc2cddbec3dbc9
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers