| 2018-07-17 08:34:12 |
Balint Reczey |
bug |
|
|
added bug |
| 2018-07-17 09:13:29 |
Balint Reczey |
description |
[Impact]
* PNG optimization and changelog truncation can happen in parallel with symlinking docs across packages. When checking for duplicates files the duplicate check may compare the processed files with non-processed ones and in that case they are found to be different despite the processed (or the unprocessed) files would be identical. The resulting binary packages would be still valid, but may be different for different architectures since the parallel build process may hit this race condition when processing different files making the arch:any packages not coinstallable on a multiarch-enabled system.
* One manifestation of the problem can be seen in https://launchpadlibrarian.net/355140796/buildlog_ubuntu-bionic-i386.gpm_1.20.7-5_BUILDING.txt.gz where symlinking's file comparison occurs while truncating the changelog:
...
Searching for duplicated docs in dependency libgpm2...
pkgstripfiles: Truncating usr/share/doc/libgpm2/changelog.Debian.gz to topmost ten records
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
cmp: EOF on /dev/fd/62 which is empty
...
* The fix is protecting png optimization and trucating changelogs with the same lock as used to serialize doc symlinking thus those steps are serializes as well.
[Test Case]
* Build an affected package (like gpm) for i386 and amd64 with fixed pkgbinarymangler and observe the files in /usr/share/doc in the built packages to be identical and there should be no line like the following in the build log:
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
* Note that due to the non-deterministic nature of the failure several builds could be necessary to reproduce the original problem and also for verifying that the problem is fixed.
[Regression Potential]
* Due to serialization the build times may increase for packages where the serialized steps ran in parallel originally. IMO there is not much we can do about that apart from trying to speed up the steps themselves.
* Since no additional locks were introduced I believe the builds won't break or stall due to this change.
[Other Info]
* There are some packages which need to be rebuilt with the updated pkgbinarymangler, collecting them is in progress. |
[Impact]
* PNG optimization and changelog truncation can happen in parallel with symlinking docs across packages. When checking for duplicates files the duplicate check may compare the processed files with non-processed ones and in that case they are found to be different despite the processed (or the unprocessed) files would be identical. The resulting binary packages would be still valid, but may be different for different architectures since the parallel build process may hit this race condition when processing different files making the arch:any packages not coinstallable on a multiarch-enabled system.
* One manifestation of the problem can be seen in https://launchpadlibrarian.net/355140796/buildlog_ubuntu-bionic-i386.gpm_1.20.7-5_BUILDING.txt.gz where symlinking's file comparison occurs while truncating the changelog:
...
Searching for duplicated docs in dependency libgpm2...
pkgstripfiles: Truncating usr/share/doc/libgpm2/changelog.Debian.gz to topmost ten records
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
cmp: EOF on /dev/fd/62 which is empty
...
* The fix is protecting png optimization and trucating changelogs with the same lock as used to serialize doc symlinking thus those steps are serializes as well.
[Test Case]
* Build an affected package (like gpm) for i386 and amd64 with fixed pkgbinarymangler and observe the files in /usr/share/doc in the built packages to be identical and there should be no line like the following in the build log:
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
* Note that due to the non-deterministic nature of the failure several builds could be necessary to reproduce the original problem and also for verifying that the problem is fixed. You can increase the chances of observing the failure by building the packages with higher level of parallelism (-jX).
[Regression Potential]
* Due to serialization the build times may increase for packages where the serialized steps ran in parallel originally. IMO there is not much we can do about that apart from trying to speed up the steps themselves.
* Since no additional locks were introduced I believe the builds won't break or stall due to this change.
[Other Info]
* There are some packages which need to be rebuilt with the updated pkgbinarymangler, collecting them is in progress. |
|
| 2018-07-17 12:36:24 |
Francis Ginther |
tags |
|
id-5b2bdda1b1a060708081efe3 |
|
| 2018-07-17 15:32:38 |
Launchpad Janitor |
pkgbinarymangler (Ubuntu): status |
New |
Fix Released |
|
| 2018-07-18 08:05:24 |
Balint Reczey |
description |
[Impact]
* PNG optimization and changelog truncation can happen in parallel with symlinking docs across packages. When checking for duplicates files the duplicate check may compare the processed files with non-processed ones and in that case they are found to be different despite the processed (or the unprocessed) files would be identical. The resulting binary packages would be still valid, but may be different for different architectures since the parallel build process may hit this race condition when processing different files making the arch:any packages not coinstallable on a multiarch-enabled system.
* One manifestation of the problem can be seen in https://launchpadlibrarian.net/355140796/buildlog_ubuntu-bionic-i386.gpm_1.20.7-5_BUILDING.txt.gz where symlinking's file comparison occurs while truncating the changelog:
...
Searching for duplicated docs in dependency libgpm2...
pkgstripfiles: Truncating usr/share/doc/libgpm2/changelog.Debian.gz to topmost ten records
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
cmp: EOF on /dev/fd/62 which is empty
...
* The fix is protecting png optimization and trucating changelogs with the same lock as used to serialize doc symlinking thus those steps are serializes as well.
[Test Case]
* Build an affected package (like gpm) for i386 and amd64 with fixed pkgbinarymangler and observe the files in /usr/share/doc in the built packages to be identical and there should be no line like the following in the build log:
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
* Note that due to the non-deterministic nature of the failure several builds could be necessary to reproduce the original problem and also for verifying that the problem is fixed. You can increase the chances of observing the failure by building the packages with higher level of parallelism (-jX).
[Regression Potential]
* Due to serialization the build times may increase for packages where the serialized steps ran in parallel originally. IMO there is not much we can do about that apart from trying to speed up the steps themselves.
* Since no additional locks were introduced I believe the builds won't break or stall due to this change.
[Other Info]
* There are some packages which need to be rebuilt with the updated pkgbinarymangler, collecting them is in progress. |
[Impact]
* PNG optimization and changelog truncation can happen in parallel with symlinking docs across packages. When checking for duplicates files the duplicate check may compare the processed files with non-processed ones and in that case they are found to be different despite the processed (or the unprocessed) files would be identical. The resulting binary packages would be still valid, but may be different for different architectures since the parallel build process may hit this race condition when processing different files making the arch:any packages not coinstallable on a multiarch-enabled system.
* One manifestation of the problem can be seen in https://launchpadlibrarian.net/355140796/buildlog_ubuntu-bionic-i386.gpm_1.20.7-5_BUILDING.txt.gz where symlinking's file comparison occurs while truncating the changelog:
...
Searching for duplicated docs in dependency libgpm2...
pkgstripfiles: Truncating usr/share/doc/libgpm2/changelog.Debian.gz to topmost ten records
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
cmp: EOF on /dev/fd/62 which is empty
...
* The fix is protecting png optimization and trucating changelogs with the same lock as used to serialize doc symlinking thus those steps are serializes as well. The fix also skips optimizing PNGs and truncating changelogs of dbgsym files in pkgstripfiles.
[Test Case]
* Build an affected package (like gpm) for i386 and amd64 with fixed pkgbinarymangler and observe the files in /usr/share/doc in the built packages to be identical and there should be no line like the following in the build log:
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
* Note that due to the non-deterministic nature of the failure several builds could be necessary to reproduce the original problem and also for verifying that the problem is fixed. You can increase the chances of observing the failure by building the packages with higher level of parallelism (-jX).
[Regression Potential]
* Due to serialization the build times may increase for packages where the serialized steps ran in parallel originally. IMO there is not much we can do about that apart from trying to speed up the steps themselves.
* Since no additional locks were introduced I believe the builds won't break or stall due to this change.
* The fix also skips optimizing PNGs and truncating changelogs of dbgsym files, but PNG files are typically not present there and /usr/share/doc/<pkg>-dbgsym is symlinked to /usr/share/doc/<pkg> even with the fix thus the change of pkgstripfiles does not have an effect on the built dbgsym files' changelogs.
[Other Info]
* There are some packages which need to be rebuilt with the updated pkgbinarymangler, collecting them is in progress. |
|
| 2018-07-18 08:06:22 |
Balint Reczey |
description |
[Impact]
* PNG optimization and changelog truncation can happen in parallel with symlinking docs across packages. When checking for duplicates files the duplicate check may compare the processed files with non-processed ones and in that case they are found to be different despite the processed (or the unprocessed) files would be identical. The resulting binary packages would be still valid, but may be different for different architectures since the parallel build process may hit this race condition when processing different files making the arch:any packages not coinstallable on a multiarch-enabled system.
* One manifestation of the problem can be seen in https://launchpadlibrarian.net/355140796/buildlog_ubuntu-bionic-i386.gpm_1.20.7-5_BUILDING.txt.gz where symlinking's file comparison occurs while truncating the changelog:
...
Searching for duplicated docs in dependency libgpm2...
pkgstripfiles: Truncating usr/share/doc/libgpm2/changelog.Debian.gz to topmost ten records
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
cmp: EOF on /dev/fd/62 which is empty
...
* The fix is protecting png optimization and trucating changelogs with the same lock as used to serialize doc symlinking thus those steps are serializes as well. The fix also skips optimizing PNGs and truncating changelogs of dbgsym files in pkgstripfiles.
[Test Case]
* Build an affected package (like gpm) for i386 and amd64 with fixed pkgbinarymangler and observe the files in /usr/share/doc in the built packages to be identical and there should be no line like the following in the build log:
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
* Note that due to the non-deterministic nature of the failure several builds could be necessary to reproduce the original problem and also for verifying that the problem is fixed. You can increase the chances of observing the failure by building the packages with higher level of parallelism (-jX).
[Regression Potential]
* Due to serialization the build times may increase for packages where the serialized steps ran in parallel originally. IMO there is not much we can do about that apart from trying to speed up the steps themselves.
* Since no additional locks were introduced I believe the builds won't break or stall due to this change.
* The fix also skips optimizing PNGs and truncating changelogs of dbgsym files, but PNG files are typically not present there and /usr/share/doc/<pkg>-dbgsym is symlinked to /usr/share/doc/<pkg> even with the fix thus the change of pkgstripfiles does not have an effect on the built dbgsym files' changelogs.
[Other Info]
* There are some packages which need to be rebuilt with the updated pkgbinarymangler, collecting them is in progress. |
[Impact]
* PNG optimization and changelog truncation can happen in parallel with symlinking docs across packages. When checking for duplicates files the duplicate check may compare the processed files with non-processed ones and in that case they are found to be different despite the processed (or the unprocessed) files would be identical. The resulting binary packages would be still valid, but may be different for different architectures since the parallel build process may hit this race condition when processing different files making the arch:any packages not coinstallable on a multiarch-enabled system.
* One manifestation of the problem can be seen in https://launchpadlibrarian.net/355140796/buildlog_ubuntu-bionic-i386.gpm_1.20.7-5_BUILDING.txt.gz where symlinking's file comparison occurs while truncating the changelog:
...
Searching for duplicated docs in dependency libgpm2...
pkgstripfiles: Truncating usr/share/doc/libgpm2/changelog.Debian.gz to topmost ten records
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
cmp: EOF on /dev/fd/62 which is empty
...
* The fix is protecting png optimization and trucating changelogs with the same lock as used to serialize doc symlinking thus those steps are serialized as well. The fix also skips optimizing PNGs and truncating changelogs of dbgsym files in pkgstripfiles.
[Test Case]
* Build an affected package (like gpm) for i386 and amd64 with fixed pkgbinarymangler and observe the files in /usr/share/doc in the built packages to be identical and there should be no line like the following in the build log:
gzip: /<<PKGBUILDDIR>>/debian/libgpm-dev/../libgpm2/usr/share/doc/libgpm2/changelog.Debian.gz: unexpected end of file
* Note that due to the non-deterministic nature of the failure several builds could be necessary to reproduce the original problem and also for verifying that the problem is fixed. You can increase the chances of observing the failure by building the packages with higher level of parallelism (-jX).
[Regression Potential]
* Due to serialization the build times may increase for packages where the serialized steps ran in parallel originally. IMO there is not much we can do about that apart from trying to speed up the steps themselves.
* Since no additional locks were introduced I believe the builds won't break or stall due to this change.
* The fix also skips optimizing PNGs and truncating changelogs of dbgsym files, but PNG files are typically not present there and /usr/share/doc/<pkg>-dbgsym is symlinked to /usr/share/doc/<pkg> even with the fix thus the change of pkgstripfiles does not have an effect on the built dbgsym files' changelogs.
[Other Info]
* There are some packages which need to be rebuilt with the updated pkgbinarymangler, collecting them is in progress. |
|
| 2018-07-18 14:16:48 |
Robie Basak |
pkgbinarymangler (Ubuntu Bionic): status |
New |
Fix Committed |
|
| 2018-07-18 14:16:49 |
Robie Basak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2018-07-18 14:16:50 |
Robie Basak |
bug |
|
|
added subscriber SRU Verification |
| 2018-07-18 14:16:53 |
Robie Basak |
tags |
id-5b2bdda1b1a060708081efe3 |
id-5b2bdda1b1a060708081efe3 verification-needed verification-needed-bionic |
|
| 2018-07-20 15:47:09 |
Balint Reczey |
tags |
id-5b2bdda1b1a060708081efe3 verification-needed verification-needed-bionic |
id-5b2bdda1b1a060708081efe3 verification-done verification-done-bionic |
|
| 2018-07-23 17:44:33 |
Adam Conrad |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2018-07-23 17:44:36 |
Launchpad Janitor |
pkgbinarymangler (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
