PY_CFLAGS contains reference to -specs=/usr/share/dpkg/no-pie-compile.specs

Bug #1768644 reported by Anthony Sottile on 2018-05-02
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
python3.6 (Ubuntu)
Status tracked in Cosmic
Bionic
Undecided
Unassigned
Cosmic
Undecided
Unassigned

Bug Description

SRU:

Impact]
 * Causing a build failure

[Test Case]
 * See below. Or just check that the flag doesn't show
   up when building extensions.

[Regression Potential]

 * I don't see any. These flags are injected anyway
   in the package builds for extensions. It's only about
   virtualenv installations.

Minimal reproduction (docker):

FROM ubuntu:bionic
RUN apt-get update && \
    apt-get install -y --no-install-recommends gcc python-dev virtualenv && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*
RUN virtualenv /venv -ppython3 && \
    /venv/bin/pip install ujson

Fails with:

Collecting ujson
  Downloading https://files.pythonhosted.org/packages/16/c4/79f3409bc710559015464e5f49b9879430d8f87498ecdc335899732e5377/ujson-1.35.tar.gz (192kB)
Building wheels for collected packages: ujson
  Running setup.py bdist_wheel for ujson: started
  Running setup.py bdist_wheel for ujson: finished with status 'error'
  Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-x7e5igby/ujson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-de4xsrzr --python-tag cp36:
  running bdist_wheel
  running build
  running build_ext
  building 'ujson' extension
  creating build
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/python
  creating build/temp.linux-x86_64-3.6/lib
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./python -I./lib -I/usr/include/python3.6m -I/venv/include/python3.6m -c ./python/ujson.c -o build/temp.linux-x86_64-3.6/./python/ujson.o -D_GNU_SOURCE
  x86_64-linux-gnu-gcc: error: /usr/share/dpkg/no-pie-compile.specs: No such file or directory
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for ujson
  Running setup.py clean for ujson
Failed to build ujson
Installing collected packages: ujson
  Running setup.py install for ujson: started
    Running setup.py install for ujson: finished with status 'error'
    Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-x7e5igby/ujson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-d0l7cnq5/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python3.6/ujson:
    running install
    running build
    running build_ext
    building 'ujson' extension
    creating build
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/python
    creating build/temp.linux-x86_64-3.6/lib
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./python -I./lib -I/usr/include/python3.6m -I/venv/include/python3.6m -c ./python/ujson.c -o build/temp.linux-x86_64-3.6/./python/ujson.o -D_GNU_SOURCE
    x86_64-linux-gnu-gcc: error: /usr/share/dpkg/no-pie-compile.specs: No such file or directory
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-x7e5igby/ujson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-d0l7cnq5/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python3.6/ujson" failed with error code 1 in /tmp/pip-install-x7e5igby/ujson/
The command '/bin/sh -c virtualenv /venv -ppython3 && /venv/bin/pip install ujson' returned a non-zero code: 1

The root cause of this seems to be the PY_CFLAGS sysconfig setting which contains flags that were used to build python itself, including a reference to a file which doesn't exist:

root@71b2faa3da66:/# python3 -m sysconfig | grep no-pie-compile
 CFLAGS = "-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security"
 CONFIGURE_CFLAGS = "-g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security"
 CONFIG_ARGS = "'--enable-shared' '--prefix=/usr' '--enable-ipv6' '--enable-loadable-sqlite-extensions' '--with-dbmliborder=bdb:gdbm' '--with-computed-gotos' '--without-ensurepip' '--with-system-expat' '--with-system-libmpdec' '--with-system-ffi' 'CC=x86_64-linux-gnu-gcc' 'CFLAGS=-g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security ' 'LDFLAGS=-Wl,-Bsymbolic-functions -specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'"
 PY_CFLAGS = "-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security"
 PY_CORE_CFLAGS = "-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -IObjects -IInclude -IPython -I. -I../Include -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPy_BUILD_CORE"

Matthias Klose (doko) wrote :

confirmed. we need to change that for 18.04. For now you can work around this by install the dpkg-dev package.

affects: python3-defaults (Ubuntu) → python3.6 (Ubuntu)
Changed in python3.6 (Ubuntu):
status: New → Triaged
Changed in python3.6 (Ubuntu Bionic):
status: New → Triaged
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python3.6 - 3.6.5-6ubuntu1

---------------
python3.6 (3.6.5-6ubuntu1) cosmic; urgency=medium

  * Disable the lto build for now.

 -- Matthias Klose <email address hidden> Thu, 03 May 2018 21:35:50 +0200

Changed in python3.6 (Ubuntu Cosmic):
status: Triaged → Fix Released
Anthony Sottile (asottile) wrote :

Any ETA on when this'll land for bionic?

Matthias Klose (doko) on 2018-09-13
description: updated

Hello Anthony, or anyone else affected,

Accepted python3.6 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python3.6/3.6.6-1~18.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python3.6 (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-bionic
Anthony Sottile (asottile) wrote :

Confirmed fixed:

(venv) root@99e4d04174a9:/# pip install ujson
Collecting ujson
  Using cached https://files.pythonhosted.org/packages/16/c4/79f3409bc710559015464e5f49b9879430d8f87498ecdc335899732e5377/ujson-1.35.tar.gz
Building wheels for collected packages: ujson
  Running setup.py bdist_wheel for ujson ... done
  Stored in directory: /root/.cache/pip/wheels/28/77/e4/0311145b9c2e2f01470e744855131f9e34d6919687550f87d1
Successfully built ujson
Installing collected packages: ujson

Didn't have time to wait for proposed so I just downloaded the amd64 binary outputs and `gdebi -n` installed them.

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python3.6 - 3.6.6-1~18.04

---------------
python3.6 (3.6.6-1~18.04) bionic-proposed; urgency=medium

  * SRU: LP: #1792143. Update Python 3.6 to the recent subminor release.

python3.6 (3.6.6-1) unstable; urgency=medium

  * Python 3.6.6 release.

python3.6 (3.6.6~rc1-4) unstable; urgency=medium

  * Update to 20180622 from the 3.6 branch.
    - Fix installation of idlelib testsuite.

python3.6 (3.6.6~rc1-3) unstable; urgency=medium

  * Update to 20180619 from the 3.6 branch.
  * Let ensurepip work with pip versions 9 or 10. Closes: #901797.
  * pdb(1): Fix pointer to HTML documentation. Closes: #901603.

python3.6 (3.6.6~rc1-1) unstable; urgency=medium

  * Python 3.6.6 release candidate 1.

python3.6 (3.6.5-10) unstable; urgency=medium

  * Update to 20180608 from the 3.6 branch.
  * Update the the sysconfig-debian-schemes patch, not yet applied.
    sysconfig: The 'posix_local' schema name is the default unless the env var
    DEB_PYTHON_INSTALL_LAYOUT is set to 'deb' or 'deb_system'. Use the latter
    for package buillds.
  * Build cmath as a builtin instead of an extension.
  * Run the test_code_module, test_zipfile tests in the autopkg tests again.
  * Build using GCC 8.

python3.6 (3.6.5-9) unstable; urgency=medium

  * python3.6: Add breaks on python3-all (<< 3.6.5~rc1-1),
    python3-dev (<< 3.6.5~rc1-1), python3-venv (<< 3.6.5-2). Closes: #898401.

python3.6 (3.6.5-8) unstable; urgency=medium

  * Fix the lto build for extensions, passing -fPIC to the linker.

python3.6 (3.6.5-6) unstable; urgency=medium

  * Don't inject dpkg's compiler specs into distutils. LP: #1768644.

python3.6 (3.6.5-5) unstable; urgency=medium

  * Fix the lto build on x86_64 architectures.
  * Make the build of the shared modules verbose again.

python3.6 (3.6.5-4) unstable; urgency=medium

  * Re-enable pgo/lto builds.

 -- Matthias Klose <email address hidden> Wed, 12 Sep 2018 20:26:19 +0200

Changed in python3.6 (Ubuntu Bionic):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for python3.6 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers