| 2023-01-05 22:38:28 |
Nafees |
bug |
|
|
added bug |
| 2023-01-05 22:39:01 |
Nafees |
summary |
Python extension modules get build using wrong compiler flags with python2 |
Python extension modules get built using wrong compiler flags with python2 |
|
| 2023-01-11 16:31:43 |
William Ashley |
bug |
|
|
added subscriber William Ashley |
| 2023-01-11 18:07:51 |
Fabio Augusto Miranda Martins |
bug |
|
|
added subscriber Fabio Augusto Miranda Martins |
| 2023-01-11 21:27:00 |
Nafees |
attachment added |
|
testprog.tar.gz https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5640852/+files/testprog.tar.gz |
|
| 2023-01-24 07:46:35 |
Ghadi Rahme |
python2.7 (Ubuntu): assignee |
|
Ghadi Rahme (ghadi-rahme) |
|
| 2023-01-24 07:47:01 |
Ghadi Rahme |
python2.7 (Ubuntu): status |
New |
In Progress |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
nominated for series |
|
Ubuntu Focal |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
bug task added |
|
python2.7 (Ubuntu Focal) |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
nominated for series |
|
Ubuntu Lunar |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
bug task added |
|
python2.7 (Ubuntu Lunar) |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
nominated for series |
|
Ubuntu Bionic |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
bug task added |
|
python2.7 (Ubuntu Bionic) |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
nominated for series |
|
Ubuntu Kinetic |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
bug task added |
|
python2.7 (Ubuntu Kinetic) |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
nominated for series |
|
Ubuntu Jammy |
|
| 2023-02-07 16:12:18 |
Ghadi Rahme |
bug task added |
|
python2.7 (Ubuntu Jammy) |
|
| 2023-02-07 16:13:03 |
Ghadi Rahme |
python2.7 (Ubuntu Bionic): status |
New |
In Progress |
|
| 2023-02-07 16:13:07 |
Ghadi Rahme |
python2.7 (Ubuntu Focal): status |
New |
In Progress |
|
| 2023-02-07 16:13:12 |
Ghadi Rahme |
python2.7 (Ubuntu Jammy): status |
New |
In Progress |
|
| 2023-02-07 16:13:16 |
Ghadi Rahme |
python2.7 (Ubuntu Kinetic): status |
New |
In Progress |
|
| 2023-02-07 16:13:27 |
Ghadi Rahme |
python2.7 (Ubuntu Bionic): assignee |
|
Ghadi Rahme (ghadi-rahme) |
|
| 2023-02-07 16:13:34 |
Ghadi Rahme |
python2.7 (Ubuntu Focal): assignee |
|
Ghadi Rahme (ghadi-rahme) |
|
| 2023-02-07 16:13:38 |
Ghadi Rahme |
python2.7 (Ubuntu Jammy): assignee |
|
Ghadi Rahme (ghadi-rahme) |
|
| 2023-02-07 16:13:40 |
Ghadi Rahme |
python2.7 (Ubuntu Kinetic): assignee |
|
Ghadi Rahme (ghadi-rahme) |
|
| 2023-03-17 13:08:42 |
Heitor Alves de Siqueira |
tags |
|
se-sponsor-halves |
|
| 2023-03-17 13:08:47 |
Heitor Alves de Siqueira |
bug |
|
|
added subscriber Heitor Alves de Siqueira |
| 2023-07-06 14:43:13 |
Ghadi Rahme |
python2.7 (Ubuntu Lunar): status |
In Progress |
Won't Fix |
|
| 2023-07-06 14:45:07 |
Ghadi Rahme |
python2.7 (Ubuntu): milestone |
|
ubuntu-23.10 |
|
| 2023-07-06 14:45:14 |
Ghadi Rahme |
python2.7 (Ubuntu): milestone |
ubuntu-23.10 |
|
|
| 2023-07-06 14:45:27 |
Ghadi Rahme |
nominated for series |
|
Ubuntu Mantic |
|
| 2023-07-06 14:45:27 |
Ghadi Rahme |
bug task added |
|
python2.7 (Ubuntu Mantic) |
|
| 2023-07-06 14:45:31 |
Ghadi Rahme |
python2.7 (Ubuntu Mantic): status |
In Progress |
Won't Fix |
|
| 2023-07-10 07:20:30 |
Ghadi Rahme |
attachment added |
|
[focal]add-optimization-flags-to-cflags.diff https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5685114/+files/%5Bfocal%5Dadd-optimization-flags-to-cflags.diff |
|
| 2023-07-10 07:21:16 |
Ghadi Rahme |
attachment added |
|
[jammy]add-optimization-flags-to-cflags.diff https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5685116/+files/%5Bjammy%5Dadd-optimization-flags-to-cflags.diff |
|
| 2023-07-10 07:21:32 |
Ghadi Rahme |
attachment added |
|
[kinetic]add-optimization-flags-to-cflags.diff https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5685117/+files/%5Bkinetic%5Dadd-optimization-flags-to-cflags.diff |
|
| 2023-07-10 07:24:28 |
Ghadi Rahme |
python2.7 (Ubuntu Focal): importance |
Undecided |
High |
|
| 2023-07-10 07:24:31 |
Ghadi Rahme |
python2.7 (Ubuntu Jammy): importance |
Undecided |
High |
|
| 2023-07-10 07:24:35 |
Ghadi Rahme |
python2.7 (Ubuntu Kinetic): importance |
Undecided |
High |
|
| 2023-07-10 07:24:41 |
Ghadi Rahme |
python2.7 (Ubuntu Bionic): importance |
Undecided |
High |
|
| 2023-07-10 08:16:13 |
Ubuntu Foundations Team Bug Bot |
tags |
se-sponsor-halves |
patch se-sponsor-halves |
|
| 2023-07-10 08:16:18 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Ubuntu Sponsors |
| 2023-07-10 12:01:36 |
Ghadi Rahme |
attachment added |
|
[bionic]add-optimization-flags-to-cflags.diff https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5685152/+files/%5Bbionic%5Dadd-optimization-flags-to-cflags.diff |
|
| 2023-07-10 13:31:17 |
Heitor Alves de Siqueira |
python2.7 (Ubuntu Bionic): status |
In Progress |
Won't Fix |
|
| 2023-07-10 13:31:25 |
Heitor Alves de Siqueira |
attachment removed |
[bionic]add-optimization-flags-to-cflags.diff https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5685152/+files/%5Bbionic%5Dadd-optimization-flags-to-cflags.diff |
|
|
| 2023-07-11 16:47:54 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~dbungert/ubuntu-sponsoring/+git/ubuntu-sponsoring/+merge/446529 |
|
| 2023-07-11 16:50:42 |
Dan Bungert |
merge proposal unlinked |
https://code.launchpad.net/~dbungert/ubuntu-sponsoring/+git/ubuntu-sponsoring/+merge/446529 |
|
|
| 2023-07-11 16:52:43 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~dbungert/ubuntu-sponsoring/+git/ubuntu-sponsoring/+merge/446529 |
|
| 2023-07-11 18:29:09 |
Dan Bungert |
merge proposal unlinked |
https://code.launchpad.net/~dbungert/ubuntu-sponsoring/+git/ubuntu-sponsoring/+merge/446529 |
|
|
| 2023-07-17 16:31:10 |
Steve Langasek |
python2.7 (Ubuntu Focal): status |
In Progress |
Won't Fix |
|
| 2023-07-17 16:31:12 |
Steve Langasek |
python2.7 (Ubuntu Jammy): status |
In Progress |
Won't Fix |
|
| 2023-07-17 16:31:14 |
Steve Langasek |
python2.7 (Ubuntu Kinetic): status |
In Progress |
Won't Fix |
|
| 2023-07-17 16:31:15 |
Steve Langasek |
removed subscriber Ubuntu Sponsors |
|
|
|
| 2023-07-24 16:41:37 |
Robby Pocase |
bug |
|
|
added subscriber Robby Pocase |
| 2024-01-23 07:20:58 |
Mitchell Dzurick |
merge proposal linked |
|
https://code.launchpad.net/~mitchdz/ubuntu/+source/python2.7/+git/python2.7/+merge/458533 |
|
| 2024-01-23 07:21:21 |
Mitchell Dzurick |
python2.7 (Ubuntu Jammy): assignee |
Ghadi Rahme (ghadi-rahme) |
Mitchell Dzurick (mitchdz) |
|
| 2024-01-23 07:21:23 |
Mitchell Dzurick |
python2.7 (Ubuntu Focal): assignee |
Ghadi Rahme (ghadi-rahme) |
Mitchell Dzurick (mitchdz) |
|
| 2024-01-23 07:21:54 |
Mitchell Dzurick |
python2.7 (Ubuntu Lunar): assignee |
Ghadi Rahme (ghadi-rahme) |
|
|
| 2024-01-23 07:21:56 |
Mitchell Dzurick |
python2.7 (Ubuntu Mantic): assignee |
Ghadi Rahme (ghadi-rahme) |
|
|
| 2024-01-23 07:21:59 |
Mitchell Dzurick |
python2.7 (Ubuntu Kinetic): assignee |
Ghadi Rahme (ghadi-rahme) |
|
|
| 2024-01-23 07:22:01 |
Mitchell Dzurick |
python2.7 (Ubuntu Bionic): assignee |
Ghadi Rahme (ghadi-rahme) |
|
|
| 2024-01-23 07:22:02 |
Mitchell Dzurick |
python2.7 (Ubuntu): assignee |
Ghadi Rahme (ghadi-rahme) |
|
|
| 2024-01-23 07:22:49 |
Mitchell Dzurick |
python2.7 (Ubuntu Focal): status |
Won't Fix |
In Progress |
|
| 2024-01-23 07:22:52 |
Mitchell Dzurick |
python2.7 (Ubuntu Jammy): status |
Won't Fix |
In Progress |
|
| 2024-01-23 07:23:08 |
Mitchell Dzurick |
python2.7 (Ubuntu): status |
In Progress |
Invalid |
|
| 2024-01-23 07:23:17 |
Mitchell Dzurick |
python2.7 (Ubuntu Kinetic): status |
Won't Fix |
Invalid |
|
| 2024-01-23 07:23:19 |
Mitchell Dzurick |
python2.7 (Ubuntu Lunar): status |
Won't Fix |
Invalid |
|
| 2024-01-23 07:23:21 |
Mitchell Dzurick |
python2.7 (Ubuntu Mantic): status |
Won't Fix |
Invalid |
|
| 2024-01-29 23:38:01 |
Mitchell Dzurick |
description |
Compiling a Python extension using Python2 (Python 2.7.18) is making use of wrong compiler flags, hence dropping required optimizations when required. This is happening only when python2 is installed from Ubuntu's repositories. By default, Python's distutils module uses compiler and linker flags used to compile Python itself to be used to compile extensions.
Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig module. On my machine, the output is
Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=. -fstack-protector-strong -Wformat -Werror=format-security '
3) Build a test extension module using python2 and verify the compilation flags.
python2 setup.py build_ext --inplace
Output from below command is not matching with our expected above CFLAGS.
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c testmodule.c -o build/temp.linux-aarch64-2.7/testmodule.o
On further investigation, it looks like Ubuntu's specific patch applied on libpython2.7-stdlib package is altering the original upstream implementation of distutils/sysconfig.py code.
Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch - http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz
Below is the code block which is causing the issue, where the presence of configure_cflags is modifying cflags. This code is result of ubuntu's patch and doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
elif configure_cflags:
cflags = ' '.join(str(x) for x in (basecflags, configure_cflags, extra_cflags) if x)
ldshared = ldshared + ' ' + configure_cflags
I don't see problem on Python3 though we have extra code added from patch there as well. Patch used on python3, is not modifying the cflags completely and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
elif configure_cflags:
cflags = cflags + ' ' + configure_cflags
ldshared = ldshared + ' ' + configure_cflags
Request to update the python2 patch to behave similar to what is been done on python3. |
[ Impact ]
When compiling Python extensions using Python2, CFLAGS optimization flags are dropped.
This behavior has been caused by our update in this patch http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz which differs from upstream.
The fix modifies the portion of code in Lib/distutils/sysconfig.py which gets the cflags from the environments, and includes the dropped OPT flag from get_config_vars().
[ Test Plan ]
There will be 2 separate tests for this bug:
* Ensuring no-change rebuilds are not changed
* Ensuring local builds are not changed unless environment variable is set
Test 1) No-change rebuilds
To test that no-change rebuilds are not changed, the package python-stdlib-extensions will be built against the new python2.7, and confirm the compiler flags are not altered. This will be a manual test and visual inspection of the build logs.
Test 2) Functional test
1. Create test container
$ lxc launch ubuntu:jammy jammy-2002043
$ lxc shell ubuntu:jammy jammy-2002043
2. Install required packages
# apt update -y && apt install -y python2 python-pip
3. Create test files
# mkdir testprog
# cd testprog
# cat >setup.py <<EOL
from setuptools import setup, Extension
setup(
name="test",
ext_modules=[Extension("test", sources=["testmodule.c"])],
zip_safe=False
)
EOL
# cat >testmodule.c <<EOL
#include <stdio.h>
int main(void)
{
printf("This is test program");
return 0;
}
EOL
4. Compile a test program
# python2 setup.py build_ext --inplace
5. Check CFLAGS
# python2 -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))"
-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
6. Check the flags used to compile the test wheel
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
7. Install fixed python
Once updated, this will simply be an apt update && apt upgrade
# add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y
# apt install -y python2.7
# dpkg -s python2.7 | grep Version:
Version: 2.7.18-13ubuntu1.2~jammy9
8. Clean build
# rm -rf build/ test.so
9. Enable opt-in environment variable
# export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND=""
9. Rebuild with new python2.7 installed
# python2 setup.py build_ext --inplace
10. Check build flags
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects -fstack-protector-strong -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
[ Where problems could occur ]
* Changing optimization flags can cause a myriad of unattended side effects, but the change being opt-in means the users should be aware a change is being made
* The change is opt-in, so an informative message is printed to the console to spread awareness of the issue and how to use the workaround. A questionably made CI/CD system might see the new console output and flag it as a failure, so it is important to choose wording carefully. |
|
| 2024-01-31 19:01:00 |
Mitchell Dzurick |
description |
[ Impact ]
When compiling Python extensions using Python2, CFLAGS optimization flags are dropped.
This behavior has been caused by our update in this patch http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz which differs from upstream.
The fix modifies the portion of code in Lib/distutils/sysconfig.py which gets the cflags from the environments, and includes the dropped OPT flag from get_config_vars().
[ Test Plan ]
There will be 2 separate tests for this bug:
* Ensuring no-change rebuilds are not changed
* Ensuring local builds are not changed unless environment variable is set
Test 1) No-change rebuilds
To test that no-change rebuilds are not changed, the package python-stdlib-extensions will be built against the new python2.7, and confirm the compiler flags are not altered. This will be a manual test and visual inspection of the build logs.
Test 2) Functional test
1. Create test container
$ lxc launch ubuntu:jammy jammy-2002043
$ lxc shell ubuntu:jammy jammy-2002043
2. Install required packages
# apt update -y && apt install -y python2 python-pip
3. Create test files
# mkdir testprog
# cd testprog
# cat >setup.py <<EOL
from setuptools import setup, Extension
setup(
name="test",
ext_modules=[Extension("test", sources=["testmodule.c"])],
zip_safe=False
)
EOL
# cat >testmodule.c <<EOL
#include <stdio.h>
int main(void)
{
printf("This is test program");
return 0;
}
EOL
4. Compile a test program
# python2 setup.py build_ext --inplace
5. Check CFLAGS
# python2 -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))"
-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
6. Check the flags used to compile the test wheel
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
7. Install fixed python
Once updated, this will simply be an apt update && apt upgrade
# add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y
# apt install -y python2.7
# dpkg -s python2.7 | grep Version:
Version: 2.7.18-13ubuntu1.2~jammy9
8. Clean build
# rm -rf build/ test.so
9. Enable opt-in environment variable
# export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND=""
9. Rebuild with new python2.7 installed
# python2 setup.py build_ext --inplace
10. Check build flags
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects -fstack-protector-strong -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
[ Where problems could occur ]
* Changing optimization flags can cause a myriad of unattended side effects, but the change being opt-in means the users should be aware a change is being made
* The change is opt-in, so an informative message is printed to the console to spread awareness of the issue and how to use the workaround. A questionably made CI/CD system might see the new console output and flag it as a failure, so it is important to choose wording carefully. |
[ Impact ]
When compiling Python extensions using Python2, CFLAGS optimization flags are dropped.
This behavior has been caused by our update in this patch http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz which differs from upstream.
The fix modifies the portion of code in Lib/distutils/sysconfig.py which gets the cflags from the environments, and includes the dropped OPT flag from get_config_vars().
[ Test Plan ]
There will be 2 separate tests for this bug:
* Ensuring no-change rebuilds are not changed
* Ensuring local builds are not changed unless environment variable is set
Test 1) No-change rebuilds
To test that no-change rebuilds are not changed, the package python-stdlib-extensions will be built against the new python2.7, and confirm the compiler flags are not altered. This will be a manual test and visual inspection of the build logs.
Test 2) Functional test
1. Create test container
$ lxc launch ubuntu:jammy jammy-2002043
$ lxc shell ubuntu:jammy jammy-2002043
2. Install required packages
For Jammy
# apt update -y && apt install -y python2 python-pip
For Focal
# apt update -y && apt install -y python2 python-setuptools
3. Create test files
# mkdir testprog
# cd testprog
# cat >setup.py <<EOL
from setuptools import setup, Extension
setup(
name="test",
ext_modules=[Extension("test", sources=["testmodule.c"])],
zip_safe=False
)
EOL
# cat >testmodule.c <<EOL
#include <stdio.h>
int main(void)
{
printf("This is test program");
return 0;
}
EOL
4. Compile a test program
# python2 setup.py build_ext --inplace
5. Check CFLAGS
# python2 -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))"
-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
6. Check the flags used to compile the test wheel
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
7. Install fixed python
Once updated, this will simply be an apt update && apt upgrade
# add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y
# apt install -y python2.7
# dpkg -s python2.7 | grep Version:
Version: 2.7.18-13ubuntu1.2~jammy9
8. Clean build
# rm -rf build/ test.so
9. Enable opt-in environment variable
# export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND=""
9. Rebuild with new python2.7 installed
# python2 setup.py build_ext --inplace
10. Check build flags
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects -fstack-protector-strong -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
[ Where problems could occur ]
* Changing optimization flags can cause a myriad of unattended side effects, but the change being opt-in means the users should be aware a change is being made
* The change is opt-in, so an informative message is printed to the console to spread awareness of the issue and how to use the workaround. A questionably made CI/CD system might see the new console output and flag it as a failure, so it is important to choose wording carefully. |
|
| 2024-01-31 21:43:05 |
Ubuntu Archive Robot |
bug |
|
|
added subscriber Lena Voytek |
| 2024-02-01 19:56:44 |
Mitchell Dzurick |
merge proposal linked |
|
https://code.launchpad.net/~mitchdz/ubuntu/+source/python2.7/+git/python2.7/+merge/459844 |
|
| 2024-02-02 14:18:16 |
Timo Aaltonen |
python2.7 (Ubuntu Jammy): status |
In Progress |
Fix Committed |
|
| 2024-02-02 14:18:18 |
Timo Aaltonen |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2024-02-02 14:18:19 |
Timo Aaltonen |
bug |
|
|
added subscriber SRU Verification |
| 2024-02-02 14:18:23 |
Timo Aaltonen |
tags |
patch se-sponsor-halves |
patch se-sponsor-halves verification-needed verification-needed-jammy |
|
| 2024-02-08 21:21:28 |
Andreas Hasenack |
python2.7 (Ubuntu Focal): status |
In Progress |
Fix Committed |
|
| 2024-02-08 21:21:35 |
Andreas Hasenack |
tags |
patch se-sponsor-halves verification-needed verification-needed-jammy |
patch se-sponsor-halves verification-needed verification-needed-focal verification-needed-jammy |
|
| 2024-02-16 16:58:46 |
Mitchell Dzurick |
attachment added |
|
Python2.7_jammy_opt_flags_verification.txt https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5746685/+files/Python2.7_jammy_opt_flags_verification.txt |
|
| 2024-02-16 16:59:43 |
Mitchell Dzurick |
tags |
patch se-sponsor-halves verification-needed verification-needed-focal verification-needed-jammy |
patch se-sponsor-halves verification-done-jammy verification-needed verification-needed-focal |
|
| 2024-02-16 17:16:43 |
Mitchell Dzurick |
attachment added |
|
PYthon2.7_focal_opt_flags_verification.txt https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5746688/+files/PYthon2.7_focal_opt_flags_verification.txt |
|
| 2024-02-16 17:17:08 |
Mitchell Dzurick |
tags |
patch se-sponsor-halves verification-done-jammy verification-needed verification-needed-focal |
patch se-sponsor-halves verification-done verification-done-focal verification-done-jammy |
|
| 2024-02-16 17:20:04 |
Mitchell Dzurick |
description |
[ Impact ]
When compiling Python extensions using Python2, CFLAGS optimization flags are dropped.
This behavior has been caused by our update in this patch http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz which differs from upstream.
The fix modifies the portion of code in Lib/distutils/sysconfig.py which gets the cflags from the environments, and includes the dropped OPT flag from get_config_vars().
[ Test Plan ]
There will be 2 separate tests for this bug:
* Ensuring no-change rebuilds are not changed
* Ensuring local builds are not changed unless environment variable is set
Test 1) No-change rebuilds
To test that no-change rebuilds are not changed, the package python-stdlib-extensions will be built against the new python2.7, and confirm the compiler flags are not altered. This will be a manual test and visual inspection of the build logs.
Test 2) Functional test
1. Create test container
$ lxc launch ubuntu:jammy jammy-2002043
$ lxc shell ubuntu:jammy jammy-2002043
2. Install required packages
For Jammy
# apt update -y && apt install -y python2 python-pip
For Focal
# apt update -y && apt install -y python2 python-setuptools
3. Create test files
# mkdir testprog
# cd testprog
# cat >setup.py <<EOL
from setuptools import setup, Extension
setup(
name="test",
ext_modules=[Extension("test", sources=["testmodule.c"])],
zip_safe=False
)
EOL
# cat >testmodule.c <<EOL
#include <stdio.h>
int main(void)
{
printf("This is test program");
return 0;
}
EOL
4. Compile a test program
# python2 setup.py build_ext --inplace
5. Check CFLAGS
# python2 -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))"
-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
6. Check the flags used to compile the test wheel
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
7. Install fixed python
Once updated, this will simply be an apt update && apt upgrade
# add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y
# apt install -y python2.7
# dpkg -s python2.7 | grep Version:
Version: 2.7.18-13ubuntu1.2~jammy9
8. Clean build
# rm -rf build/ test.so
9. Enable opt-in environment variable
# export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND=""
9. Rebuild with new python2.7 installed
# python2 setup.py build_ext --inplace
10. Check build flags
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects -fstack-protector-strong -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
[ Where problems could occur ]
* Changing optimization flags can cause a myriad of unattended side effects, but the change being opt-in means the users should be aware a change is being made
* The change is opt-in, so an informative message is printed to the console to spread awareness of the issue and how to use the workaround. A questionably made CI/CD system might see the new console output and flag it as a failure, so it is important to choose wording carefully. |
[ Impact ]
When compiling Python extensions using Python2, CFLAGS optimization flags are dropped.
This behavior has been caused by our update in this patch http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz which differs from upstream.
The fix modifies the portion of code in Lib/distutils/sysconfig.py which gets the cflags from the environments, and includes the dropped OPT flag from get_config_vars().
[ Test Plan ]
There will be 2 separate tests for this bug:
* Ensuring no-change rebuilds are not changed
* Ensuring local builds are not changed unless environment variable is set
Test 1) No-change rebuilds
To test that no-change rebuilds are not changed, the package python-stdlib-extensions will be built against the new python2.7, and confirm the compiler flags are not altered. This will be a manual test and visual inspection of the build logs.
Test 2) Functional test
1. Create test container
$ lxc launch ubuntu:jammy jammy-2002043
$ lxc shell jammy-2002043
2. Install required packages
For Jammy
# apt update -y && apt install -y python2 python-pip
For Focal
# apt update -y && apt install -y python2 python-setuptools gcc
3. Create test files
# mkdir testprog
# cd testprog
# cat >setup.py <<EOL
from setuptools import setup, Extension
setup(
name="test",
ext_modules=[Extension("test", sources=["testmodule.c"])],
zip_safe=False
)
EOL
# cat >testmodule.c <<EOL
#include <stdio.h>
int main(void)
{
printf("This is test program");
return 0;
}
EOL
4. Compile a test program
# python2 setup.py build_ext --inplace
5. Check CFLAGS
# python2 -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))"
-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
6. Check the flags used to compile the test wheel
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
7. Install fixed python
Once updated, this will simply be an apt update && apt upgrade
# add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y
# apt install -y python2.7
# dpkg -s python2.7 | grep Version:
Version: 2.7.18-13ubuntu1.2~jammy9
8. Clean build
# rm -rf build/ test.so
9. Enable opt-in environment variable
# export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND=""
9. Rebuild with new python2.7 installed
# python2 setup.py build_ext --inplace
10. Check build flags
# strings build/lib.linux-x86_64-2.7/test.so | grep -- -O
GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans
GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects -fstack-protector-strong -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
[ Where problems could occur ]
* Changing optimization flags can cause a myriad of unattended side effects, but the change being opt-in means the users should be aware a change is being made
* The change is opt-in, so an informative message is printed to the console to spread awareness of the issue and how to use the workaround. A questionably made CI/CD system might see the new console output and flag it as a failure, so it is important to choose wording carefully. |
|
| 2024-03-05 14:23:21 |
William Ashley |
removed subscriber William Ashley |
|
|
|
| 2024-03-07 20:19:36 |
Andreas Hasenack |
bug watch added |
|
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956697 |
|
| 2024-03-14 18:34:51 |
Launchpad Janitor |
python2.7 (Ubuntu Jammy): status |
Fix Committed |
Fix Released |
|
| 2024-03-14 18:34:56 |
Andreas Hasenack |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2024-03-14 18:35:09 |
Launchpad Janitor |
python2.7 (Ubuntu Focal): status |
Fix Committed |
Fix Released |
|
