python3.4.3 SRU breaks awscli

Bug #1499075 reported by Nelson Elhage
154
This bug affects 26 people
Affects Status Importance Assigned to Milestone
python-botocore (Ubuntu)
Fix Released
Critical
Unassigned
Trusty
Fix Released
Critical
Unassigned

Bug Description

The recently (~6h ago) python3.4.3 SRU update contains backwards-incompatible changes to the Python language, which in particular break the version of `awscli` packaged in Ubuntu Trusty. From a fully-updated Trusty machine:

<email address hidden>:~# aws
Traceback (most recent call last):
  File "/usr/bin/aws", line 23, in <module>
    sys.exit(main())
  File "/usr/bin/aws", line 19, in main
    return awscli.clidriver.main()
  File "/usr/share/awscli/awscli/clidriver.py", line 44, in main
    driver = create_clidriver()
  File "/usr/share/awscli/awscli/clidriver.py", line 53, in create_clidriver
    event_hooks=emitter)
  File "/usr/share/awscli/awscli/plugin.py", line 49, in load_plugins
    plugin.awscli_initialize(event_hooks)
  File "/usr/share/awscli/awscli/handlers.py", line 73, in awscli_initialize
    register_removals(event_handlers)
  File "/usr/share/awscli/awscli/customizations/removals.py", line 32, in register_removals
    'verify-email-address'])
  File "/usr/share/awscli/awscli/customizations/removals.py", line 45, in remove
    self._create_remover(remove_commands))
  File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 64, in register
    self._verify_accept_kwargs(handler)
  File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 84, in _verify_accept_kwargs
    argspec = inspect.getargspec(func)
  File "/usr/lib/python3.4/inspect.py", line 936, in getargspec
    raise ValueError("Function has keyword-only arguments or annotations"
ValueError: Function has keyword-only arguments or annotations, use getfullargspec() API which can support them

This bug has been fixed upstream via https://github.com/aws/aws-cli/issues/800, but the version in Ubuntu hasn't been updated.

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: python3.4 3.4.3-1ubuntu1~14.04.1
ProcVersionSignature: Ubuntu 3.13.0-63.103-generic 3.13.11-ckt25
Uname: Linux 3.13.0-63-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.13
Architecture: amd64
Date: Wed Sep 23 21:06:39 2015
Ec2AMI: ami-8d5b5dbd
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-west-2c
Ec2InstanceType: c4.8xlarge
Ec2Kernel: unavailable
Ec2Ramdisk: unavailable
ProcEnviron:
 TERM=screen
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: python3.4
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
Nelson Elhage (nelhage) wrote :
Evan Broder (broder)
Changed in python-botocore (Ubuntu):
status: New → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-botocore (Ubuntu Trusty):
status: New → Confirmed
Changed in python3.4 (Ubuntu Trusty):
status: New → Confirmed
Changed in python3.4 (Ubuntu):
status: New → Confirmed
Revision history for this message
Evan Broder (broder) wrote :
Changed in python3.4 (Ubuntu Trusty):
importance: Undecided → Critical
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

ACK on the debdiff, uploaded to queue for the SRU team.

Changed in python-botocore (Ubuntu Trusty):
status: Confirmed → In Progress
Revision history for this message
Matthias Klose (doko) wrote :

not sure what the best thing to do ... either revert that, or fix the packages affected by this incompatible upstream change.

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Nelson, or anyone else affected,

Accepted python-botocore into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-botocore/0.29.0+repack-2ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-botocore (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Adrien Siebert (asiebert) wrote :

Hi Adam,

just successfully tested out 0.29.0+repack-2ubuntu0.1 (upgraded from 0.29.0+repack-2) on trusty-proposed − thanks for providing all the instructions to quickly get to it.

We had the exact same issue when using the aws-cli after the python3.4 upgrade, and from our limited testing it appear to be fixed with the proposed package.

Revision history for this message
Slobodan Miskovic (slobodan) wrote :

0.29.0+repack-2ubuntu0.1 fixes the problem for us

tags: added: verification-done
removed: verification-needed
Revision history for this message
Wayne Clancy (wayneclancy80) wrote :
Download full text (11.1 KiB)

I'm still seeing the problem. Fully updated version of 14.04 LTS running in AWS ap-northeast-1. It appears have the latest and greatest python3-botocore.

Can this ticket please be re-opened?

root@ip-10-0-100-139:~# aws
Traceback (most recent call last):
  File "/usr/bin/aws", line 23, in <module>
    sys.exit(main())
  File "/usr/bin/aws", line 19, in main
    return awscli.clidriver.main()
  File "/usr/share/awscli/awscli/clidriver.py", line 44, in main
    driver = create_clidriver()
  File "/usr/share/awscli/awscli/clidriver.py", line 53, in create_clidriver
    event_hooks=emitter)
  File "/usr/share/awscli/awscli/plugin.py", line 49, in load_plugins
    plugin.awscli_initialize(event_hooks)
  File "/usr/share/awscli/awscli/handlers.py", line 73, in awscli_initialize
    register_removals(event_handlers)
  File "/usr/share/awscli/awscli/customizations/removals.py", line 32, in register_removals
    'verify-email-address'])
  File "/usr/share/awscli/awscli/customizations/removals.py", line 45, in remove
    self._create_remover(remove_commands))
  File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 64, in register
    self._verify_accept_kwargs(handler)
  File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 84, in _verify_accept_kwargs
    argspec = inspect.getargspec(func)
  File "/usr/lib/python3.4/inspect.py", line 936, in getargspec
    raise ValueError("Function has keyword-only arguments or annotations"
ValueError: Function has keyword-only arguments or annotations, use getfullargspec() API which can support them

--Packages installed---

ii awscli 1.2.9-2 all Universal Command Line Environment for AWS
ii dh-python 1.20140128-1ubuntu8.2 all Debian helper tools for packaging Python libraries and applications
ii libpython-stdlib:amd64 2.7.5-5ubuntu3 amd64 interactive high-level object-oriented language (default python version)
ii libpython2.7:amd64 2.7.6-8ubuntu0.2 amd64 Shared Python runtime library (version 2.7)
ii libpython2.7-minimal:amd64 2.7.6-8ubuntu0.2 amd64 Minimal subset of the Python language (version 2.7)
ii libpython2.7-stdlib:amd64 2.7.6-8ubuntu0.2 amd64 Interactive high-level object-oriented language (standard library, version 2.7)
ii libpython3-stdlib:amd64 3.4.0-0ubuntu2 amd64 interactive high-level object-oriented language (default python3 version)
ii libpython3.4-minimal:amd64 3.4.3-1ubuntu1~14.04.1 amd64 Minimal subset of the Python language (version 3.4)
ii libpython3.4-stdlib:amd64 3.4.3-1ubuntu1~14.04.1 amd64 Interactive high-level object-oriented language (standard library, version 3.4)
ii python 2.7.5-5ubuntu3 amd64 interactive high-level object-oriented language (default version)
ii python-apt 0.9.3.5ubuntu1 amd64 Python interface to libapt-pkg
ii python-apt-common ...

Revision history for this message
Ricardo Pardini (rpardini) wrote :

@adconrad: I can confirm that python3-botocore (0.29.0+repack-2ubuntu0.1) fixes the problem. Thanks!
@wayne: no, you don't have the updated botocore. Try this:

wget "http://launchpadlibrarian.net/218773283/python3-botocore_0.29.0%2Brepack-2ubuntu0.1_all.deb"
dpkg -i python3-botocore_0.29.0+repack-2ubuntu0.1_all.deb

Revision history for this message
Tom Wills (r-mail-q) wrote :

I can run awscli successfully after upgrading to 0.29.0+repack-2ubuntu0.1 from trusty-proposed. Thanks.

Revision history for this message
Barry Warsaw (barry) wrote :

Since this is fixed in boto, are the Python bugtasks still relevant?

Revision history for this message
Matthias Klose (doko) wrote :

won't fix in python3.4 for trusty; let's update other packages in the archive if we find them. Third party software is much more likely to expect the 3.4.3 behaviour over the 3.4.0 behaviour.

no longer affects: python3.4 (Ubuntu)
Changed in python3.4 (Ubuntu Trusty):
status: Confirmed → Won't Fix
Revision history for this message
Jaap-Andre de Hoop (j-dehoop) wrote :

Just update python3-botocore from ubuntu-proposed and aws (awscli) is running without a problem (on two computers) again,

Revision history for this message
Stefan Horning (stefan-horning) wrote :

I don't know what I am doing wrong but I am still seing the same error when executing the aws command even after I installed the python-botocore (0.29.0+repack-2ubuntu0.1) package from the proposed repo.

Steps
1. Added deb http://archive.ubuntu.com/ubuntu/ trusty-proposed restricted main multiverse universe to sources.list
2. apt-get update
3. apt-get install python-botocore, which correctly installed the patched versios as far I can see:
Selecting previously unselected package python-ply.
(Reading database ... 94892 files and directories currently installed.)
Preparing to unpack .../python-ply_3.4-3ubuntu2_all.deb ...
Unpacking python-ply (3.4-3ubuntu2) ...
Selecting previously unselected package python-jmespath.
Preparing to unpack .../python-jmespath_0.2.1-1_all.deb ...
Unpacking python-jmespath (0.2.1-1) ...
Selecting previously unselected package python-dateutil.
Preparing to unpack .../python-dateutil_1.5+dfsg-1ubuntu1_all.deb ...
Unpacking python-dateutil (1.5+dfsg-1ubuntu1) ...
Selecting previously unselected package python-botocore.
Preparing to unpack .../python-botocore_0.29.0+repack-2ubuntu0.1_all.deb ...
Unpacking python-botocore (0.29.0+repack-2ubuntu0.1) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up python-ply (3.4-3ubuntu2) ...
Setting up python-jmespath (0.2.1-1) ...
Setting up python-dateutil (1.5+dfsg-1ubuntu1) ...
Setting up python-botocore (0.29.0+repack-2ubuntu0.1) ...

4. execute aws and still get
Traceback (most recent call last):
  File "/usr/bin/aws", line 23, in <module>
    sys.exit(main())
  File "/usr/bin/aws", line 19, in main
    return awscli.clidriver.main()
  File "/usr/share/awscli/awscli/clidriver.py", line 44, in main
    driver = create_clidriver()
  File "/usr/share/awscli/awscli/clidriver.py", line 53, in create_clidriver
    event_hooks=emitter)
  File "/usr/share/awscli/awscli/plugin.py", line 49, in load_plugins
    plugin.awscli_initialize(event_hooks)
  File "/usr/share/awscli/awscli/handlers.py", line 73, in awscli_initialize
    register_removals(event_handlers)
  File "/usr/share/awscli/awscli/customizations/removals.py", line 32, in register_removals
    'verify-email-address'])
  File "/usr/share/awscli/awscli/customizations/removals.py", line 45, in remove
    self._create_remover(remove_commands))
  File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 64, in register
    self._verify_accept_kwargs(handler)
  File "/usr/lib/python3/dist-packages/botocore/hooks.py", line 84, in _verify_accept_kwargs
    argspec = inspect.getargspec(func)
  File "/usr/lib/python3.4/inspect.py", line 936, in getargspec
    raise ValueError("Function has keyword-only arguments or annotations"
ValueError: Function has keyword-only arguments or annotations, use getfullargspec() API which can support them

Or do I have to reinstall awscli for some reason too?

Revision history for this message
Stefan Horning (stefan-horning) wrote :

Here some system information of where I tried it:

Linux ip-172-16-1-173 3.13.0-65-generic #105-Ubuntu SMP Mon Sep 21 18:50:58 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty

Revision history for this message
Stefan Horning (stefan-horning) wrote :

Ok the trick seems to be that you have to activate the proposed repo before installing awscli and any package it brings onto the system. So if you already have awscli installed you should to a apt-get remove awscli AND a apt-get autoclean and then to a fresh install of awscli, afther that the bug is fixed.

Revision history for this message
Stefan Horning (stefan-horning) wrote :

I of course meant `apt-get autoremove`

Revision history for this message
Jeff Hansen (sqrammi) wrote :

When will python-botocore hit updates? I still get the error right now on a fresh Ubuntu instance.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-botocore - 0.29.0+repack-2ubuntu0.1

---------------
python-botocore (0.29.0+repack-2ubuntu0.1) trusty; urgency=low

  * debian/patches/support-python3.4.1: Fix use of inspect with Python
    3.4.1 or later. (LP: #1499075)

 -- Evan Broder <email address hidden> Wed, 23 Sep 2015 17:35:34 -0400

Changed in python-botocore (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Chris J Arges (arges) wrote : Update Released

The verification of the Stable Release Update for python-botocore has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Mathew Hodson (mhodson)
no longer affects: python3.4 (Ubuntu Trusty)
Changed in python-botocore (Ubuntu):
importance: Undecided → Critical
Changed in python-botocore (Ubuntu Trusty):
importance: Undecided → Critical
tags: added: regression-update
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.