Regression in argparse for Python 2.7, 3.2 and 3.3

[Stéphane Graber]

Using the following example:
---
#!/usr/bin/python

import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--test", dest="test", type=str,
    default=[], action='append')

args = parser.parse_args()

args.test.append("something")
---

Then running it with simply "python test.py" (WITHOUT passing --test), you get the following:
Traceback (most recent call last):
  File "test.py", line 11, in <module>
    args.test.append("something")
AttributeError: 'str' object has no attribute 'append'

My understanding of argparse is that when using both default=[] and action="append", args.test should always be a list, empty if not passing --test or a list containing the --test values if passed. After the recent update, it seems to instead default to an empty string.

This must somehow be related to the type field as for some reason, removing "type=str" fixes the issue though I don't really see why and it's still a clear regression.

[Stéphane Graber (stgraber)]

Barry actually noticed that it's not an empty string as I had assumed, it's the string "[]".

So it looks like the new version of python now sets the default value to type(default) which in this case would be str([]) explaining that value. It's still different from what we'd get with earlier versions.

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1048710/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Barry Warsaw (barry)]

This is going to affect 3.3 also when Ubuntu gets 3.3 final, since I just checked upstream hg head and it is broken there, even though it works in today's python3.3 package.

[Steve Langasek (vorlon)]

Bug #1048710 reports this to cause a critical regression with juju. Matthias, please take a look ASAP.

[Steve Langasek (vorlon)]

Sorry, wrong bug #; I meant bug #1048864.

[Clint Byrum (clint-fewbar)]

This seems related, might even be the same, but fails with a different case/error message:

#-- begin code--
import argparse
import sys
parser = argparse.ArgumentParser()
parser.add_argument("--log-file", "-l", default=sys.stderr, type=argparse.FileType('a'))
args = parser.parse_args()
#-- end code--

$ python bin/test-broken.py
usage: test-broken.py [-h] [--log-file LOG_FILE]
test-broken.py: error: argument --log-file/-l: invalid FileType('a') value: <open file '<stderr>', mode 'w' at 0x7f201a0fc270>

[Barry Warsaw (barry)]

Upstream branches should now have the proper fixes for this issue. There may be some follow-on doc clarifications but others, but functionally, you should be good to go. Assigning the Ubuntu tasks back to Doko for package updates as per irc.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python2.7 - 2.7.3-5ubuntu2

---------------
python2.7 (2.7.3-5ubuntu2) quantal; urgency=low

  * Update to 20120915, taken from the 2.7 branch. Posix relevant patches:
    - Issue #15906: Fix a regression in argparse caused by the preceding change,
      when action='append', type='str' and default=[]. LP: #1048710.
    - Issue #15908: Fix misbehaviour of the sha1 module when called on data
      larger than 2**32 bytes.
    - Issue #15910: Fix misbehaviour of _md5 and sha1 modules when "updating"
      on data larger than 2**32 bytes.
 -- Matthias Klose <email address hidden> Sat, 15 Sep 2012 12:11:28 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package python3.2 - 3.2.3-6ubuntu2

---------------
python3.2 (3.2.3-6ubuntu2) quantal; urgency=low

  * Update to 20120915 from the 3.2 branch. Linux relevant fixes:
    - Issue #15842: the SocketIO.{readable,writable,seekable} methods now
      raise ValueError when the file-like object is closed.
    - Issue #15906: Fix a regression in argparse caused by the preceding change,
      when action='append', type='str' and default=[]. LP: #1048710.
    - Issue #15793: Stack corruption in ssl.RAND_egd().
 -- Matthias Klose <email address hidden> Sat, 15 Sep 2012 12:14:18 +0200

[Clint Byrum (clint-fewbar)]

Thanks for doing this on a Saturday doko. Confirming, this fixes Juju.