When using Ensemble, add-apt-repository no longer functions properly

[Brian Thomason]

One of the changes made between Jul 22-25 caused add-apt-repository to quit working when being called from within an ensemble formula:

+ apt-add-repository ppa:cloudfoundry/ppa

2011-07-25 22:55:32,383: hook.output@ERROR: Fatal Python error: Py_Initialize: Unable to get the locale encoding
  File "/usr/lib/python2.7/encodings/__init__.py", line 123
    raise CodecRegistryError,\
                            ^

2011-07-25 22:55:32,384: hook.output@ERROR: SyntaxError: invalid syntax

Ensemble does not set many env vars, so this may be the culprit.

When running this from within an Ensemble script on an Oneiric install, the error above is yielded, and while the file in sources.list.d is written, it uses sid for the distro rather than oneiric.

If manually logging in and subsequently running the command, everything works as it should, which leads me to believe it is an env var issue.

[Adam Gandelman (gandelman-a)]

Some additional logs. Importing the runpy module from hooks also seems to be broken, as well.?field.comment=Some additional logs. Importing the runpy module from hooks also seems to be broken, as well.

[Dustin Kirkland  (kirkland)]

Adding python2.7 as affected.

According to the error, it's a "syntax error" in a file owned by python2.7-minimal:
  python2.7-minimal: /usr/lib/python2.7/encodings/__init__.py

Looking at line 123, the \ line continuations look a little odd and the lack of preceding whitespace. Just thoughts.

[Juan L. Negron (negronjl)]

This seems to be fixed in the lates oneiric. I tested with ami-id: ami-b7c203de and m1.large today and it all seems to work.

-Juan

[Juan L. Negron (negronjl)]

Just retested this per adam_g's request and found out that apt-add-repository is actually creating the files in /etc/apt/sources.list.d correctly but, the logs show the following erros:

2011-08-10 16:32:38,185: hook.output@ERROR: + apt-add-repository ppa:facter-plugins/ppa

2011-08-10 16:32:38,358: hook.output@ERROR: Fatal Python error: Py_Initialize: Unable to get the locale encoding

2011-08-10 16:32:38,358: hook.output@ERROR: File "
2011-08-10 16:32:38,359: hook.output@ERROR: /usr/lib/python2.7/encodings/__init__.py
2011-08-10 16:32:38,359: hook.output@ERROR: ", line
2011-08-10 16:32:38,359: hook.output@ERROR: 123
2011-08-10 16:32:38,359: hook.output@ERROR:

2011-08-10 16:32:38,359: hook.output@ERROR:
2011-08-10 16:32:38,359: hook.output@ERROR: raise CodecRegistryError,\

2011-08-10 16:32:38,363: hook.output@ERROR: SyntaxError
2011-08-10 16:32:38,363: hook.output@ERROR: :
2011-08-10 16:32:38,363: hook.output@ERROR: invalid syntax

So there is still something broken here.

-Juan

[Juan L. Negron (negronjl)]

need to re-open this bug.

-Juan

[Adam Gandelman (gandelman-a)]

Still affecting oneiric daily AMI

[Adam Gandelman (gandelman-a)]

Still affecting oneiric daily AMIs

[Barry Warsaw (barry)]

This doesn't make much sense. First, afaict the installed encodings/__init__.py looks just fine. A little ugly, but legal syntax. Also, the md5sum of my installed version exactly matches upstream's current Python 2.7 version, and I see nothing relevant in the upstream tracker.

The Py_Initialize bit in the traceback is also odd. Are there any extension modules involved here? This smells suspiciously like a reference counting bug that is triggering a traceback in unrelated Python code. What third party (i.e. not part of Python) code is involved here?

Any possibility you can boil the problem down to something small and reproducible?

[Matthias Klose (doko)]

try using the python-dbg interpreter to track down the reference counting issue (if every extension involved is built for the debug interperter)

[Adam Gandelman (gandelman-a)]

 I've just reproduced the bug using ensemble on oneiric images going back to alpha1. It has definitely worked on ensemble versions since then, so I will mark invalid WRT python and software-properties. Thanks for the input.

[Adam Gandelman (gandelman-a)]

Apologies! As it turns out, the failures I was seeing testing older images was due to something unrelated in the hook executed by ensemble. Eliminating that, the test succeeds on alpha1 cloud AMIs out-of-the-box. However, alpha1 does not have python2.6 installed alongside 2.7 by default. Installing 2.6 and re-running add-apt-repository via ensemble hook fails with a similar runpy.py error.

Unfortunately, I'm having trouble finding any way to reproduce this manually without the use of ensemble.

[Clint Byrum (clint-fewbar)]

Whats the status on this now? Has it just been worked around or is it still a critical issue in Ensemble?

[Juan L. Negron (negronjl)]

It's still an issue in oneiric.

Thanks,

Juan

On Sun, Aug 21, 2011 at 11:25 AM, Clint Byrum <email address hidden> wrote:

> Whats the status on this now? Has it just been worked around or is it
> still a critical issue in Ensemble?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/816169
>
> Title:
> When using Ensemble, add-apt-repository no longer functions properly
>
> Status in “ensemble” package in Ubuntu:
> Confirmed
> Status in “python2.7” package in Ubuntu:
> Confirmed
> Status in “software-properties” package in Ubuntu:
> Confirmed
>
> Bug description:
> One of the changes made between Jul 22-25 caused add-apt-repository to
> quit working when being called from within an ensemble formula:
>
> + apt-add-repository ppa:cloudfoundry/ppa
>
> 2011-07-25 22:55:32,383: hook.output@ERROR: Fatal Python error:
> Py_Initialize: Unable to get the locale encoding
> File "/usr/lib/python2.7/encodings/__init__.py", line 123
> raise CodecRegistryError,\
> ^
>
> 2011-07-25 22:55:32,384: hook.output@ERROR: SyntaxError: invalid
> syntax
>
> Ensemble does not set many env vars, so this may be the culprit.
>
> When running this from within an Ensemble script on an Oneiric
> install, the error above is yielded, and while the file in
> sources.list.d is written, it uses sid for the distro rather than
> oneiric.
>
> If manually logging in and subsequently running the command,
> everything works as it should, which leads me to believe it is an env
> var issue.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/ensemble/+bug/816169/+subscriptions
>
>

[Kapil Thangavelu (hazmat)]

This might have been fixed as a result of bug #816264

ensemble was leaking a custom PYTHONPATH to hooks, which invoked add-apt-repository, it now uses a separate variable to pass this value (for the hook cli api scripts) to enable development mode (non global ensemble package installation).