having oslo.sphinx in namespace package causes issues with devstack

Bug #1277168 reported by Doug Hellmann on 2014-02-06
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
Medium
Dirk Mueller
Cinder
Undecided
Unassigned
Glance
Undecided
Unassigned
OpenStack Compute (nova)
High
Unassigned
OpenStack Core Infrastructure
Fix Released
Undecided
Andreas Jaeger
OpenStack Dashboard (Horizon)
High
Cyril Roelandt
OpenStack Heat
Fix Released
Undecided
Dirk Mueller
OpenStack Identity (keystone)
Medium
Dolph Mathews
django-openstack-auth
Undecided
Dirk Mueller
oslo-incubator
High
Doug Hellmann
oslo.messaging
High
Dirk Mueller
tempest
Undecided
Dirk Mueller
tripleo
Medium
Unassigned
zaqar
Undecided
Sergey Lukjanov

Bug Description

http://lists.openstack.org/pipermail/openstack-dev/2014-January/023759.html

We've decided to rename oslo.sphinx to oslosphinx. This will require small changes in the doc builds for a lot of the other projects.

The problem seems to be when we pip install -e oslo.config on the system, then pip install oslo.sphinx in a venv. oslo.config is unavailable in the venv, apparently because the namespace package for o.s causes the egg-link for o.c to be ignored.

Doug Hellmann (doug-hellmann) wrote :

The git repository has been moved from openstack/oslo.sphinx to openstack/oslosphinx, so we're ready to work on updating consuming projects.

Ben Nemec (bnemec) on 2014-02-07
Changed in nova:
importance: Undecided → High
Changed in nova:
status: New → Confirmed
milestone: none → icehouse-3
tags: added: oslo
Mark McLoughlin (markmc) wrote :
Download full text (3.4 KiB)

Uggh, digging up the details behind this was pretty terrible ...

There is clearly some underlying issue we're papering over here ... but nowhere do I see any analysis of that underlying issue recorded for posterity

For example, in https://review.openstack.org/71589

  "Having the sphinx theme in the same namespace package
    as the production code has been causing issues with
    devstack installations."

The thread linked above ends with http://lists.openstack.org/pipermail/openstack-dev/2013-December/023002.html

 "Basically you can very easily get yourself in a situation where *just
   enough* of the distro package is left behind that pip thinks its there,
   so won't install it, but the python loader doesn't, so won't work.

   Then much sadness."

Or in https://review.openstack.org/#/c/65414/1//COMMIT_MSG

  "After merging https://review.openstack.org/64226 devstack gate started failing."

Oh, now I'm getting somewhere http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-01-08.log

   <dhellmann> fungi: the tl;dr is that when oslo.config is installed globally with "pip install -e" and oslo.sphinx is installed in a virtualenv with "pip install" python in the virtualenv can't find oslo.config
  ...
  <dhellmann> sdague: I think bnemec worked out that it's caused by a combination of pip install -e and something else in the same namespace package not being installed in that mode
  <dhellmann> sdague: alternatives to fix it seem to be install oslo.sphinx with pip install -e or change its package name
  <dhellmann> sdague: or don't install it at all, of course
  <bnemec> dhellmann: sdague: Right, the problem is that we pip install -e oslo.config in the system site packages, then pip install oslo.sphinx in the venv.
  <bnemec> That combination results in oslo.config being unavailable in the venv.
  <dhellmann> fungi: common namespace, split, *and* installed with .egg-link in some cases (I think)
  <dstufft> namespaces don't play well with -e
  <dhellmann> so for normal production oslo libs, we'd just install them from devstack with pip install -e and be done
  <dhellmann> but because oslo.sphinx is not a production lib, that's not necessarily the best answer
  <dhellmann> we should probably just reserve the oslo namespace for production libraries, and rename the theme package
  <dhellmann> well, it could also be said that if devstack didn't use that -e option this wouldn't be an issue, so it's a combination of all of it

And, at last, I find Ben's email ! http://lists.openstack.org/pipermail/openstack-dev/2014-January/023759.html

  The problem seems to be when we pip install -e oslo.config on the
  system, then pip install oslo.sphinx in a venv. oslo.config is
  unavailable in the venv, apparently because the namespace package for
  o.s causes the egg-link for o.c to be ignored. Pretty much every other
  combination I've tried (regular pip install of both, or pip install -e
  of both, regardless of where they are) works fine, but there seem to be
  other issues with all of the other options we've explored so far.

  We can't remove the pip install -e of oslo.config because it has to be
  used for gating...

Read more...

Changed in oslo.messaging:
importance: Undecided → High
status: New → Fix Committed
assignee: nobody → Dirk Mueller (dmllr)
Mark McLoughlin (markmc) on 2014-02-14
description: updated
Mark McLoughlin (markmc) on 2014-02-14
description: updated
Doug Hellmann (doug-hellmann) wrote :
Changed in ceilometer:
assignee: nobody → Dirk (dirk-mueller-riesa)
importance: Undecided → Medium
status: New → Fix Committed
assignee: Dirk (dirk-mueller-riesa) → Dirk Mueller (dmllr)
milestone: none → icehouse-3
Changed in tempest:
assignee: nobody → Dirk Mueller (dmllr)
status: New → In Progress

Reviewed: https://review.openstack.org/72991
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c926216a9bd64d5cdd9731bd34e03613940d32e9
Submitter: Jenkins
Branch: master

commit c926216a9bd64d5cdd9731bd34e03613940d32e9
Author: Ben Nemec <email address hidden>
Date: Fri Feb 7 23:38:39 2014 +0000

    Replace oslo.sphinx with oslosphinx

    oslosphinx is now available as a replacement for oslo.sphinx that
    won't conflict with oslo.config in virtual envs.

    Change-Id: I7c116f816af895261e76af385ee3e9288e6fa70f
    Closes-Bug: #1277168

Changed in nova:
status: Confirmed → Fix Committed
Changed in cinder:
status: New → In Progress
Changed in glance:
status: New → In Progress
Changed in keystone:
status: New → In Progress
Changed in heat:
status: New → In Progress
Changed in django-openstack-auth:
status: New → In Progress
Changed in tripleo:
status: New → In Progress
Changed in oslo.messaging:
milestone: none → icehouse-3
Doug Hellmann (doug-hellmann) wrote :

Dirk is working on patches for all the projects currently listed. I don't know why launchpad won't let me assign the ticket to him.

Dirk Mueller (dmllr) on 2014-02-14
Changed in cinder:
assignee: nobody → Dirk Mueller (dmllr)
Doug Hellmann (doug-hellmann) wrote :

Andreas Jaeger is working on this for the CI docs in https://review.openstack.org/#/c/73709/

Changed in openstack-ci:
status: New → In Progress
Dirk Mueller (dmllr) on 2014-02-14
Changed in heat:
assignee: nobody → Dirk Mueller (dmllr)

Reviewed: https://review.openstack.org/73278
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=de6d92fb924911aaf8bcae72995012b5e1781045
Submitter: Jenkins
Branch: master

commit de6d92fb924911aaf8bcae72995012b5e1781045
Author: Dirk Mueller <email address hidden>
Date: Thu Feb 13 14:39:21 2014 +0100

    Switch over to oslosphinx

    Having the sphinx theme in the same namespace package
    as the production code has been causing issues with
    devstack installations. The solution settled on was
    to rename oslo.sphinx to oslosphinx because oslo.sphinx
    is not a "production" library.

    See the linked bug report for more background details.

    Closes-Bug: #1277168
    Change-Id: I220b8901cef36499e91b92719f1e8e5461a95e92

Reviewed: https://review.openstack.org/73273
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=82ba4286e0ed3b6633ea9a97200fa6301776f4f7
Submitter: Jenkins
Branch: master

commit 82ba4286e0ed3b6633ea9a97200fa6301776f4f7
Author: Dirk Mueller <email address hidden>
Date: Thu Feb 13 14:27:54 2014 +0100

    Switch over to oslosphinx

    oslosphinx is the new name of oslo.sphinx

    Closes-Bug: #1277168
    Change-Id: Ibfb9e59fbb3ef51a821b4f412601e26cb7f703f4

Changed in tempest:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/73282
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=69df886b761dd6ee47a15cad4f11e030f1e1c286
Submitter: Jenkins
Branch: master

commit 69df886b761dd6ee47a15cad4f11e030f1e1c286
Author: Dirk Mueller <email address hidden>
Date: Thu Feb 13 14:44:32 2014 +0100

    Switch over to oslosphinx

    oslosphinx is the new name of oslo.sphinx

    Closes-Bug: #1277168
    Change-Id: I8aa08e3e16d46209b9aeb20a70b3984b2a4ae5bb

Changed in glance:
status: In Progress → Fix Committed
Changed in oslo:
status: In Progress → Fix Committed
Changed in keystone:
assignee: nobody → Dolph Mathews (dolph)

Reviewed: https://review.openstack.org/73277
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=40898f1bd01caa17ad8fec9ec3887a042d8a0386
Submitter: Jenkins
Branch: master

commit 40898f1bd01caa17ad8fec9ec3887a042d8a0386
Author: Dirk Mueller <email address hidden>
Date: Thu Feb 13 14:35:23 2014 +0100

    Switch over to oslosphinx

    oslosphinx is the new name of oslo.sphinx

    Closes-Bug: #1277168
    Change-Id: I9c52b7ad5459c51ed3d2a92d9011c34c9de4de5a

Changed in keystone:
status: In Progress → Fix Committed
Changed in openstack-ci:
assignee: nobody → Andreas Jaeger (jaegerandi)
Ben Nemec (bnemec) wrote :

I think this is actually done for tripleo. I don't see any remaining references to oslo.sphinx in my devtest environment.

Changed in tripleo:
importance: Undecided → Medium
status: In Progress → Fix Committed
Robert Collins (lifeless) wrote :

@dhellman, whats needed for this in tripleo?

Doug Hellmann (doug-hellmann) wrote :

@lifeless - If you're building docs with sphinx using the OpenStack theme contained in oslo.sphinx, that should change to "oslosphinx" (removing the namespace package). I don't see oslo.sphinx being used in tripleo, so you might not need to do anything at all.

Thierry Carrez (ttx) on 2014-03-05
Changed in glance:
milestone: none → icehouse-3
status: Fix Committed → Fix Released

Reviewed: https://review.openstack.org/73367
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=e53f7792ad6156ad7b16a613dd86a477d5d537e3
Submitter: Jenkins
Branch: master

commit e53f7792ad6156ad7b16a613dd86a477d5d537e3
Author: Dirk Mueller <email address hidden>
Date: Thu Feb 13 18:44:59 2014 +0100

    Switch over to oslosphinx

    oslosphinx is now available as a replacement for oslo.sphinx that
    won't conflict with oslo.config in virtual envs

    Change-Id: I7dc3349c8102cc8f358de7e0088128b6705c5f8e
    Closes-Bug: #1277168

Changed in heat:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2014-03-05
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-03-05
Changed in keystone:
milestone: none → icehouse-3
status: Fix Committed → Fix Released
Changed in heat:
milestone: none → icehouse-3
status: Fix Committed → Fix Released
Dolph Mathews (dolph) on 2014-03-05
Changed in keystone:
importance: Undecided → Medium
Thierry Carrez (ttx) on 2014-03-05
Changed in ceilometer:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-03-06
Changed in oslo:
status: Fix Committed → Fix Released
Changed in oslo.messaging:
status: Fix Committed → Fix Released
Doug Hellmann (doug-hellmann) wrote :

tripleo is still using oslo.sphinx in their new os-cloud-config repository

Changed in django-openstack-auth:
assignee: nobody → Zhenguo Niu (niu-zglinux)
Changed in horizon:
assignee: nobody → Zhenguo Niu (niu-zglinux)

Fix proposed to branch: master
Review: https://review.openstack.org/81187

Changed in horizon:
status: New → In Progress
Akihiro Motoki (amotoki) on 2014-03-20
Changed in django-openstack-auth:
assignee: Zhenguo Niu (niu-zglinux) → Dirk Mueller (dmllr)

Reviewed: https://review.openstack.org/73280
Committed: https://git.openstack.org/cgit/openstack/django_openstack_auth/commit/?id=5f5641482c4c841a657daa66752fbab71de3308a
Submitter: Jenkins
Branch: master

commit 5f5641482c4c841a657daa66752fbab71de3308a
Author: Dirk Mueller <email address hidden>
Date: Thu Feb 13 14:41:55 2014 +0100

    Switch over to oslosphinx

    oslosphinx is the new name of oslo.sphinx

    Closes-Bug: #1277168
    Change-Id: I717c92bf62b8820517d1c0f1f83c75713a680ed7

Changed in django-openstack-auth:
status: In Progress → Fix Committed
Changed in tripleo:
status: Fix Committed → Fix Released
Changed in horizon:
assignee: Zhenguo Niu (niu-zglinux) → nobody
Changed in horizon:
assignee: nobody → Cyril Roelandt (cyril-roelandt)
Changed in marconi:
assignee: nobody → Sergey Lukjanov (slukjanov)
status: New → In Progress

Reviewed: https://review.openstack.org/83042
Committed: https://git.openstack.org/cgit/openstack/marconi/commit/?id=733e6622027900e424dff83737b7723431c273f9
Submitter: Jenkins
Branch: master

commit 733e6622027900e424dff83737b7723431c273f9
Author: Sergey Lukjanov <email address hidden>
Date: Wed Mar 26 15:24:15 2014 +0400

    Switch to oslosphinx

    It was renamed.

    Change-Id: I7ed8e5d62979020af2f26921e50efad9a3489410
    Closes-Bug: #1277168

Changed in marconi:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/79303
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=96df098becbc33ecc28a8ea89e352b443cb34a2e
Submitter: Jenkins
Branch: master

commit 96df098becbc33ecc28a8ea89e352b443cb34a2e
Author: Cyril Roelandt <email address hidden>
Date: Mon Mar 10 13:32:08 2014 +0100

    Switch over to oslosphinx

    oslosphinx is the new name of oslo.sphinx

    Closes-Bug: #1277168
    Change-Id: I4cebcffb36724d699147a26eb1132b122cfe669c

Changed in horizon:
status: In Progress → Fix Committed
Akihiro Motoki (amotoki) on 2014-03-28
Changed in horizon:
milestone: none → icehouse-rc1
importance: Undecided → High
Thierry Carrez (ttx) on 2014-03-31
Changed in horizon:
status: Fix Committed → Fix Released
Changed in openstack-ci:
status: In Progress → Fix Released
Thierry Carrez (ttx) on 2014-04-16
Changed in marconi:
milestone: none → icehouse-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-17
Changed in oslo:
milestone: icehouse-3 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in keystone:
milestone: icehouse-3 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in oslo.messaging:
milestone: icehouse-3 → 1.3.0
Thierry Carrez (ttx) on 2014-04-17
Changed in ceilometer:
milestone: icehouse-3 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in nova:
milestone: icehouse-3 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in horizon:
milestone: icehouse-rc1 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in heat:
milestone: icehouse-3 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in glance:
milestone: icehouse-3 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in marconi:
milestone: icehouse-rc1 → 2014.1
Akihiro Motoki (amotoki) on 2014-09-01
Changed in django-openstack-auth:
milestone: none → 1.1.5
David Lyle (david-lyle) on 2015-07-15
Changed in django-openstack-auth:
milestone: 1.1.5 → none
status: Fix Committed → Fix Released
Sean McGinnis (sean-mcginnis) wrote :

Automatically unassigning due to inactivity.

Changed in cinder:
assignee: Dirk Mueller (dmllr) → nobody
status: In Progress → Triaged
Changed in cinder:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers