test_migration fails with type error

Bug #1503847 reported by Swaminathan Vasudevan on 2015-10-07
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Critical
Kevin Benton
tags: added: py34
removed: neutron
Changed in neutron:
assignee: nobody → Cedric Brandily (cbrandily)
summary: - Python34 test failures in gate - Logging Error
+ test_migration fails with encoding error
tags: added: db
summary: - test_migration fails with encoding error
+ test_migration fails with type error

Fix proposed to branch: master
Review: https://review.openstack.org/232239

Changed in neutron:
assignee: Cedric Brandily (cbrandily) → Armando Migliaccio (armando-migliaccio)
status: Confirmed → In Progress

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/232239
Reason: There's plenty of fixes

tags: added: liberty-backport-potential
tags: removed: db
Changed in neutron:
assignee: Armando Migliaccio (armando-migliaccio) → Ihar Hrachyshka (ihar-hrachyshka)
Changed in neutron:
assignee: Ihar Hrachyshka (ihar-hrachyshka) → Kevin Benton (kevinbenton)

Reviewed: https://review.openstack.org/232265
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8f58bbf38f5984e70d6e0be7427deb99d7782d1d
Submitter: Jenkins
Branch: master

commit 8f58bbf38f5984e70d6e0be7427deb99d7782d1d
Author: Kevin Benton <email address hidden>
Date: Tue Oct 6 20:16:15 2015 -0700

    Fixed multiple py34 gate issues

    1. Scope mock of 'open' to module

    By mocking 'open' at the module level, we can avoid affecting
    'open' calls from other modules.

    2. Stop using LOG.exception in contexts with no sys.exc_info set

    Python 3.4 logger fills in record.exc_info with sys.exc_info() result
    [1], and then it uses it to determine the current exception [2] to
    append to the log message. Since there is no exception, exc_info[1] is
    None, and we get AttributeError inside traceback module.

    It's actually a bug in Python interpreter that it attempt to access the
    attribute when there is no exception. It turns out that it's fixed in
    latest master of cPython [3] (the intent of the patch does not seem
    relevant, but it removes the offending code while reshuffling the code).
    Note that now cPython correctly checks the exception value before
    accessing its attributes [4].

    The patch in cPython that resulted in the failure is [5] and is present
    since initial Python 3k releases.

    The patch in fixtures that broke us is [6].

    [1]: https://hg.python.org/cpython/file/tip/Lib/logging/__init__.py#l1412
    [2]: https://hg.python.org/cpython/file/tip/Lib/logging/__init__.py#l575
    [3]: https://hg.python.org/cpython/rev/73afda5a4e4c
    [4]: https://hg.python.org/cpython/rev/73afda5a4e4c#l6.484
    [5]: https://hg.python.org/cpython/rev/2ee09afee126
    [6]: https://github.com/testing-cabal/fixtures/commit/67dd2956943261e845a866dab155208c51da937e

    Closes-Bug: #1503847
    Closes-Bug: #1504053
    Co-Authored-By: Ihar Hrachyshka <email address hidden>
    Change-Id: I456b7846b8a53e4d3f8c91583685e0e1eaa84719

Changed in neutron:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/232716
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3330c5554518be056b8f2506567bf5f4c1f12db7
Submitter: Jenkins
Branch: master

commit 3330c5554518be056b8f2506567bf5f4c1f12db7
Author: Kevin Benton <email address hidden>
Date: Thu Oct 8 12:31:32 2015 -0700

    Add OpenFixture and get rid of 'open' mocks

    This patch adds a new fixture that allows a specific
    file to be mocked for 'open' calls while leaving the
    normal 'open' behavior for everything else. This is to
    prevent cases like the related bug where a library we
    depend on can unexpectedly call 'open' on its own
    (in that case it was debtcollector->warnings->linecache).

    It also replaces all open mocks in the unit tests with
    this new fixture.

    Related-Bug: #1503847
    Change-Id: Ie9e1a01acd5cb231718efd028d30df6c8dbcd3f0

Thierry Carrez (ttx) on 2015-10-14
Changed in neutron:
milestone: none → liberty-rc3
Download full text (4.2 KiB)

Reviewed: https://review.openstack.org/232270
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=6dcfe3a9362ae5fcf18e5cfb59663e43446cd59c
Submitter: Jenkins
Branch: stable/liberty

commit 6dcfe3a9362ae5fcf18e5cfb59663e43446cd59c
Author: Kevin Benton <email address hidden>
Date: Tue Oct 6 19:28:47 2015 -0700

    Mock oslo policy HTTPCheck instead of urllib

    We were mocking internal behavior of oslo policy by
    patching urllib. This will break with the upcoming oslo
    release that switches to requests.

    This patch changes the mock to the HTTPCheck level and we
    can leave implementation details testing up to oslo_policy.

    Change-Id: I07957f01307e25f1547197c720eea6e3e7f0ef5a
    Closes-Bug: #1503890
    (cherry picked from commit a0f1d9d6de1560be91d3001c8ac9f880a7a5a7e0)

    Add testresources used by oslo.db fixture

    If we use oslo.db fixtures, we'll need the package or
    the next version of oslo.db release will break us.

    Closes-Bug: #1503501
    Change-Id: I7dfbf240333095d91a414ba15a439bdc4804eb25
    (cherry picked from commit 86ad967e40c2c6752ec0fb46cfd3098ede0c7178)

    Fix functional test_server tests

    Now oslo.service 0.10.0 no longer sends SIGHUP to parent and
    children services.

    This was a chance introduced by 286a6ea, and since it invalidated
    the very logic under test, this must be revised.

    (cherry picked from commit 090fe713592c2b6398d999bfa03b80cbb2054609)

    Change-Id: I18a11283925369bc918002477774f196010a1bc3
    Closes-bug: #1505438
    (cherry picked from commit 090fe713592c2b6398d999bfa03b80cbb2054609)

    Make test_server work with older versions of oslo.service

    Change I18a11283925369bc918002477774f196010a1bc3 fixed the test for
    oslo.service >= 0.10.0, but it also broke it for older versions of
    oslo.service. Since the library has minimal version of >= 0.7.0 in
    requirements.txt, test should pass for those versions too.

    Now, instead of validating that either reset() or restart() of workers
    are triggered on SIGHUP, just validate that .start() is triggered the
    expected number of times (either way, no matter how oslo.service decide
    to clean up the children, they exit and then are respawned).

    Change-Id: I41f9d3af780b3178b075bc1e7084f417a2bd1378
    Closes-Bug: #1505645
    (cherry picked from commit 7bb40921660cf29beb68e338e205499efd6ffa36)

    Fixed multiple py34 gate issues

    1. Scope mock of 'open' to module

    By mocking 'open' at the module level, we can avoid affecting
    'open' calls from other modules.

    2. Stop using LOG.exception in contexts with no sys.exc_info set

    Python 3.4 logger fills in record.exc_info with sys.exc_info() result
    [1], and then it uses it to determine the current exception [2] to
    append to the log message. Since there is no exception, exc_info[1] is
    None, and we get AttributeError inside traceback module.

    It's actually a bug in Python interpreter that it attempt to access the
    attribute when there is no exception. It turns out that it's fixed in
    latest master of cPyth...

Read more...

tags: added: in-stable-liberty
Thierry Carrez (ttx) on 2015-10-14
Changed in neutron:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2015-10-15
Changed in neutron:
milestone: liberty-rc3 → 7.0.0
Download full text (16.6 KiB)

Reviewed: https://review.openstack.org/235300
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=140ccc36d172bead2605968b3d61b36cca8a0040
Submitter: Jenkins
Branch: master

commit 6dcfe3a9362ae5fcf18e5cfb59663e43446cd59c
Author: Kevin Benton <email address hidden>
Date: Tue Oct 6 19:28:47 2015 -0700

    Mock oslo policy HTTPCheck instead of urllib

    We were mocking internal behavior of oslo policy by
    patching urllib. This will break with the upcoming oslo
    release that switches to requests.

    This patch changes the mock to the HTTPCheck level and we
    can leave implementation details testing up to oslo_policy.

    Change-Id: I07957f01307e25f1547197c720eea6e3e7f0ef5a
    Closes-Bug: #1503890
    (cherry picked from commit a0f1d9d6de1560be91d3001c8ac9f880a7a5a7e0)

    Add testresources used by oslo.db fixture

    If we use oslo.db fixtures, we'll need the package or
    the next version of oslo.db release will break us.

    Closes-Bug: #1503501
    Change-Id: I7dfbf240333095d91a414ba15a439bdc4804eb25
    (cherry picked from commit 86ad967e40c2c6752ec0fb46cfd3098ede0c7178)

    Fix functional test_server tests

    Now oslo.service 0.10.0 no longer sends SIGHUP to parent and
    children services.

    This was a chance introduced by 286a6ea, and since it invalidated
    the very logic under test, this must be revised.

    (cherry picked from commit 090fe713592c2b6398d999bfa03b80cbb2054609)

    Change-Id: I18a11283925369bc918002477774f196010a1bc3
    Closes-bug: #1505438
    (cherry picked from commit 090fe713592c2b6398d999bfa03b80cbb2054609)

    Make test_server work with older versions of oslo.service

    Change I18a11283925369bc918002477774f196010a1bc3 fixed the test for
    oslo.service >= 0.10.0, but it also broke it for older versions of
    oslo.service. Since the library has minimal version of >= 0.7.0 in
    requirements.txt, test should pass for those versions too.

    Now, instead of validating that either reset() or restart() of workers
    are triggered on SIGHUP, just validate that .start() is triggered the
    expected number of times (either way, no matter how oslo.service decide
    to clean up the children, they exit and then are respawned).

    Change-Id: I41f9d3af780b3178b075bc1e7084f417a2bd1378
    Closes-Bug: #1505645
    (cherry picked from commit 7bb40921660cf29beb68e338e205499efd6ffa36)

    Fixed multiple py34 gate issues

    1. Scope mock of 'open' to module

    By mocking 'open' at the module level, we can avoid affecting
    'open' calls from other modules.

    2. Stop using LOG.exception in contexts with no sys.exc_info set

    Python 3.4 logger fills in record.exc_info with sys.exc_info() result
    [1], and then it uses it to determine the current exception [2] to
    append to the log message. Since there is no exception, exc_info[1] is
    None, and we get AttributeError inside traceback module.

    It's actually a bug in Python interpreter that it attempt to access the
    attribute when there is no exception. It turns out that it's fixed in
    latest master of cPython [3] (...

This issue was fixed in the openstack/neutron 8.0.0.0b1 development milestone.

tags: removed: liberty-backport-potential
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers