neutron-ns-metadata-proxy process failing under python3.5

Bug #1661303 reported by Michael Johnson
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
Kevin Benton

Bug Description

When running under python 3.5, we are seeing the neutron-ns-metadata-proxy fail repeatedly on Ocata RC1 master.

This is causing instances to fail to boot under a python3.5 devstack.

A gate example is here:
http://logs.openstack.org/99/407099/25/check/gate-rally-dsvm-py35-neutron-neutron-ubuntu-xenial/4741e0d/logs/screen-q-l3.txt.gz?level=ERROR#_2017-02-02_11_41_52_029

2017-02-02 11:41:52.029 29906 ERROR neutron.agent.linux.external_process [-] metadata-proxy for router with uuid 79af72b9-6b17-4864-8088-5dc96b9271df not found. The process should not have died

Running this locally I see the debug output of the configuration settings and it immediately exits with no error output.

To reproduce:
Stack a fresh devstack with the "USE_PYTHON3=True" setting in your localrc (NOTE: There are other python3x devstack bugs that may reconfigure your host in bad ways once you do this. Plan to only stack with this setting on a throw away host or one you plan to use for Python3.x going forward)

Once this devstack is up and running, setup a neuron network and subnet, then boot a cirros instance on that new subnet.

Check the cirros console.log to see that it cannot find a metadata datasource (Due to this change disabling configdrive: https://github.com/openstack-dev/devstack/commit/7682ea88a6ab8693b215646f16748dbbc2476cc4).

Check the q-l3.txt log to see the repeated "The process should not have died" messages.

You will also note that the cirros instance did not receive it's ssh keys and is requiring password login due to the missing datasource.

Changed in neutron:
milestone: none → ocata-rc1
importance: Undecided → High
tags: added: ocata-rc-potential
Changed in neutron:
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/428504

Changed in neutron:
assignee: nobody → Kevin Benton (kevinbenton)
status: Confirmed → In Progress
tags: added: gate-failure
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/428504
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=33129f2ac6a41afe2f59a0d50d5db0257f588a79
Submitter: Jenkins
Branch: master

commit 33129f2ac6a41afe2f59a0d50d5db0257f588a79
Author: Kevin Benton <email address hidden>
Date: Thu Feb 2 16:12:40 2017 -0800

    Use bytes for python3 friendly os.write

    Bytes not str. Otherwise we get
    TypeError: a bytes-like object is required, not 'str'
    in the metadata proxy and it dies.

    Closes-Bug: #1661303
    Change-Id: If6b6f19130c965436a637a03a4cf72203e0786b0

Changed in neutron:
status: In Progress → Fix Released
tags: added: neutron-easy-proactive-backport-potential
tags: added: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 10.0.0.0rc1

This issue was fixed in the openstack/neutron 10.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/444312

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/mitaka)

Change abandoned by Daniel Alvarez (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/444312
Reason: Wrong branch, I wanted to do it into stable/newton since it's a trivial fix

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/444319

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/newton)

Reviewed: https://review.openstack.org/444319
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fc605900f7187b09c374fb018a25d63484ffb580
Submitter: Jenkins
Branch: stable/newton

commit fc605900f7187b09c374fb018a25d63484ffb580
Author: Kevin Benton <email address hidden>
Date: Thu Feb 2 16:12:40 2017 -0800

    Use bytes for python3 friendly os.write

    Bytes not str. Otherwise we get
    TypeError: a bytes-like object is required, not 'str'
    in the metadata proxy and it dies.

    Closes-Bug: #1661303
    Change-Id: If6b6f19130c965436a637a03a4cf72203e0786b0
    (cherry picked from commit 33129f2ac6a41afe2f59a0d50d5db0257f588a79)

tags: added: in-stable-newton
tags: removed: neutron-easy-proactive-backport-potential neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 9.3.0

This issue was fixed in the openstack/neutron 9.3.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.