neutron

Key error when try to assign a floating ip to a VM

Bug #1629484 reported by Hao Chen on 2016-09-30

This bug affects 9 people

Affects		Status	Importance	Assigned to	Milestone
	neutron	Fix Released	High	Hao Chen	neutron ocata-1 "o1"

Bug Description

We have an environment with designate and DNS plugin enabled. When we upgrade to newton we got a problem that we can't assign the floating ip to the VM because of an key error.

Here is the error message from neutron server

./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource Traceback (most recent call last):
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/api/v2/resource.py", line 79, in resource
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource result = method(request=request, **args)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/api/v2/base.py", line 604, in update
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return self._update(request, id, body, **kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 88, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource self.force_reraise()
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 84, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return f(*args, **kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 151, in wrapper
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource self.force_reraise()
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 139, in wrapper
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return f(*args, **kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 124, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource traceback.format_exc())
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource self.force_reraise()
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 119, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/api/v2/base.py", line 652, in _update
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource obj = obj_updater(request.context, id, **kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 159, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return method(*args, **kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 88, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource self.force_reraise()
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 84, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return f(*args, **kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 151, in wrapper
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource self.force_reraise()
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 139, in wrapper
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return f(*args, **kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 124, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource traceback.format_exc())
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource self.force_reraise()
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 119, in wrapped
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 1015, in update_floatingip
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource context, id, floatingip)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1355, in _update_floatingip
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource context, floatingip_db)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/dns_db.py", line 210, in _process_dns_floatingip_update_precommit
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource context, floatingip_data))
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/db/dns_db.py", line 313, in _get_requested_state_for_external_dns_service_update
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource if floatingip_data['port_id']:
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 54, in __getitem__
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource return getattr(self, key)
./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource AttributeError: 'FloatingIP' object has no attribute 'port_id'

After digging more it seems like there are several places in the code try to get the value for "port_id" which doesn't exist

https://github.com/openstack/neutron/blob/stable/newton/neutron/db/dns_db.py#L313
https://github.com/openstack/neutron/blob/stable/newton/neutron/db/dns_db.py#L275
https://github.com/openstack/neutron/blob/stable/newton/neutron/db/dns_db.py#L281

from https://github.com/openstack/neutron/blob/stable/newton/neutron/db/l3_db.py#L1041 we can see that port_id is actually the "fix_port_id" attribute in floatingip_db object.

Fix the previous key I mentioned will fix the floating ip assignment.

See original description

Tags:

Hao Chen (chenh1987) on 2016-09-30

Changed in neutron:
assignee:	nobody → Hao Chen (chenh1987)

Abhishek Chanda (abhishek-i) on 2016-09-30

description:

updated

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-30: Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/380669

Changed in neutron:
status:	New → In Progress

Ihar Hrachyshka (ihar-hrachyshka) on 2016-10-03

tags:	added: db l3-ipam-dhcp newton-rc-potential
Changed in neutron:
importance:	Undecided → High
tags:	added: dns

Armando Migliaccio (armando-migliaccio) on 2016-10-03

Changed in neutron:
milestone:	none → ocata-1

Revision history for this message

Miguel Lavalle (minsel) wrote on 2016-10-03:

Unit tests in this patchset can be used to verify the fix for this bug: https://review.openstack.org/#/c/301809

Armando Migliaccio (armando-migliaccio) on 2016-10-03

tags:

added: newton-backport-potential
removed: newton-rc-potential

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-04: Fix merged to neutron (master)

Reviewed: https://review.openstack.org/380669
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=19af726e39b30670314b3bf7025b3a682e523953
Submitter: Jenkins
Branch: master

commit 19af726e39b30670314b3bf7025b3a682e523953
Author: chen, hao <email address hidden>
Date: Fri Sep 30 16:06:38 2016 -0700

Fix the 'port_id' attribute not exist error

Change-Id: Ie9e42771788d8a20572cb650b15edb95923dd426
Closes-Bug: #1629484

Changed in neutron:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-04: Fix proposed to neutron (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/381829

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-06: Fix merged to neutron (stable/newton)

Reviewed: https://review.openstack.org/381829
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1865eb5de087d5f2e7d47ef27f7a3ac2d8dd6996
Submitter: Jenkins
Branch: stable/newton

commit 1865eb5de087d5f2e7d47ef27f7a3ac2d8dd6996
Author: chen, hao <email address hidden>
Date: Fri Sep 30 16:06:38 2016 -0700

Fix the 'port_id' attribute not exist error

    Change-Id: Ie9e42771788d8a20572cb650b15edb95923dd426
    Closes-Bug: #1629484
    (cherry picked from commit 19af726e39b30670314b3bf7025b3a682e523953)

tags:

added: in-stable-newton

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-11-04: Fix included in openstack/neutron 9.1.0

This issue was fixed in the openstack/neutron 9.1.0 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-11-16: Fix included in openstack/neutron 10.0.0.0b1

This issue was fixed in the openstack/neutron 10.0.0.0b1 development milestone.

Ihar Hrachyshka (ihar-hrachyshka) on 2017-01-17

tags:

added: neutron-proactive-backport-potential

Daniel Alvarez (dalvarezs) on 2017-03-10

tags:	removed: in-stable-newton neutron-proactive-backport-potential newton-backport-potential
tags:	added: in-stable-newton

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.