Neutron functional job is broken
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
neutron |
Fix Released
|
Critical
|
Maru Newby | ||
Icehouse |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
I encountered this as part of "check-
Contextual log snippet (from console.html)[1]:
-------
.
.
.
2014-07-01 03:31:05.323 | 2014-07-01 03:31:05.304 | Command: ['sudo', 'ovs-vsctl', '--timeout=10', 'get', 'Interface', 'peertest1-e9c183', 'ofport']
2014-07-01 03:31:05.325 | 2014-07-01 03:31:05.306 | Exit code: 1
2014-07-01 03:31:05.327 | 2014-07-01 03:31:05.308 | Stdout: ''
2014-07-01 03:31:05.329 | 2014-07-01 03:31:05.310 | Stderr: 'sudo: no tty present and no askpass program specified\n'
2014-07-01 03:31:05.331 | 2014-07-01 03:31:05.312 | 2014-07-01 03:31:05,077 ERROR [neutron.
2014-07-01 03:31:05.333 | 2014-07-01 03:31:05.314 | Command: ['sudo', 'ovs-vsctl', '--timeout=10', '--', '--if-exists', 'del-br', 'patchtest-e9c183']
2014-07-01 03:31:05.336 | 2014-07-01 03:31:05.316 | Exit code: 1
2014-07-01 03:31:05.338 | 2014-07-01 03:31:05.319 | Stdout: ''
2014-07-01 03:31:05.340 | 2014-07-01 03:31:05.321 | Stderr: 'sudo: no tty present and no askpass program specified\n'
2014-07-01 03:31:05.342 | 2014-07-01 03:31:05.323 | 2014-07-01 03:31:05,078 ERROR [neutron.
2014-07-01 03:31:05.345 | 2014-07-01 03:31:05.325 | Command: ['sudo', 'ovs-vsctl', '--timeout=10', '--', '--if-exists', 'del-br', 'patchtest-e9c183']
2014-07-01 03:31:05.347 | 2014-07-01 03:31:05.328 | Exit code: 1
2014-07-01 03:31:05.349 | 2014-07-01 03:31:05.330 | Stdout: ''
2014-07-01 03:31:05.351 | 2014-07-01 03:31:05.332 | Stderr: 'sudo: no tty present and no askpass program specified\n'
2014-07-01 03:31:05.353 | 2014-07-01 03:31:05.334 | }}}
2014-07-01 03:31:05.355 | 2014-07-01 03:31:05.336 |
2014-07-01 03:31:05.357 | 2014-07-01 03:31:05.338 | Traceback (most recent call last):
2014-07-01 03:31:05.359 | 2014-07-01 03:31:05.340 | File "neutron/
2014-07-01 03:31:05.361 | 2014-07-01 03:31:05.342 | checks.
2014-07-01 03:31:05.363 | 2014-07-01 03:31:05.343 | File "neutron/
2014-07-01 03:31:05.365 | 2014-07-01 03:31:05.346 | port = br.add_
2014-07-01 03:31:05.367 | 2014-07-01 03:31:05.348 | File "neutron/
2014-07-01 03:31:05.369 | 2014-07-01 03:31:05.350 | return self.get_
2014-07-01 03:31:05.371 | 2014-07-01 03:31:05.352 | File "neutron/
2014-07-01 03:31:05.373 | 2014-07-01 03:31:05.354 | int(ofport)
2014-07-01 03:31:05.375 | 2014-07-01 03:31:05.356 | TypeError: int() argument must be a string or a number, not 'NoneType'
2014-07-01 03:31:05.377 | 2014-07-01 03:31:05.358 | =======
2014-07-01 03:31:05.379 | 2014-07-01 03:31:05.360 | FAIL: process-returncode
2014-07-01 03:31:05.381 | 2014-07-01 03:31:05.362 | tags: worker-2
2014-07-01 03:31:05.383 | 2014-07-01 03:31:05.363 | -------
2014-07-01 03:31:05.385 | 2014-07-01 03:31:05.366 | Binary content:
2014-07-01 03:31:05.387 | 2014-07-01 03:31:05.368 | traceback (test/plain; charset="utf8")
2014-07-01 03:31:05.389 | 2014-07-01 03:31:05.369 | =======
2014-07-01 03:31:05.391 | 2014-07-01 03:31:05.371 | FAIL: process-returncode
2014-07-01 03:31:05.393 | 2014-07-01 03:31:05.374 | tags: worker-2
2014-07-01 03:31:05.395 | 2014-07-01 03:31:05.376 | -------
2014-07-01 03:31:05.397 | 2014-07-01 03:31:05.378 | Binary content:
2014-07-01 03:31:05.399 | 2014-07-01 03:31:05.380 | traceback (test/plain; charset="utf8")
2014-07-01 03:31:05.401 | 2014-07-01 03:31:05.382 | Ran 9 tests in 0.367s
2014-07-01 03:31:05.403 | 2014-07-01 03:31:05.384 | FAILED (id=0, failures=4, skips=2)
2014-07-01 03:31:05.405 | 2014-07-01 03:31:05.386 | error: testr failed (1)
2014-07-01 03:31:05.408 | 2014-07-01 03:31:05.388 | ERROR: InvocationError: '/opt/stack/
2014-07-01 03:31:05.409 | 2014-07-01 03:31:05.390 | _______
2014-07-01 03:31:05.411 | 2014-07-01 03:31:05.392 | ERROR: functional: commands failed
.
.
.
-------
Here's[2] the logstash query for "int() argument must be a string or a number, not 'NoneType'" -- looking at, 'build_name' in the logs, most of them are 'check-
[1] http://
[2] http://
summary: |
neutron/agent/linux/ovs_lib.py: int(ofport) returns 'NoneType' instead - of string or number + of string or a number |
Changed in neutron: | |
assignee: | nobody → Henry Gessau (gessau) |
Changed in neutron: | |
importance: | Undecided → Critical |
tags: | added: gate-failure |
Changed in neutron: | |
milestone: | none → juno-2 |
Changed in neutron: | |
milestone: | juno-2 → juno-3 |
Changed in neutron: | |
status: | In Progress → Fix Committed |
Changed in neutron: | |
status: | Fix Committed → Fix Released |
Changed in neutron: | |
milestone: | juno-3 → 2014.2 |
Trying to understand the Traceback:
'get_port_ofport' -- the function where it's 'ofport' is returning None type. agent/linux/ ovs_lib. py) ofport( self, port_name): get_val( "Interface" , port_name, "ofport") INVALID_ OFPORT
(from ./neutron/
---------------
178 def get_port_
179 ofport = self.db_
180 # This can return a non-integer string, like '[]' so ensure a
181 # common failure case
182 try:
183 int(ofport)
184 return ofport
185 except ValueError:
186 return constants.
--------------
So, 'ofport' value to be obtained from 'db_get_val' function (refer
below).
'get_port_ofport' is called from 'add_patch_port' (from agent/linux/ ovs_lib. py) port(self, local_name, remote_name): vsctl([ "add-port" , self.br_name, local_name, port_ofport( local_name)
./neutron/
--------------
271 def add_patch_
272 self.run_
273 "--", "set", "Interface", local_name,
274 "type=patch", "options:peer=%s" % remote_name])
275 return self.get_
---------------
'add_patch_port' is called from 'patch_supported (from cmd/sanity/ checks. py) (root_helper) : random_ string( 6) OVSBridge( name, root_helper) as br: patch_port( patch_name, peer_name) INVALID_ OFPORT
./neutron/
--------------
29 def patch_supported
30 seed = utils.get_
31 name = "patchtest-" + seed
32 peer_name = "peertest0-" + seed
33 patch_name = "peertest1-" + seed
34 with ovs_lib.
35 port = br.add_
36 return port != ovs_const.
--------------
'patch_supported' is called from 'test_ovs_ patch_support_ runs' (from tests/functiona l/sanity/ test_sanity. py) patch_support_ runs(self) : patch_supported (self.root_ helper)
./neutron/
--------------
57 def test_ovs_
58 checks.
--------------
db_get_val: vsctl([ "get", table, record, column], check_error) rstrip( "\n\r")
--------------
284 def db_get_val(self, table, record, column, check_error=False):
285 output = self.run_
286 if output:
287 return output.
--------------