Spawn virt machine on the slave node failed with 404 HTTP error

Bug #1529869 reported by Alexander Kurenyshev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Dmitry Kalashnik

Bug Description

Tests:
Spawn one vm node on one slave node
Spawn two vm nodes on one slave node
Spawn three vm nodes across three slave nodes
failed with 404 HTTP error while uploading instance detailes

on attemps put the data to node
2015-12-29 18:19:32.634 DEBUG [7f8316480880] (logger) Response code '200 OK' for GET /api/nodes/ from 10.109.15.1:39624
2015-12-29 18:19:32.645 DEBUG [7f8316480880] (logger) Request PUT /api/nodes/1/vms_conf/ from 10.109.15.1:39625 {"vms_conf": [{"mem": 1, "id": 1, "cpu": 1}]}
2015-12-29 18:19:32.653 DEBUG [7f8316480880] (logger) Response code '404 Not Found' for PUT /api/nodes/1/vms_conf/ from 10.109.15.1:39625
2015-12-29 18:19:32.656 DEBUG [7f8316480880] (logger) Request PUT /api/logs/package from 10.109.15.1:39626 {}
2015-12-29 18:19:32.735 DEBUG [7f8316480880] (logger) Response code '202 Accepted' for PUT /api/logs/package from 10.109.15.1:39626

at the same time - it works fine over UI in debugger:
http://10.109.0.2:8000/api/nodes/19/vms_conf with request body {"vms_conf":[{"mem":1,"id":1}]} success code and vms_conf is updated
Failed job:
https://product-ci.infra.mirantis.net/job/8.0.system_test.ubuntu.reduced_footprint/89/console

Traceback (most recent call last):
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/8.0.system_test.ubuntu.reduced_footprint/fuelweb_test/helpers/decorators.py", line 81, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/8.0.system_test.ubuntu.reduced_footprint/fuelweb_test/tests/test_reduced_footprint.py", line 83, in spawn_one_vm_on_one_virt_node
    "cpu": 1
  File "/home/jenkins/workspace/8.0.system_test.ubuntu.reduced_footprint/fuelweb_test/__init__.py", line 57, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/8.0.system_test.ubuntu.reduced_footprint/fuelweb_test/helpers/decorators.py", line 131, in wrapped
    response = func(*args, **kwargs)
  File "/home/jenkins/workspace/8.0.system_test.ubuntu.reduced_footprint/fuelweb_test/models/nailgun_client.py", line 549, in create_vm_nodes
    return self.client.put(url, {'vms_conf': data})
  File "/home/jenkins/workspace/8.0.system_test.ubuntu.reduced_footprint/fuelweb_test/helpers/http.py", line 83, in put
    return self._open(req)
  File "/home/jenkins/workspace/8.0.system_test.ubuntu.reduced_footprint/fuelweb_test/helpers/http.py", line 92, in _open
    return self._get_response(req)
  File "/home/jenkins/workspace/8.0.system_test.ubuntu.reduced_footprint/fuelweb_test/helpers/http.py", line 116, in _get_response
    return self.opener.open(req)
  File "/usr/lib/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found

Fuel ISO #365
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "370"
  build_id: "370"
  fuel-nailgun_sha: "e15b205d8a9ccea5f096348500602745a8a6dc61"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "1d2f6bc1123b9364ba65b41afcc0608662bb25b9"
  fuel-nailgun-agent_sha: "92ebd5ade6fab60897761bfa084aefc320bff246"
  astute_sha: "c7ca63a49216744e0bfdfff5cb527556aad2e2a5"
  fuel-library_sha: "25dee09644d355d2eef9f8f189ee2bf2a27f2b8c"
  fuel-ostf_sha: "889ddb0f1a4fa5f839fd4ea0c0017a3c181aa0c1"
  fuel-mirror_sha: "8bb8c70efc61bcf633e02d6054dbf5ec8dcf6699"
  fuelmenu_sha: "f69e2ffb4a64cb66bd23f9762737504482b200a9"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "9f0ba4577915ce1e77f5dc9c639a5ef66ca45896"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "5dd650b541d40251e07afde4072d948be577dc0c"

Revision history for this message
Alexander Kurenyshev (akurenyshev) wrote :
Revision history for this message
Alexander Kurenyshev (akurenyshev) wrote :
Revision history for this message
Alexander Kurenyshev (akurenyshev) wrote :
tags: added: area-library feature-reduced-footprint
Changed in fuel:
importance: Undecided → High
description: updated
description: updated
tags: added: area-qa
removed: area-library
Changed in fuel:
assignee: Fuel QA Team (fuel-qa) → Dmitry Kalashnik (dkalashnik)
Revision history for this message
Sergey Shevorakov (sshevorakov) wrote :

Swarm blocker since fails 3 TCs.

tags: added: swarm-blocker
Changed in fuel:
status: New → Confirmed
Revision history for this message
Dmitry Kalashnik (dkalashnik) wrote :

Root cause is: https://bugs.launchpad.net/fuel/+bug/1526874

[root@nailgun ~]# fuel2 node create-vms-conf 1 --conf '{"id": 1, "mem": 1, "cpu": 1}'
404 Client Error: Not Found

[root@nailgun ~]# dockerctl restart nailgun
fuel-core-8.0-nailgun

[root@nailgun ~]# fuel2 --debug node create-vms-conf 1 --conf '{"id": 1, "mem": 1, "cpu": 1}'
{u'vms_conf': [{u'mem': 1, u'id': 1, u'cpu': 1}]}

Revision history for this message
Dmitry Kalashnik (dkalashnik) wrote :

Test failed with the last iso containing fix for 1526874
Investigation is going

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-qa (master)

Reviewed: https://review.openstack.org/262736
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=651f112e2ed2b1512f51dd0150e1b03096f2a848
Submitter: Jenkins
Branch: master

commit 651f112e2ed2b1512f51dd0150e1b03096f2a848
Author: Dmitry Kalashnik <email address hidden>
Date: Thu Dec 31 13:09:59 2015 +0300

    Add wait for feature_group enabled

    Change-Id: I08558ff0df57cad50163f1062f93a004bae145a7
    Closes-Bug: #1529869

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-qa (stable/8.0)

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/266780

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-qa (stable/8.0)

Reviewed: https://review.openstack.org/266780
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=ce315c50e941bae793856efd9e8852f57cf89f24
Submitter: Jenkins
Branch: stable/8.0

commit ce315c50e941bae793856efd9e8852f57cf89f24
Author: Dmitry Kalashnik <email address hidden>
Date: Thu Dec 31 13:09:59 2015 +0300

    Add wait for feature_group enabled

    Change-Id: I08558ff0df57cad50163f1062f93a004bae145a7
    Closes-Bug: #1529869
    (cherry picked from commit 651f112e2ed2b1512f51dd0150e1b03096f2a848)

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.