Nodes do not discovered with 400 bad request from agent on bootstrap

Bug #1292010 reported by Tatyanka
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Critical
Ihor Kalnytskyi

Bug Description

{"build_id": "2014-03-13_03-01-16", "mirantis": "yes", "build_number": "26", "nailgun_sha": "80ab77501960c1981d513eb2cc2ee6df5f3002ca", "ostf_sha": "4d4633f2410ecb1c5aa669f53319a7826ba509e6", "fuelmain_sha": "1678746a5080d22b8d350612c15447ac0b4f8271", "astute_sha": "97c91351062ed3f3bf745dc58a3c0e6387a9801a", "release": "5.0", "fuellib_sha": "faa0ecd9beacc9327076b8a29ba99df3ad27f6b3"}

Steps to Reproduce:
1. setup master node(kvm)
2. enable several slaves and wait until it bootstraped
3. wait while nodes discovered

Current Result:
nodes are not discovered with next error on bootsrapped nodes in agent.log
2014-03-13T15:32:50.737794+00:00 debug: 15:32:50.560163 #1066] DEBUG -- : Found block device: ram0
2014-03-13T15:32:50.737876+00:00 debug: 15:32:50.560235 #1066] DEBUG -- : Block device info: {"size"=>"32768", "removable"=>"0"}
2014-03-13T15:32:50.737967+00:00 debug: 15:32:50.560373 #1066] DEBUG -- : Detailed meta disks: [{:disk=>"disk/by-path/pci-0000:00:0a.0-virtio-pci-virtio7", :removable=>"0", :extra=>[], :size=>53687091200, :name=>"vdc", :model=>nil}, {:disk=>"disk/by-path/pci-0000:00:09.0-virtio-pci-virtio6", :removable=>"0", :extra=>[], :size=>53687091200, :name=>"vdb", :model=>nil}, {:disk=>"disk/by-path/pci-0000:00:08.0-virtio-pci-virtio5", :removable=>"0", :extra=>[], :size=>53687091200, :name=>"vda", :model=>nil}]
2014-03-13T15:32:50.738076+00:00 debug: 15:32:50.582677 #1066] DEBUG -- : Response: status: 400 body: None is not of type 'string'
2014-03-13T15:32:50.738135+00:00 info:
2014-03-13T15:32:50.738217+00:00 info: Failed validating 'type' in schema['properties']['meta']['properties']['disks']['items']['properties']['model']:
2014-03-13T15:32:50.738296+00:00 info: {'type': 'string'}
2014-03-13T15:32:50.738371+00:00 info:
2014-03-13T15:32:50.738471+00:00 info: On instance['meta']['disks'][0]['model']:
2014-03-13T15:32:50.738571+00:00 info: None
2014-03-13T15:32:50.738614+00:00 err: 15:32:50.582716 #1066] ERROR -- : None is not of type 'string'
2014-03-13T15:32:50.738688+00:00 info:
2014-03-13T15:32:50.738768+00:00 info: Failed validating 'type' in schema['properties']['meta']['properties']['disks']['items']['properties']['model']:
2014-03-13T15:32:50.738847+00:00 info: {'type': 'string'}
2014-03-13T15:32:50.738933+00:00 info:
2014-03-13T15:32:50.739008+00:00 info: On instance['meta']['disks'][0]['model']:
2014-03-13T15:32:50.739085+00:00 info: None
2014-03-13T15:33:20.785677+00:00 info: 15:33:01.669375 #1256] INFO -- : Trying to load agent config /etc/nailgun-agent/config.yaml

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
Revision history for this message
Ihor Kalnytskyi (ikalnytskyi) wrote :

The issue occurs because of node tries to post invalid disk's model.

I guess we have to discuss how to fix it. I see two ways:

1/ Change validation schema to allow nodes to post invalid information (such as model name of disk).

2/ Fix our agent to prevent posting invalid information.

Changed in fuel:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
Nastya Urlapova (aurlapova) wrote :

>>>nodes are not discovered - critical point

Changed in fuel:
importance: Medium → Critical
Revision history for this message
Evgeniy L (rustyrobot) wrote :

Agree, it's critical.

Igor, could you please provide a fix where null is allowed value for disk model field?

Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Igor Kalnitsky (ikalnitsky)
milestone: 4.1.1 → 5.0
Revision history for this message
Evgeniy L (rustyrobot) wrote :

It should be fixed not in 4.1.1, because we have unbroken code in 4.1, moved to 5.0.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/80522

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/80522
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=06d22c86f0875943b91395cc1dc5fe92df0cefa9
Submitter: Jenkins
Branch: master

commit 06d22c86f0875943b91395cc1dc5fe92df0cefa9
Author: Igor Kalnitsky <email address hidden>
Date: Fri Mar 14 10:35:54 2014 +0200

    Allow nodes to send `null` in disk model field

    Sometimes our discovery agents collects invalid data and sends it to
    Nailgun. Because of adding JsonSchema for validation node model, the
    requests with partially invalid data will fail and agents will not
    be able to register themself.

    Closes-Bug: #1292010
    Change-Id: I319b142628e29e6706df71f0336a4d3ea71ddc22

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

verified on 29 iso - works fine. Thanks .

Changed in fuel:
status: Fix Committed → Fix Released
milestone: 5.0 → 4.1.1
Revision history for this message
Egor Kotko (ykotko) wrote :

Checked on:
{"build_id": "2014-03-14_13-49-12", "mirantis": "yes", "build_number": "29", "nailgun_sha": "06d22c86f0875943b91395cc1dc5fe92df0cefa9", "ostf_sha": "0ad6f6de46e615022c2672c345a11fb44ffaa4e1", "fuelmain_sha": "d67783166401529d1c0fe82b8a4885b6573ad7b4", "astute_sha": "e260b36adb73bb32aca1660767a988e1fce736d8", "release": "5.0", "fuellib_sha": "45320c9167979d73c6768e251f61ba2c8f928626"}

Changed in fuel:
milestone: 4.1.1 → 5.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.