plugin metadata is broken

Bug #1544505 reported by Matthew Mosesohn on 2016-02-11
48
This bug affects 9 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Critical
Bulat Gaifullin
8.0.x
Critical
Bulat Gaifullin
Mitaka
Critical
Bulat Gaifullin

Bug Description

Steps to reproduce:
1 - Deploy 9.0-liberty iso 204 with detach-database plugin using all other defaults
2 - Add 1 controller, 1 database node
3 - Deploy

Expected results:
 detach-database hash in astute.yaml on nodes
Actual results:
 plugin is listed in plugins array, but no metadata is present

This was tested across multiple plugins, but I'm reporting this against a well known, established plugin.

Matthew Mosesohn (raytrac3r) wrote :

Note that this does NOT affect any 8.0 environments at this time.

Changed in fuel:
milestone: none → 9.0
assignee: nobody → Fuel Python Team (fuel-python)
Matthew Mosesohn (raytrac3r) wrote :
Ilya Kutukov (ikutukov) on 2016-02-11
tags: added: area-python
Ilya Kutukov (ikutukov) wrote :

Here is attributes_metadata field in database, it seems that validator output was recorded instead of metadata itself

{"yaml_additional_config": {"regex": {"source": ".+", "error": "This field should not be empty"}, "description": "Input must be valid YAML", "weight": 52, "type": "textarea", "valu
e": "#Uncomment to enable remote DB:\n#remote_db: '10.20.99.1'\n#remote_db_password: 'secret'\ncustom_mysql_setup_class: 'galera'\n", "label": "Additional config"}, "metadata": {"we
ight": 90, "label": "Detach Database Plugin"}}

Ilya Kutukov (ikutukov) wrote :

the previous comment is invalid this is a hints for the gui

Ilya Kutukov (ikutukov) wrote :

Not reproduced on 8.0 RC1 (529 build)

Ilya Kutukov (ikutukov) wrote :

If the problem occurred in 2 weeks timeframe as Matthew says this commits could be the reason, because they are affecting cluster attributes processing:

Merge "Changed JSON on Mutable(Dict/List) in Cluster object"
https://github.com/openstack/fuel-web/commit/5e88734cf4c3556ae1b888fc11ff0e52f228c933

Merge "Fixed constructing of a Mutable objects tree"
https://github.com/openstack/fuel-web/commit/b0a0b3cac16dc21cab33b3430080cfb8d9398d39

Ilya Kutukov (ikutukov) wrote :

DB dump of the environment

tags: added: feature-plugins
Changed in fuel:
status: Confirmed → In Progress
Ilya Kutukov (ikutukov) wrote :

Please, be aware that "Changed JSON on Mutable(Dict/List) in Cluster object" patch could break not only plug-ins metadata, but any part of logic that working with data from JSON fields.

@Ilya

Plugin developers are blocked by this bug now and full covering of changed logic will take a time, thus I propose to land this fix for critical bug and implement checking for Mutable DB fields as separate bug: https://bugs.launchpad.net/fuel/+bug/1546063

Ilya Kutukov (ikutukov) wrote :

Is mutable JSON patch roll-back an option?
I don't think there is any new logic that work only with mutable fields.

Vladimir Kuklin (vkuklin) wrote :

Folks, I think that we need to revert the patch that caused the problem as this is a critical regression. After that we can fix all the stuff that needs to fixed, test things properly and land the proper code.

Fix proposed to branch: master
Review: https://review.openstack.org/281447

Changed in fuel:
assignee: Alexander Kislitsky (akislitsky) → Bulat Gaifullin (bgaifullin)
Changed in fuel:
assignee: Bulat Gaifullin (bgaifullin) → Alexander Kislitsky (akislitsky)
Changed in fuel:
assignee: Alexander Kislitsky (akislitsky) → Bulat Gaifullin (bgaifullin)
Changed in fuel:
assignee: Bulat Gaifullin (bgaifullin) → Alexander Kislitsky (akislitsky)
Changed in fuel:
assignee: Alexander Kislitsky (akislitsky) → Bulat Gaifullin (bgaifullin)
Changed in fuel:
assignee: Bulat Gaifullin (bgaifullin) → Alexander Kislitsky (akislitsky)
Changed in fuel:
assignee: Alexander Kislitsky (akislitsky) → Bulat Gaifullin (bgaifullin)

Reviewed: https://review.openstack.org/281447
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=13e7962001bc036b0f3255a61d9704546aab909e
Submitter: Jenkins
Branch: master

commit 13e7962001bc036b0f3255a61d9704546aab909e
Author: Bulat Gaifullin <email address hidden>
Date: Wed Feb 17 20:49:26 2016 +0300

    Fixed missed plugin attributes in deployment info

    After Mutable DB fields were introduced the changing of nested data in such
    fields leads to saving changes into DB.
    Now we can't use Mutable fields for transfer temporary changed data inside
    the business logic and all places with implicit using of JSON field for
    transferring temporary changed data were removed.
    This is the reason of loosing plugins data from the serialized
    deployment_info. For fix we are adding cluster plugins attributes into
    deployment_info explicitly.

    Change-Id: I0807c816c9d3af22a0d7e31f9ce96e974749c525
    Closes-Bug: #1544505

Changed in fuel:
status: In Progress → Fix Committed
Vladimir (vushakov) on 2016-03-31
tags: added: on-verification
Vladimir (vushakov) wrote :

Verified on:
    Fuel 9.0 ISO #135

tags: removed: on-verification
Illia Polliul (ipolliul) wrote :

Bug is actual for Fuel 8.0 and was reproduced multiple times.

See details in https://bugs.launchpad.net/fuel/+bug/1595266

tags: added: customer-found
Illia Polliul (ipolliul) wrote :

Here are detail how to reproduce the bug:

1) Create an environment with 2 plugins installed and enabled: EMC VNX and Zabbix
2) Deploy environment with 3 Controller nodes + 1 Compute
3) Add one more Compute
4) Press "deploy changes"

Links to plugins code:
https://github.com/openstack/fuel-plugin-external-emc/tree/3.0
https://github.com/openstack/fuel-plugin-external-zabbix/tree/2.5

I suppose that problem is triggered by mixed format of plugins tasks, but this doesn't explain why deployment passes first time.

Reviewed: https://review.openstack.org/332929
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=a5f3cc2a0b5a4d29bb709c72ad5e25416ef33f7e
Submitter: Jenkins
Branch: stable/8.0

commit a5f3cc2a0b5a4d29bb709c72ad5e25416ef33f7e
Author: Bulat Gaifullin <email address hidden>
Date: Wed Feb 17 20:49:26 2016 +0300

    Fixed missed plugin attributes in deployment info

    After Mutable DB fields were introduced the changing of nested data in such
    fields leads to saving changes into DB.
    Now we can't use Mutable fields for transfer temporary changed data inside
    the business logic and all places with implicit using of JSON field for
    transferring temporary changed data were removed.
    This is the reason of loosing plugins data from the serialized
    deployment_info. For fix we are adding cluster plugins attributes into
    deployment_info explicitly.

    Change-Id: I0807c816c9d3af22a0d7e31f9ce96e974749c525
    Closes-Bug: #1544505

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers