plugin metadata is broken

Bug #1544505 reported by Matthew Mosesohn
48
This bug affects 9 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Critical
Bulat Gaifullin
8.0.x
Fix Released
Critical
Bulat Gaifullin
Mitaka
Fix Released
Critical
Bulat Gaifullin

Bug Description

Steps to reproduce:
1 - Deploy 9.0-liberty iso 204 with detach-database plugin using all other defaults
2 - Add 1 controller, 1 database node
3 - Deploy

Expected results:
 detach-database hash in astute.yaml on nodes
Actual results:
 plugin is listed in plugins array, but no metadata is present

This was tested across multiple plugins, but I'm reporting this against a well known, established plugin.

Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

Note that this does NOT affect any 8.0 environments at this time.

Changed in fuel:
milestone: none → 9.0
assignee: nobody → Fuel Python Team (fuel-python)
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :
Ilya Kutukov (ikutukov)
tags: added: area-python
Revision history for this message
Ilya Kutukov (ikutukov) wrote :

Here is attributes_metadata field in database, it seems that validator output was recorded instead of metadata itself

{"yaml_additional_config": {"regex": {"source": ".+", "error": "This field should not be empty"}, "description": "Input must be valid YAML", "weight": 52, "type": "textarea", "valu
e": "#Uncomment to enable remote DB:\n#remote_db: '10.20.99.1'\n#remote_db_password: 'secret'\ncustom_mysql_setup_class: 'galera'\n", "label": "Additional config"}, "metadata": {"we
ight": 90, "label": "Detach Database Plugin"}}

Revision history for this message
Ilya Kutukov (ikutukov) wrote :

the previous comment is invalid this is a hints for the gui

Revision history for this message
Ilya Kutukov (ikutukov) wrote :

Not reproduced on 8.0 RC1 (529 build)

Revision history for this message
Ilya Kutukov (ikutukov) wrote :

If the problem occurred in 2 weeks timeframe as Matthew says this commits could be the reason, because they are affecting cluster attributes processing:

Merge "Changed JSON on Mutable(Dict/List) in Cluster object"
https://github.com/openstack/fuel-web/commit/5e88734cf4c3556ae1b888fc11ff0e52f228c933

Merge "Fixed constructing of a Mutable objects tree"
https://github.com/openstack/fuel-web/commit/b0a0b3cac16dc21cab33b3430080cfb8d9398d39

Revision history for this message
Ilya Kutukov (ikutukov) wrote :

DB dump of the environment

tags: added: feature-plugins
Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Ilya Kutukov (ikutukov) wrote :

Please, be aware that "Changed JSON on Mutable(Dict/List) in Cluster object" patch could break not only plug-ins metadata, but any part of logic that working with data from JSON fields.

Revision history for this message
Alexander Kislitsky (akislitsky) wrote :

@Ilya

Plugin developers are blocked by this bug now and full covering of changed logic will take a time, thus I propose to land this fix for critical bug and implement checking for Mutable DB fields as separate bug: https://bugs.launchpad.net/fuel/+bug/1546063

Revision history for this message
Ilya Kutukov (ikutukov) wrote :

Is mutable JSON patch roll-back an option?
I don't think there is any new logic that work only with mutable fields.

Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

Folks, I think that we need to revert the patch that caused the problem as this is a critical regression. After that we can fix all the stuff that needs to fixed, test things properly and land the proper code.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/281447

Changed in fuel:
assignee: Alexander Kislitsky (akislitsky) → Bulat Gaifullin (bgaifullin)
Changed in fuel:
assignee: Bulat Gaifullin (bgaifullin) → Alexander Kislitsky (akislitsky)
Changed in fuel:
assignee: Alexander Kislitsky (akislitsky) → Bulat Gaifullin (bgaifullin)
Changed in fuel:
assignee: Bulat Gaifullin (bgaifullin) → Alexander Kislitsky (akislitsky)
Changed in fuel:
assignee: Alexander Kislitsky (akislitsky) → Bulat Gaifullin (bgaifullin)
Changed in fuel:
assignee: Bulat Gaifullin (bgaifullin) → Alexander Kislitsky (akislitsky)
Changed in fuel:
assignee: Alexander Kislitsky (akislitsky) → Bulat Gaifullin (bgaifullin)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/281447
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=13e7962001bc036b0f3255a61d9704546aab909e
Submitter: Jenkins
Branch: master

commit 13e7962001bc036b0f3255a61d9704546aab909e
Author: Bulat Gaifullin <email address hidden>
Date: Wed Feb 17 20:49:26 2016 +0300

    Fixed missed plugin attributes in deployment info

    After Mutable DB fields were introduced the changing of nested data in such
    fields leads to saving changes into DB.
    Now we can't use Mutable fields for transfer temporary changed data inside
    the business logic and all places with implicit using of JSON field for
    transferring temporary changed data were removed.
    This is the reason of loosing plugins data from the serialized
    deployment_info. For fix we are adding cluster plugins attributes into
    deployment_info explicitly.

    Change-Id: I0807c816c9d3af22a0d7e31f9ce96e974749c525
    Closes-Bug: #1544505

Changed in fuel:
status: In Progress → Fix Committed
Vladimir (vushakov)
tags: added: on-verification
Revision history for this message
Vladimir (vushakov) wrote :

Verified on:
    Fuel 9.0 ISO #135

tags: removed: on-verification
Revision history for this message
Illia Polliul (ipolliul) wrote :

Bug is actual for Fuel 8.0 and was reproduced multiple times.

See details in https://bugs.launchpad.net/fuel/+bug/1595266

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/8.0)

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/332929

tags: added: customer-found
Revision history for this message
Illia Polliul (ipolliul) wrote :

Here are detail how to reproduce the bug:

1) Create an environment with 2 plugins installed and enabled: EMC VNX and Zabbix
2) Deploy environment with 3 Controller nodes + 1 Compute
3) Add one more Compute
4) Press "deploy changes"

Links to plugins code:
https://github.com/openstack/fuel-plugin-external-emc/tree/3.0
https://github.com/openstack/fuel-plugin-external-zabbix/tree/2.5

I suppose that problem is triggered by mixed format of plugins tasks, but this doesn't explain why deployment passes first time.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (stable/8.0)

Reviewed: https://review.openstack.org/332929
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=a5f3cc2a0b5a4d29bb709c72ad5e25416ef33f7e
Submitter: Jenkins
Branch: stable/8.0

commit a5f3cc2a0b5a4d29bb709c72ad5e25416ef33f7e
Author: Bulat Gaifullin <email address hidden>
Date: Wed Feb 17 20:49:26 2016 +0300

    Fixed missed plugin attributes in deployment info

    After Mutable DB fields were introduced the changing of nested data in such
    fields leads to saving changes into DB.
    Now we can't use Mutable fields for transfer temporary changed data inside
    the business logic and all places with implicit using of JSON field for
    transferring temporary changed data were removed.
    This is the reason of loosing plugins data from the serialized
    deployment_info. For fix we are adding cluster plugins attributes into
    deployment_info explicitly.

    Change-Id: I0807c816c9d3af22a0d7e31f9ce96e974749c525
    Closes-Bug: #1544505

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers