oslo.db needs a method for selecting mysql storage engine

Bug #1564110 reported by Octave Orgeron on 2016-03-30
24
This bug affects 3 people
Affects Status Importance Assigned to Milestone
oslo.db
Wishlist
Octave Orgeron

Bug Description

oslo.db assumes that a MySQL database can only have a storage engine of InnoDB. This causes complications for OpenStack to support other MySQL storage engines, such as MySQL Cluster (NDB). Oslo.db should have a configuration string (i.e. mysql_storage_engine) in the oslo_db database group that can be used by SQLalchemy, Alembic, and OpenStack to implement the desired support and behavior for alternative MySQL storage engines.

Tags: rfe Edit Tag help
no longer affects: ceilometer
summary: - oslo.db should support MySQL Cluster (NDB)
+ OpenStack should support MySQL Cluster (NDB)

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Cinder requires changes to SQLalchemy DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

Changed in cinder:
assignee: nobody → Octave Orgeron (octave-orgeron)
Octave Orgeron (octave-orgeron) wrote :

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Glance requires changes to SQLalchemy DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

Changed in glance:
assignee: nobody → Octave Orgeron (octave-orgeron)
Octave Orgeron (octave-orgeron) wrote :

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Ironic requires changes to Alembic DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

Changed in ironic:
assignee: nobody → Octave Orgeron (octave-orgeron)
Octave Orgeron (octave-orgeron) wrote :

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Nova requires changes to SQLalchemy DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

Changed in nova:
assignee: nobody → Octave Orgeron (octave-orgeron)
Octave Orgeron (octave-orgeron) wrote :

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Keystone requires changes to SQLalchemy DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

Changed in keystone:
assignee: nobody → Octave Orgeron (octave-orgeron)
Octave Orgeron (octave-orgeron) wrote :

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Neutron requires changes to Alembic DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

Changed in neutron:
assignee: nobody → Octave Orgeron (octave-orgeron)
Changed in oslo.db:
assignee: nobody → Octave Orgeron (octave-orgeron)
Changed in ceilometer:
assignee: nobody → Octave Orgeron (octave-orgeron)
Steve Martinelli (stevemar) wrote :

Not really sure how this affects keystone, we use oslo.db and any changes made to oslo.db will benefit keystone.

Changed in keystone:
status: New → Incomplete
Octave Orgeron (octave-orgeron) wrote :

Hi Steve,

By adding the patch for oslo.db, the additional patches for OpenStack can be applied to each service. For example, when the mysql_storage_engine is set to NDBCLUSTER, it will trigger the appropriate logic in each service for their respective db_sync operation for creating the initial database and/or doing upgrades. The oslo.db patch lays the foundation for the rest of OpenStack to take advantage of this setting.

Octave

Dmitry Tantsur (divius) wrote :

Please don't create such catch-all bugs covering many projects. Launchpad is not very good in handling them.

Also as it's stated it looks like an RFE anyway, and may even require a spec/blueprint - depending on how each project track it. Maybe even a cross-project spec.

Changed in ironic:
status: New → Confirmed
importance: Undecided → Wishlist
Matt Riedemann (mriedem) wrote :
Changed in nova:
status: New → Opinion
importance: Undecided → Wishlist
Changed in cinder:
status: New → Opinion
importance: Undecided → Wishlist
Changed in keystone:
assignee: Octave Orgeron (octave-orgeron) → nobody
Akihiro Motoki (amotoki) wrote :

oslo.db is apparently required, but it is unclear what kind change is required in each project.
It should start in oslo.db or as a cross project spec.

Changed in neutron:
status: New → Incomplete
status: Incomplete → Opinion
Akihiro Motoki (amotoki) wrote :

I read #8 but i have not understood yet why each project needs a logic to support MySQL cluster. It should be discussed in a cross-project spec or oslo.db blueprint.

Changed in neutron:
status: Opinion → Incomplete
assignee: Octave Orgeron (octave-orgeron) → nobody
ZhiQiang Fan (aji-zqfan) on 2016-04-05
Changed in ceilometer:
status: New → Incomplete
Changed in nova:
status: Opinion → Confirmed
Sam Betts (sambetts) on 2016-09-14
tags: added: rfe
Changed in neutron:
status: Incomplete → Confirmed

@Satya, please provide rationale why neutron should care about it; I believe no code changes are needed in neutron.

Changed in neutron:
status: Confirmed → Incomplete

Hi,

There are changes required for Neutron and all of the other services to
support NDB. I've created the patches for Mitaka. I will be creating a
blueprint/spec for the support and work with each team to get the
patches pushed upstream.

Thanks,
Octave

On 10/20/2016 2:32 PM, Ihar Hrachyshka wrote:
> @Satya, please provide rationale why neutron should care about it; I
> believe no code changes are needed in neutron.
>
> ** Changed in: neutron
> Status: Confirmed => Incomplete
>

--

Oracle <http://www.oracle.com/>
Octave J. Orgeron | Sr. Principal Software Engineer
Oracle Solaris OpenStack
Mobile: +1-720-616-1550 <tel:+17206161550>
500 Eldorado Blvd. | Broomfield, CO 80021
Certified Oracle Enterprise Architect: Systems Infrastructure
<http://www.oracle.com/us/solutions/enterprise-architecture/index.html>
Green Oracle <http://www.oracle.com/commitment> Oracle is committed to
developing practices and products that help protect the environment

no one worked on ceilometer legacy storage for over a year. use gnocchi or something else

Changed in ceilometer:
status: Incomplete → Won't Fix
Changed in keystone:
status: Incomplete → Opinion
importance: Undecided → Wishlist

Working on the patch set now for upstream.

Changed in oslo.db:
status: New → In Progress
Changed in oslo.db:
importance: Undecided → Wishlist
Changed in keystone:
assignee: nobody → Octave Orgeron (octave-orgeron)
status: Opinion → In Progress
Changed in heat:
assignee: nobody → Octave Orgeron (octave-orgeron)
Changed in neutron:
assignee: nobody → Octave Orgeron (octave-orgeron)
no longer affects: ceilometer
no longer affects: cinder
no longer affects: glance
no longer affects: ironic
no longer affects: nova
no longer affects: keystone
no longer affects: heat
no longer affects: neutron
summary: - OpenStack should support MySQL Cluster (NDB)
+ oslo.db needs a method for selecting mysql storage engine
description: updated

This issue was fixed in the openstack/oslo.db 4.24.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers