oslo.db needs a method for selecting mysql storage engine

[Octave Orgeron]

oslo.db assumes that a MySQL database can only have a storage engine of InnoDB. This causes complications for OpenStack to support other MySQL storage engines, such as MySQL Cluster (NDB). Oslo.db should have a configuration string (i.e. mysql_storage_engine) in the oslo_db database group that can be used by SQLalchemy, Alembic, and OpenStack to implement the desired support and behavior for alternative MySQL storage engines.

[Octave Orgeron (octave-orgeron)]

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Cinder requires changes to SQLalchemy DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

[Octave Orgeron (octave-orgeron)]

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Glance requires changes to SQLalchemy DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

[Octave Orgeron (octave-orgeron)]

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Ironic requires changes to Alembic DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

[Octave Orgeron (octave-orgeron)]

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Nova requires changes to SQLalchemy DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

[Octave Orgeron (octave-orgeron)]

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Keystone requires changes to SQLalchemy DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

[Octave Orgeron (octave-orgeron)]

MySQL Cluster (NDB) provides an active/active HA and scale-out DB that can be used for OpenStack. Neutron requires changes to Alembic DB upgrade and migrations to support MySQL Cluster (NDB). The primary enhancements are:

1. Implementation of an oslo.db configuration parameter to specify the MySQL storage engine (mysql_storage_engine).
2. Replacement of hardcoded SQL statements that set the engine to "InnoDB" to the above configuration value.
3. Logic to handle SQL differences between MySQL InnoDB and MySQL Cluster (NDB). This includes column lengths, constraints, foreign keys, and indexes.

I have a change-set patch for this. Awaiting CLA before submitting.

[Steve Martinelli (stevemar)]

Not really sure how this affects keystone, we use oslo.db and any changes made to oslo.db will benefit keystone.

[Octave Orgeron (octave-orgeron)]

Hi Steve,

By adding the patch for oslo.db, the additional patches for OpenStack can be applied to each service. For example, when the mysql_storage_engine is set to NDBCLUSTER, it will trigger the appropriate logic in each service for their respective db_sync operation for creating the initial database and/or doing upgrades. The oslo.db patch lays the foundation for the rest of OpenStack to take advantage of this setting.

Octave

[Dmitry Tantsur (divius)]

Please don't create such catch-all bugs covering many projects. Launchpad is not very good in handling them.

Also as it's stated it looks like an RFE anyway, and may even require a spec/blueprint - depending on how each project track it. Maybe even a cross-project spec.

[Matt Riedemann (mriedem)]

Yeah, this should start as a cross-project spec, see:

https://github.com/openstack/openstack-specs

http://specs.openstack.org/openstack/openstack-specs/

[Akihiro Motoki (amotoki)]

oslo.db is apparently required, but it is unclear what kind change is required in each project.
It should start in oslo.db or as a cross project spec.

[Akihiro Motoki (amotoki)]

I read #8 but i have not understood yet why each project needs a logic to support MySQL cluster. It should be discussed in a cross-project spec or oslo.db blueprint.

[Ihar Hrachyshka (ihar-hrachyshka)]

@Satya, please provide rationale why neutron should care about it; I believe no code changes are needed in neutron.

[Octave Orgeron (octave-orgeron)]

Hi,

There are changes required for Neutron and all of the other services to
support NDB. I've created the patches for Mitaka. I will be creating a
blueprint/spec for the support and work with each team to get the
patches pushed upstream.

Thanks,
Octave

On 10/20/2016 2:32 PM, Ihar Hrachyshka wrote:
> @Satya, please provide rationale why neutron should care about it; I
> believe no code changes are needed in neutron.
>
> ** Changed in: neutron
> Status: Confirmed => Incomplete
>

--

Oracle <http://www.oracle.com/>
Octave J. Orgeron | Sr. Principal Software Engineer
Oracle Solaris OpenStack
Mobile: [phone] <tel:[phone]>
500 Eldorado Blvd. | Broomfield, CO 80021
Certified Oracle Enterprise Architect: Systems Infrastructure
<http://www.oracle.com/us/solutions/enterprise-architecture/index.html>
Green Oracle <http://www.oracle.com/commitment> Oracle is committed to
developing practices and products that help protect the environment

[gordon chung (chungg)]

no one worked on ceilometer legacy storage for over a year. use gnocchi or something else

[Octave Orgeron (octave-orgeron)]

Working on the patch set now for upstream.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.db 4.24.0 release.