MariaDB's table storage format change is causing errors after a FFU

Bug #1908232 reported by Damien Ciabrini
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
In Progress
High
Damien Ciabrini

Bug Description

In Queens - which uses MariaDB 10.1 - the internal storage format used
for InnoDB tables is the 'COMPACT' format [1]. Whereas in Train -
which uses Mariadb 10.3 - the default storage format has changed from
'COMPACT' to 'DYNAMIC'. This means the Openstack tables internally use
different storage format whether they are created in Queens or in
Train.

During an FFU from Queens to Train, the database's internal tables
are upgraded with mysql_upgrade, but that tool doesn't ALTER the
Openstack tables to use the new default internal storage format
as it is not needed when upgrading the database.

Mariadb 10.3 has more DDL checks that Mariadb 10.1, such as for example
stricter table create/alter checks and table size checks. And recent
Mariadb (e.g. 10.3.27) has changed the severity of some of those checks
when InnoDB strict mode is enforced (the default). Consequently,
some checks are failing with a few OpenStack tables after the FFU
has upgraded the database without changing the row format in use, which
at least breaks some db_sync containers:

    [root@controller-0 stdouts]# more nova_db_sync.log
    2020-12-14T11:54:54.186418426+00:00 stdout F ERROR: Could not access cell0.
    2020-12-14T11:54:54.186418426+00:00 stdout F Has the nova_api database been created?
    2020-12-14T11:54:54.186418426+00:00 stdout F Has the nova_cell0 database been created?
    2020-12-14T11:54:54.186418426+00:00 stdout F Has "nova-manage api_db sync" been run?
    2020-12-14T11:54:54.186418426+00:00 stdout F Has "nova-manage cell_v2 map_cell0" been run?
    2020-12-14T11:54:54.186418426+00:00 stdout F Is [api_database]/connection set in nova.conf?
    2020-12-14T11:54:54.186418426+00:00 stdout F Is the cell0 database connection URL correct?
    2020-12-14T11:54:54.186418426+00:00 stdout F Error: (pymysql.err.InternalError) (1118, 'Row size too large. The maximum row size for the used table type, not counting BLOBs, is 8126. This includes storage overhead, check the manual.
     You have to change some columns to TEXT or BLOBs')
    2020-12-14T11:54:54.186418426+00:00 stdout F [SQL:
    2020-12-14T11:54:54.186418426+00:00 stdout F ALTER TABLE instances ADD hidden BOOL]
    2020-12-14T11:54:54.186418426+00:00 stdout F (Background on this error at: http://sqlalche.me/e/2j85)

[1] https://mariadb.com/kb/en/innodb-row-formats-overview/

Revision history for this message
Michele Baldessari (michele) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 13.5.0

This issue was fixed in the openstack/puppet-tripleo 13.5.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 14.0.0

This issue was fixed in the openstack/puppet-tripleo 14.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 12.5.0

This issue was fixed in the openstack/puppet-tripleo 12.5.0 release.

Changed in tripleo:
milestone: wallaby-2 → wallaby-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 11.5.0

This issue was fixed in the openstack/puppet-tripleo 11.5.0 release.

Changed in tripleo:
milestone: wallaby-3 → wallaby-rc1
Changed in tripleo:
milestone: wallaby-rc1 → xena-1
Changed in tripleo:
milestone: xena-1 → xena-2
Changed in tripleo:
milestone: xena-2 → xena-3
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.