Server fails to start after upgrade because of customized config and obsolete/renamed directives

[Lars Tangvald]

Separate from LP: #1571865 because this concerns options we don't try to automatically fix.

When users upgrade from 5.5 (and especially if that was also an upgrade from earlier versions) and have custom configs, 5.7 may refuse to start because the config contains options that have been removed. The same can happen with any other upgrade, like from 5.7 to 8.0.

By "custom config", we actually mean two possibilities:
a) non-default config file (like /etc/mysql/my.cnf)
b) non-default config option (like NO_AUTO_CREATE_USER, which was dropped in 8, but was never part of the default config files shipped with the package)

Options may have been removed due to becoming obsolete, or due to being renamed. Postinst will print the offending option to the terminal, but this might not be apparent in the text dump from a large upgrade operation.

http://dev.mysql.com/doc/refman/5.7/en/mysqld-option-tables.html has a list of options that will be accepted for 5.7

The advantage of the current solution (print option name and throw error) is that after fixing the config, apt-get -f install should complete the upgrade fully.
The disadvantage is that it will abort the upgrade process, which might be a full distro upgrade.

The alternative would be to print a warning, which might not be noticed, and complete the install without running mysql_upgrade or starting the service.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Tim Riker (timriker)]

2017-05-11T04:47:57.464320Z 0 [ERROR] unknown variable 'thread_concurrency=8'

[Lars Tangvald (lars-tangvald)]

thread_concurrency was removed in 5.7. It only had an effect on older Solaris systems, so simply removing it from the config should not cause any change.

https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_thread_concurrency

[Brian Morris (brian-morris-h)]

This failure also occurs because "log_slow_queries" has become "slow_query_log", but the installer does not handle this change automagically. Like the OP's error, nothing specific to it is mentioned in the installation failure message.

[Robie Basak (racb)]

We're getting some additional duplicates of this bug against MySQL 8.0. It might be an idea to look through the recent dupes and try to identify a list of common failures that we could automatically attempt to fix in postinst.

[Cygnus (cygnusnsf)]

Thanks will do have a look cheers Robbie

On Tue, 19 Nov 2019, 11:46 Robie Basak, <email address hidden> wrote:

> We're getting some additional duplicates of this bug against MySQL 8.0.
> It might be an idea to look through the recent dupes and try to identify
> a list of common failures that we could automatically attempt to fix in
> postinst.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1852758).
> https://bugs.launchpad.net/bugs/1612517
>
> Title:
> Server fails to start after upgrade because of customized config and
> obsolete/renamed directives
>
> Status in mysql-5.7 package in Ubuntu:
> Confirmed
> Status in mysql-8.0 package in Ubuntu:
> Confirmed
>
> Bug description:
> Separate from LP: #1571865 because this concerns options we don't try
> to automatically fix.
>
> When users upgrade from 5.5 (and especially if that was also an
> upgrade from earlier versions) and have custom configs, 5.7 may refuse
> to start because the config contains options that have been removed.
> The same can happen with any other upgrade, like from 5.7 to 8.0.
>
> By "custom config", we actually mean two possibilities:
> a) non-default config file (like /etc/mysql/my.cnf)
> b) non-default config option (like NO_AUTO_CREATE_USER, which was
> dropped in 8, but was never part of the default config files shipped with
> the package)
>
> Options may have been removed due to becoming obsolete, or due to
> being renamed. Postinst will print the offending option to the
> terminal, but this might not be apparent in the text dump from a large
> upgrade operation.
>
> http://dev.mysql.com/doc/refman/5.7/en/mysqld-option-tables.html has a
> list of options that will be accepted for 5.7
>
> The advantage of the current solution (print option name and throw
> error) is that after fixing the config, apt-get -f install should complete
> the upgrade fully.
> The disadvantage is that it will abort the upgrade process, which might
> be a full distro upgrade.
>
> The alternative would be to print a warning, which might not be
> noticed, and complete the install without running mysql_upgrade or
> starting the service.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/mysql-5.7/+bug/1612517/+subscriptions
>

[Cygnus (cygnusnsf)]

Robie even

On Tue, 19 Nov 2019, 17:34 Neale Francis, <email address hidden> wrote:

> Thanks will do have a look cheers Robbie
>
> On Tue, 19 Nov 2019, 11:46 Robie Basak, <email address hidden>
> wrote:
>
>> We're getting some additional duplicates of this bug against MySQL 8.0.
>> It might be an idea to look through the recent dupes and try to identify
>> a list of common failures that we could automatically attempt to fix in
>> postinst.
>>
>> --
>> You received this bug notification because you are subscribed to a
>> duplicate bug report (1852758).
>> https://bugs.launchpad.net/bugs/1612517
>>
>> Title:
>> Server fails to start after upgrade because of customized config and
>> obsolete/renamed directives
>>
>> Status in mysql-5.7 package in Ubuntu:
>> Confirmed
>> Status in mysql-8.0 package in Ubuntu:
>> Confirmed
>>
>> Bug description:
>> Separate from LP: #1571865 because this concerns options we don't try
>> to automatically fix.
>>
>> When users upgrade from 5.5 (and especially if that was also an
>> upgrade from earlier versions) and have custom configs, 5.7 may refuse
>> to start because the config contains options that have been removed.
>> The same can happen with any other upgrade, like from 5.7 to 8.0.
>>
>> By "custom config", we actually mean two possibilities:
>> a) non-default config file (like /etc/mysql/my.cnf)
>> b) non-default config option (like NO_AUTO_CREATE_USER, which was
>> dropped in 8, but was never part of the default config files shipped with
>> the package)
>>
>> Options may have been removed due to becoming obsolete, or due to
>> being renamed. Postinst will print the offending option to the
>> terminal, but this might not be apparent in the text dump from a large
>> upgrade operation.
>>
>> http://dev.mysql.com/doc/refman/5.7/en/mysqld-option-tables.html has a
>> list of options that will be accepted for 5.7
>>
>> The advantage of the current solution (print option name and throw
>> error) is that after fixing the config, apt-get -f install should complete
>> the upgrade fully.
>> The disadvantage is that it will abort the upgrade process, which might
>> be a full distro upgrade.
>>
>> The alternative would be to print a warning, which might not be
>> noticed, and complete the install without running mysql_upgrade or
>> starting the service.
>>
>> To manage notifications about this bug go to:
>>
>> https://bugs.launchpad.net/ubuntu/+source/mysql-5.7/+bug/1612517/+subscriptions
>>
>

[Eduardo Bonato (ebonato)]

As this bug still unresolved (surprised to see its from 2016), I have to post another night night headache bug #1899075.
I should agree with this bug author.
Release upgrade script should not block due any package upgrade error. Any fail in a package upgrade should be just an warning, to be handled after full distro upgrade.
Or you can at least leave grub correctly installed even with packages upgrade errors, to avoid servers stucks at boot screen. Most servers today are remote and virtualized, and most people will not know how to handle with this situation. Luckly MS Azure has an vm repair extension to be used in azure console, but its still hidden for people which only uses Web UI.

[Christian Ehrhardt  (paelzer)]

FYI - a new "category" for this bug is recently in a few bug reports.

If sql_mode was set to NO_AUTO_CREATE_USER this won't work anymore.
Thereby triggering the usual "failed to upgrade, can't restart" condition.

In that case please check upstreams documentation on sql_mode, so far all users were ok just removing that statement and things worked again.