[SRU] cqrlog needs to depend on mariadb instead of mysql

[Manoj Iyer]

[ Impact ]

cqrlog is unusable for all users who will want to save their data locally. It will fail to connect with the local database server when any user tries to use it.

Whenever any user tries to run the application for the first time, the user will be asked in a dialogue box:
It seems you are trying to run this program for the first time, are you going to save data to local machine?
If you say Yes, new databases will be created. This may take a while, please be patient.

The user has to click "Yes" to create local database to save data locally.

And, then it will fail and a new dialogue box will display:
MySQL could not be started. Please check if the MySQL server is installed properly.

Select close.

Another dialogue box will popup with the error:
Error during connection to database: TMySQL57Connection : Server connect failed.

[ Test Plan ]

install cqrlog

run cqrlog from a terminal or from the launcher
Select "Yes" in the dialogue box saying:
It seems you are trying to run this program for the first time, are you going to save data to local machine?
If you say Yes, new databases will be created. This may take a while, please be patient.

if the package is fixed, it will display the window of "Database connection" with Log001 on the first row.

Click on "Open log" -> that will display a window of "Changelog"
click "Close" -> that will display the main window of cqrlog.

Additional question will popup about new QSL managers database and new DXCC tables. Select "No" for both these question.

Just to confirm cqrlog is working, enter some dummy data.

Enter "srutest" in the box labelled "Call", move to the box labelled Name, and enter "name1".
Click on "Save QSO".

Close cqrlog window.

Start cqrlog again. This time it will directly go to the window of "Database connection" with Log001 on the first row.
Click on "Open log". This time it will directly go to the "cqrlog" windows and will ask the additional questions about new QSL managers database and new DXCC tables.
Select No for both.

Enter "srutest" in the box labelled "Call", move to the box labelled Name. The name will be autofilled now (taken from the previous record). And on the top part of the window we should be able to see the previous entry with timestamp.

[ Where problems could occur ]

There is no change in the code. The only change is in the runtime dependency to install mariadb instead of mysql and only impacts users who uses local database. There is very miinimum chance of any regression due to this change but as a worst case scenario if the change causes some regression then the users who are using local database will not see any new issue but will continue to see the same problem as is now.

[ Other Info ]

1. It has been fixed in the Debian package 2.5.2-5 which has been synced to Oracular.
2. As mentioned in the original Bug Description, the issue was caused by a mismatch of mysql and mariadb. cqrlog needs mariadb to work.

[ Original Bug Description ]

[Impact]
Due to the difference in how Debian and Ubuntu ship the mysql-* packages, cqrlog has been using mysql-* packages in Ubuntu, which the package is not compatible with. This means users can successfully install, but the package fails on launch.

In Debian, we ship mariadb-* to fulfil mysql-* requirements, but this is not true with ubuntu [https://salsa.debian.org/mariadb-team/mysql/-/blob/mysql-defaults/debian/master/debian/rules](Debian Salsa )

[Test Plan]
This fix has already been tested and implemented in Debian upstream. However was not available in Debian until post-Noble release.

This has already been tested manually by others, but as soon as a version with the patch present is available, a user should install the updated cqrlog package and the appropriate dependencies are installed so the program should work as usual.

[Regression Potential]
Extremely limited. Users on systems dating back to pre-jammy have not been able to use this package without manually installing mariadb, so no database migration is required unless they are upgrading from before hirsute, and they would need to migrate regardless of this bug.

[Where problems could occur]

Uncertain

[Other Info]
This is already available in Debian Unstable, and also in Oracular due to the sync.

I am the DD responsible for cqrlog in Debian but the SRU process is unfamiliar to me, so some handholding and sponsorship is required!

[Original Description]

Please update the cqrlog package (debian/control) to depend on mariadb server instead of mysql server.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Luc Langehegermann (lucl)]

This is still an issue in recent ubuntu Versions. CQRLOG does not work with recent Versions of mysql, only with mariadb.

The dependency in the package is default-mysql-core, which points to mysql on Ubuntu. Changing that to mariadb would likely fix the issue.

On the CQRLOG Forums, there are multiple such issues reported, one recent post: https://www.cqrlog.com/node/3476

Luc

[Koos van den Hout (koos-kzdoos)]

Still happening with Ubuntu 22.04. I'll attach the output of cqrlog --debug=1 and the mysql.err file.

[Koos van den Hout (koos-kzdoos)]

console messages with cqrlog --debug=1

[Koos van den Hout (koos-kzdoos)]

mysql.err file

[AsciiWolf (asciiwolf)]

CQRLOG (2.5.2-3ubuntu2) still does not seem to run on Ubuntu 24.04 because it cannot connect to the MySQL database. See the attached screenshot. I cannot believe that this was still not fixed after 4 years!

This is in the mysql.err log file:
2024-06-18T10:30:05.440022Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.37-0ubuntu0.24.04.1) starting as process 15289
2024-06-18T10:30:05.441321Z 0 [Warning] [MY-013242] [Server] --character-set-server: 'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.
2024-06-18T10:30:05.441336Z 0 [Warning] [MY-013244] [Server] --collation-server: 'utf8mb3_bin' is a collation of the deprecated character set UTF8MB3. Please consider using UTF8MB4 with an appropriate collation instead.
2024-06-18T10:30:05.459006Z 0 [Warning] [MY-010075] [Server] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: ba6b7bc4-2d5d-11ef-963e-525400dd5534.
2024-06-18T10:30:05.465986Z 1 [ERROR] [MY-011011] [Server] Failed to find valid data directory.
2024-06-18T10:30:05.466138Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2024-06-18T10:30:05.466158Z 0 [ERROR] [MY-010119] [Server] Aborting
2024-06-18T10:30:05.466733Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.37-0ubuntu0.24.04.1) (Ubuntu).
2024-06-18T10:51:44.918174Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.37-0ubuntu0.24.04.1) starting as process 3066
2024-06-18T10:51:44.919415Z 0 [Warning] [MY-013242] [Server] --character-set-server: 'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.
2024-06-18T10:51:44.919426Z 0 [Warning] [MY-013244] [Server] --collation-server: 'utf8mb3_bin' is a collation of the deprecated character set UTF8MB3. Please consider using UTF8MB4 with an appropriate collation instead.
2024-06-18T10:51:44.938263Z 1 [ERROR] [MY-011011] [Server] Failed to find valid data directory.
2024-06-18T10:51:44.938440Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2024-06-18T10:51:44.938463Z 0 [ERROR] [MY-010119] [Server] Aborting
2024-06-18T10:51:44.938981Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.37-0ubuntu0.24.04.1) (Ubuntu).

The issue was resolved and CQRLOG started working correctly after running "sudo apt install mariadb-server" (replacing the mysql server with mariadb).

I don't know how the mysql/mariadb package is handled in Debian, but someone should probably send a Merge request to the Debian GitLab repository: https://salsa.debian.org/debian-hamradio-team/cqrlog

[AsciiWolf (asciiwolf)]

This should ideally be fixed in Debian and then in 24.04 / 22.04 as a Stable Release Update.

https://wiki.ubuntu.com/StableReleaseUpdates

[AsciiWolf (asciiwolf)]

So, as Dave Hibberd found out, the problem is that Ubuntu is shipping mysql-* as mysql-* and Debian is shipping mariadb-* as mysql-*.

See his original comment with more details here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1073790#10

One possible Debian-side solution could be changing the virtual-mysql-client-core and virtual-mysql-server-core to mariadb-client-core and mariadb-server-core, default-mysql-server to mariadb-server and removing the default-mysql-server-core dependency. That would cause dependency issues on systems that already have mysql-server-8 installed, but since CQRLOG does not seem to work with mysql anymore (only with mariadb), it should not be a big problem, at least in my opinion.

The other possible solution would be to patch this in Ubuntu. However, I am not sure whether the CQRLOG package has any Ubuntu maintainers and also it would probably break auto-sync of the package from Debian.

CQRLOG should also fix their MySQL 8 support if possible. That would probably be the best solution.

It looks like that there sadly are not other possible fixes to this issue.

[Dave Hibberd (hibby)]

Hi all,

It turns out I agreed after some more thinking on the subject, so have committed a fix for this as 2.5.2-5 in Debian: https://tracker.debian.org/news/1538653/accepted-cqrlog-252-5-source-into-unstable/

Code's on salsa if you want to try a build and install - I ran it through my build infrastructure for jammy and noble and it builds successfully, but I haven't tested installing and running as I don't have a VM to hand.

Cheers!

Hibby

[AsciiWolf (asciiwolf)]

Hi Dave,

Thanks a lot! The new build is already synced in Oracular (24.10):

https://launchpad.net/ubuntu/+source/cqrlog/2.5.2-5

I have tested it on a clean amd64 24.04 VM and can confirm that it works fine now! :-)

So, if anyone wants this to also be fixed in Noble (24.04 LTS), it should now be possible to make a SRU:

https://wiki.ubuntu.com/StableReleaseUpdates

[Sudip Mukherjee (sudipmuk)]

Uploaded for Noble. Jammy and Focal. Waiting in SRU queue now for SRU review.

[Robie Basak (racb)]

The upstream website at https://www.cqrlog.com/ says:

> CQRLOG is an advanced ham radio logger based on MySQL database.

In Ubuntu, we default to MySQL. I think switching to MariaDB by default requires exceptional justification, such as the upstream and codebase very clearly not being able to function with MySQL for a specific, stated reason.

If that's the case, please could you provide it?

Otherwise, "cqrlog needs to depend on mariadb instead of mysql" seems like an unjustified assumption.

[Dave Hibberd (hibby)]

The application will not start with MySQL. The referred statement is from a post dated Jan 2009 just before mariadb first appeared. It is my opinion the shortdesc has not been updated either there or on the project's Github repo.

The github repo lists a dependency on mariadb specifically [1] as does his control file for self-generated debs [2].
Testing reveals it builds with mysql but does not initialise correctly.

Do affected users need to supply logs of the application failing to start on firstrun or similar?

[1] https://github.com/ok2cqr/cqrlog?tab=readme-ov-file#dependencies
[2] https://github.com/ok2cqr/cqrlog/blob/master/debian/control

--
  Hibby
  Debian Developer
  Packet Radioist
  MM0RFN

On Thu, 29 Aug 2024, at 4:45 PM, Robie Basak wrote:
> The upstream website at https://www.cqrlog.com/ says:
>
>> CQRLOG is an advanced ham radio logger based on MySQL database.
>
> In Ubuntu, we default to MySQL. I think switching to MariaDB by default
> requires exceptional justification, such as the upstream and codebase
> very clearly not being able to function with MySQL for a specific,
> stated reason.
>
> If that's the case, please could you provide it?
>
> Otherwise, "cqrlog needs to depend on mariadb instead of mysql" seems
> like an unjustified assumption.
>
> --
> You received this bug notification because you are a member of Ubuntu
> ham developers, which is subscribed to the bug report.
> https://bugs.launchpad.net/bugs/1872002
>
> Title:
> [SRU] cqrlog needs to depend on mariadb instead of mysql
>
> Status in cqrlog package in Ubuntu:
> Fix Released
> Status in cqrlog source package in Focal:
> In Progress
> Status in cqrlog source package in Jammy:
> In Progress
> Status in cqrlog source package in Noble:
> In Progress
>
> Bug description:
> [ Impact ]
>
> cqrlog is unusable for all users who will want to save their data
> locally. It will fail to connect with the local database server when
> any user tries to use it.
>
> Whenever any user tries to run the application for the first time,
> the user will be asked in a dialogue box:
> It seems you are trying to run this program for the first time, are
> you going to save data to local machine?
> If you say Yes, new databases will be created. This may take a while,
> please be patient.
>
> The user has to click "Yes" to create local database to save data
> locally.
>
> And, then it will fail and a new dialogue box will display:
> MySQL could not be started. Please check if the MySQL server is
> installed properly.
>
> Select close.
>
> Another dialogue box will popup with the error:
> Error during connection to database: TMySQL57Connection : Server
> connect failed.
>
>
> [ Test Plan ]
>
> install cqrlog
>
> run cqrlog from a terminal or from the launcher
> Select "Yes" in the dialogue box saying:
> It seems you are trying to run this program for the first time, are
> you going to save data to local machine?
> If you say Yes, new databases will be created. This may take a while,
> please be patient.
>
> if the package is fixed, it will display the window of "Database
> connection" with Log001 on the first row.
>
> C...

[Robie Basak (racb)]

On Thu, Aug 29, 2024 at 04:08:04PM -0000, Dave Hibberd wrote:
> The application will not start with MySQL.

If it works with MariaDB but not MySQL, that doesn't necessarily mean
that it only supports MariaDB, or that there's not just a bug that needs
fixing to keep it compatible with both.

> Do affected users need to supply logs of the application failing to
> start on firstrun or similar?

No. That's a given. What I'd like to see is an explanation of why the
package cannot be fixed to work with MySQL properly.

[Dave Hibberd (hibby)]

HI there!

> On 29 Aug 2024, at 17:31, Robie Basak <email address hidden> wrote:
>
> On Thu, Aug 29, 2024 at 04:08:04PM -0000, Dave Hibberd wrote:
>> The application will not start with MySQL.
>
> If it works with MariaDB but not MySQL, that doesn't necessarily mean
> that it only supports MariaDB, or that there's not just a bug that needs
> fixing to keep it compatible with both.

The best I have is upstream saying it depends on mariadb not mysql[1] since version 2.5.

I really don’t have time or knowledge to dig in to why this is the case, until this came around it was my understanding that they were compatible and I’ve followed upstream’s precedent in the Debian package.

>
>> Do affected users need to supply logs of the application failing to
>> start on firstrun or similar?
>
> No. That's a given. What I'd like to see is an explanation of why the
> package cannot be fixed to work with MySQL properly.
>

The problem that falls out of all this is that based on this bug when it got sent upstream to debian and that justification, I’ve moved the dependency to mariadb[2], not the virtual package.
This has already been mirrored in the next release - granted, I’ve not checked if your mirror will automatically check

If the preference in Ubuntu is to keep using mysql with this package, someone’s going to need to revert the change in the package that’s been mirrored for to not only depend on mysql and investigate why it’s not working or work with upstream to understand why they’re choosing to depend on mariadb instead.

Cheers!

[1] https://github.com/ok2cqr/cqrlog/issues/291#issuecomment-770383265
[2] https://salsa.debian.org/debian-hamradio-team/cqrlog/-/blob/master/debian/control?ref_type=heads#L36

--
  Hibby
  Debian Developer
  Packet Radioist
  MM0RFN

[Robie Basak (racb)]

> If the preference in Ubuntu is to keep using mysql with this package, someone’s going to need to revert the change in the package that’s been mirrored for to not only depend on mysql and investigate why it’s not working or work with upstream to understand why they’re choosing to depend on mariadb instead.

Indeed. Thanks! I don't think it's appropriate to commit to such a change in Ubuntu's stable releases without this question resolved.

I'm not aware of any reason that cqrlog shouldn't be able to be compatibility with both MySQL and MariaDB, and it's surprising to me that it isn't. This could even be trivial to fix.

[Dave Hibberd (hibby)]

Someone has emailed the upstream author asking if he wishes to clarify his design choice for you.

If there's any updates, I'll make sure they'll appear here.

--
  Hibby
  Debian Developer
  Packet Radioist
  MM0RFN

[AsciiWolf (asciiwolf)]

If changing the CQRLOG MySQL Server dependency to depend on MariaDB specifically instead of the meta package is not suitable for Ubuntu, then I think that it will be better to remove the cqrlog package from the Ubuntu repository instead of shipping a completely broken one.

It is very clearly stated on multiple places that CQRLOG requires MariaDB:

https://github.com/ok2cqr/cqrlog/blob/master/help/index_right.html#L101
https://github.com/ok2cqr/cqrlog/commit/95ba6e6e96d644b01413d1f395cafa1d3855f1e1
https://github.com/ok2cqr/cqrlog/issues/291#issuecomment-770383265

Anyway, let's wait for the upstream author reply.

[Robie Basak (racb)]

To be clear:

MySQL and MariaDB remain broadly compatible. Ubuntu has maintained its status quo in defaulting to MySQL since before MariaDB existed. Users expect consistency, and this means that packages in Ubuntu should default to using MySQL over MariaDB unless the user has explicitly chosen otherwise.

If crqlog cannot work against MySQL, for example because it uses a feature of MariaDB that MySQL doesn't have or is too difficult technically to patch to use MySQL's equivalent feature instead, then that's fine - it would be appropriate then to make an exception for this package. But given how close the two forks are, and the nature of what cqrlog does, this seems unlikely to me.

If on the other hand a trivial patch could be used to maintain compatibility against both MySQL and MariaDB, then I expect developers who want to look after cqrlog in Ubuntu to write that trivial patch, send it to Debian (since MySQL is still maintained in Debian sid) and to upstream, etc. This requires investigation by an interested developer - they don't have to be upstream. Given that MySQL has become more strict about the SQL it accepts over time, it seems likely to me that this is the issue and the fix would not only work for both sides of the fork, but also be preferable for a change for upstream to accept anyway.

We should avoid going back and forth in stable releases, so the long term decision here for ongoing development and maintenance of the cqrlog package in Ubuntu should be settled _before_ making changes to the stable releases.

If as reported this hasn't worked since Ubuntu 20.04, then clearly there is no need for urgency here, and we should take the time to resolve this situation properly to give the best experience to our users in the future.

For now, then, I'm rejecting these uploads from the queue.

[Dave Hibberd (hibby)]

No problems.

It is probably worth filing a bug against https://launchpad.net/ubuntu/+source/cqrlog/2.5.2-5 to help find someone to write the required patch and/or to stop it hitting the upcoming 24.10 release with a dependency on mariadb instead of mysql.

As I'm generally unfamiliar with Ubuntu process and requirements, I'll leave that decision and task to others.

Thanks for the review!

--
  Hibby
  Debian Developer
  Packet Radioist
  MM0RFN

On Wed, 4 Sep 2024, at 2:38 PM, Robie Basak wrote:
> To be clear:
>
> MySQL and MariaDB remain broadly compatible. Ubuntu has maintained its
> status quo in defaulting to MySQL since before MariaDB existed. Users
> expect consistency, and this means that packages in Ubuntu should
> default to using MySQL over MariaDB unless the user has explicitly
> chosen otherwise.
>
> If crqlog cannot work against MySQL, for example because it uses a
> feature of MariaDB that MySQL doesn't have or is too difficult
> technically to patch to use MySQL's equivalent feature instead, then
> that's fine - it would be appropriate then to make an exception for this
> package. But given how close the two forks are, and the nature of what
> cqrlog does, this seems unlikely to me.
>
> If on the other hand a trivial patch could be used to maintain
> compatibility against both MySQL and MariaDB, then I expect developers
> who want to look after cqrlog in Ubuntu to write that trivial patch,
> send it to Debian (since MySQL is still maintained in Debian sid) and to
> upstream, etc. This requires investigation by an interested developer -
> they don't have to be upstream. Given that MySQL has become more strict
> about the SQL it accepts over time, it seems likely to me that this is
> the issue and the fix would not only work for both sides of the fork,
> but also be preferable for a change for upstream to accept anyway.
>
> We should avoid going back and forth in stable releases, so the long
> term decision here for ongoing development and maintenance of the cqrlog
> package in Ubuntu should be settled _before_ making changes to the
> stable releases.
>
> If as reported this hasn't worked since Ubuntu 20.04, then clearly there
> is no need for urgency here, and we should take the time to resolve this
> situation properly to give the best experience to our users in the
> future.
>
> For now, then, I'm rejecting these uploads from the queue.
>
> ** Changed in: cqrlog (Ubuntu Focal)
> Status: In Progress => Incomplete
>
> ** Changed in: cqrlog (Ubuntu Jammy)
> Status: In Progress => Incomplete
>
> ** Changed in: cqrlog (Ubuntu Noble)
> Status: In Progress => Incomplete
>
> --
> You received this bug notification because you are a member of Ubuntu
> ham developers, which is subscribed to the bug report.
> https://bugs.launchpad.net/bugs/1872002
>
> Title:
> [SRU] cqrlog needs to depend on mariadb instead of mysql
>
> Status in cqrlog package in Ubuntu:
> Fix Released
> Status in cqrlog source package in Focal:
> Incomplete
> Status in cqrlog source package in Jammy:
> Incomplete
> Status in cqrlog source package in Noble:
> Incomplete
>
> Bug description:
> [ Impact ]
>
> cqrlog is unusable for ...