mysqli: Using a cursor with get_result() and prepared statements causes a segmentation fault

[Matthew Ruffell]

[Impact]

If you attempt to use a prepared statement with the mysqli database driver to create a cursor which you then execute() and fetch rows with get_result() and then fetch_assoc(), php with hit a segmentation fault on every query and terminate.

This is because cursors aren't actually implemented for prepared statements for the mysqli driver in php 7.2 and 7.4, the versions in Bionic and Focal. When we try and use a cursor, we just segfault on a type mismatch when the cursor calls fetch_row().

The fix comes in two forms. The first commit fixes the segfault and makes php return an error to the user, and the second commit implements support for cursors on prepared statements. When combined, these commits fix the issue.

A workaround is to not use prepared statements, and instead use query() directly.

[Test case]

Install PHP and mysql-client:

Focal:
$ sudo apt install php7.4 php7.4-mysql mysql-client

Bionic:
$ sudo apt install php7.2 php7.2-mysql mysql-client

Next, install and configure mysql 5.3:

$ sudo apt install libncurses5 libaio1 libmecab2
$ wget https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.35-linux-glibc2.12-x86_64.tar.gz

$ groupadd mysql
$ useradd -r -g mysql -s /bin/false mysql
$ cd /usr/local
$ tar zxvf /home/ubuntu/mysql-5.7.35-linux-glibc2.12-x86_64.tar.gz
$ ln -s mysql-5.7.35-linux-glibc2.12-x86_64 mysql
$ cd mysql
$ mkdir mysql-files
$ chown mysql:mysql mysql-files
$ chmod 750 mysql-files
$ bin/mysqld --initialize --user=mysql
$ bin/mysql_ssl_rsa_setup
$ bin/mysqld_safe --user=mysql &

# [Note] A temporary password is generated for root@localhost: *rjfy#_w(8kM

Access the DBMS and add users, databases and tables, along with some data:

$ mysql -h 127.0.0.1 -P 3306 -u root -p

ALTER USER 'root'@'localhost' IDENTIFIED BY 'ubuntu';

CREATE DATABASE ubuntu_releases;
use ubuntu_releases;
CREATE TABLE ubuntu_releases (year INT, month INT, name VARCHAR(20)) ;
INSERT INTO ubuntu_releases VALUES (21, 04, 'hirsute');
INSERT INTO ubuntu_releases VALUES (20, 10, 'groovy');
INSERT INTO ubuntu_releases VALUES (20, 04, 'focal');
CREATE USER 'ubuntu' IDENTIFIED BY 'ubuntu';
GRANT ALL PRIVILEGES ON ubuntu_releases.* TO 'ubuntu';
exit

Save the following script to testcase.php:

<?php
 $dbConn= new mysqli("127.0.0.1", "ubuntu", "ubuntu", "ubuntu_releases", "3306");
 $SQL="SELECT * from ubuntu_releases";
 $stmt=$dbConn->prepare($SQL);
 $stmt->attr_set(MYSQLI_STMT_ATTR_CURSOR_TYPE, MYSQLI_CURSOR_TYPE_READ_ONLY);
 $stmt->execute();
 if ($stmt) {
  $res = $stmt->get_result();
  while($row = $res->fetch_assoc()) {
   echo json_encode($row) . "\n";
  }
 }
 $dbConn->close()
?>

Run the php script:

$ php testcase.php
Segmentation fault (core dumped)

A ppa with test packages is available below:

https://launchpad.net/~mruffell/+archive/ubuntu/sf315485-test

When you install the test packages, you should see:

$ php testcase.php
{"year":21,"month":4,"name":"hirsute"}
{"year":20,"month":10,"name":"groovy"}
{"year":20,"month":4,"name":"focal"}

[Where problems can occur]

We are changing the behaviour of how cursors work in the mysqli database driver backend. Luckily, we are only changing how they work for prepared statements in mysqli, and it doesn't affect any other database driver backend, or regular queries with mysqli.

Since attempting to use a cursor with prepared statements on mysqli backend results in a segfault due to it not being implemented, there won't be any users that are using cursors with prepared statements, since their applications would crash. Adding support likely won't break any existing users, as attempting to use such features before would result in a hard crash, versus making existing code start working or behave differently.

There is still risk these changes could introduce a regression, and it would be restricted to users using the mysqli database driver, which I imagine are a significant amount of the userbase, due to the popularity of mysql and mariadb. If a regression were to occur, users might need to change their database driver interface code, or in worst case, change to direct queries from prepared statements while a fix is created.

[Other Info]

The following commit fixes the segmentation fault and changes it to an not implemented error:

commit b5481defe64c991d0e4307372d69c0ea3cd83378
Author: Dharman <email address hidden>
Date: Thu Sep 17 12:35:26 2020 +0100
Subject: Fix bug #72413: Segfault with get_result and PS cursors
Link: https://github.com/php/php-src/commit/b5481defe64c991d0e4307372d69c0ea3cd83378

The following commit implements support for cursors on prepared statements:

commit bc166844e37a6e1531a18dc0916fbe508152fc6c
Author: Nikita Popov <email address hidden>
Date: Wed Dec 16 12:12:06 2020 +0100
Subject: MySQLnd: Support cursors in store/get result
Link: https://github.com/php/php-src/commit/bc166844e37a6e1531a18dc0916fbe508152fc6c

Both of these commits landed in php 7.4.13 and 7.4.15, which means they are already present in hirsute 7.5.16 and impish 8.0.

Both commits required backports to remove the NEWS hunk, and the second commit needed indentation adjustments. For Bionic, the second commit required two changes. There are two hunks in ext/mysqlnd/mysqlnd_ps.c particularly around the function mysqlnd_stmt_send_cursor_fetch_command() and a small change to ext/mysqli/tests/bug77935.phpt, moving ); after SQL block.

[Matthew Ruffell (mruffell)]

Attached is a debdiff for Focal.

I have run the three tests included in these changes, and they all succeed:

ext/mysqli/tests/ps_cursor_multiple_result_sets.phpt
ext/mysqli/tests/bug77935.phpt
ext/mysqli/tests/mysqli_stmt_get_result.phpt

https://paste.ubuntu.com/p/ZBz9wtXcDr/

[Matthew Ruffell (mruffell)]

Attached is a debdiff for Bionic.

I also ran the three tests in this patch, and they all pass:

ext/mysqli/tests/bug77935.phpt
ext/mysqli/tests/ps_cursor_multiple_result_sets.phpt
ext/mysqli/tests/mysqli_stmt_get_result.phpt

https://paste.ubuntu.com/p/5422Hb6YMg/

[Dan Streetman (ddstreet)]

uploaded to b/f queues, thanks!

[Brian Murray (brian-murray)]

Hello Matthew, or anyone else affected,

Accepted php7.4 into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/php7.4/7.4.3-4ubuntu2.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted php7.4 (7.4.3-4ubuntu2.6) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

php-luasandbox/3.0.3-2build2 (arm64)
php-parser/4.2.2-2 (amd64, ppc64el, armhf, arm64, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#php7.4

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Chris Halse Rogers (raof)]

Hello Matthew, or anyone else affected,

Accepted php7.2 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/php7.2/7.2.24-0ubuntu0.18.04.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Matthew Ruffell (mruffell)]

Performing verification for Focal.

I installed php7.4 and php7.4-mysql from -updates, version 7.4.3-4ubuntu2.5.

I also installed mysql and set up the database table and inserted data into it.

$ sudo apt-cache policy php7.4 | grep Installed
  Installed: 7.4.3-4ubuntu2.5

$ cat testcase.php
<?php
 $dbConn= new mysqli("127.0.0.1", "ubuntu", "ubuntu", "ubuntu_releases", "3306");
 $SQL="SELECT * from ubuntu_releases";
 $stmt=$dbConn->prepare($SQL);
 $stmt->attr_set(MYSQLI_STMT_ATTR_CURSOR_TYPE, MYSQLI_CURSOR_TYPE_READ_ONLY);
 $stmt->execute();
 if ($stmt) {
  $res = $stmt->get_result();
  while($row = $res->fetch_assoc()) {
   echo json_encode($row) . "\n";
  }
 }
 $dbConn->close()
?>

$ php testcase.php
Segmentation fault (core dumped)
$ tail /var/log/kern.log
Sep 1 02:58:22 ubuntu kernel: [ 9394.000341] php[12973]: segfault at 1700000119 ip 00007f0f2bc575d0 sp 00007ffceb8d0bb0 error 4 in mysqlnd.so[7f0f2bc3c000+20000]
Sep 1 02:58:22 ubuntu kernel: [ 9394.000363] Code: 24 08 48 8b 04 24 4c 8b 44 24 10 48 8b 3e 8b 56 08 48 89 38 89 50 08 c7 46 08 01 00 00 00 eb 82 66 2e 0f 1f 84 00 00 00 00 00 <49> 8b bc 24 18 01 00 00 48 8b 15 11 5a 01 00 be de 07 00 00 48 8b

We segfault trying to use the mysqli cursor.

I then enabled -proposed, and installed php7.4 version 7.4.3-4ubuntu2.6.

$ sudo apt-cache policy php7.4 | grep Installed
  Installed: 7.4.3-4ubuntu2.6

$ php testcase.php
{"year":21,"month":4,"name":"hirsute"}
{"year":20,"month":10,"name":"groovy"}
{"year":20,"month":4,"name":"focal"}

We no longer segfault, and we can read rows from the mysqli cursor correctly.

The fixed package in -proposed solves the issue. Happy to mark Focal as verified.

[Matthew Ruffell (mruffell)]

Performing verification for Bionic.

I installed php7.2 and php7.2-mysql from -updates, version 7.4.3-4ubuntu2.5.

I also installed mysql and set up the database table and inserted data into it.

$ sudo apt-cache policy php7.2 | grep Installed
  Installed: 7.2.24-0ubuntu0.18.04.8

$ cat testcase.php
<?php
 $dbConn= new mysqli("127.0.0.1", "ubuntu", "ubuntu", "ubuntu_releases", "3306");
 $SQL="SELECT * from ubuntu_releases";
 $stmt=$dbConn->prepare($SQL);
 $stmt->attr_set(MYSQLI_STMT_ATTR_CURSOR_TYPE, MYSQLI_CURSOR_TYPE_READ_ONLY);
 $stmt->execute();
 if ($stmt) {
  $res = $stmt->get_result();
  while($row = $res->fetch_assoc()) {
   echo json_encode($row) . "\n";
  }
 }
 $dbConn->close()
?>

$ php testcase.php
Segmentation fault (core dumped)
$ tail /var/log/kern.log
Sep 1 03:33:13 ubuntu kernel: [ 843.642697] do_general_protection: 5 callbacks suppressed
Sep 1 03:33:13 ubuntu kernel: [ 843.642705] traps: php[10167] general protection ip:7f9256b72f90 sp:7ffc6b263670 error:0 in mysqlnd.so[7f9256b4a000+38000]

We segfault trying to use the mysqli cursor.

I then enabled -proposed, and installed php7.2 version 7.2.24-0ubuntu0.18.04.9.

$ sudo apt-cache policy php7.2 | grep Installed
  Installed: 7.2.24-0ubuntu0.18.04.9

$ php testcase.php
{"year":21,"month":4,"name":"hirsute"}
{"year":20,"month":10,"name":"groovy"}
{"year":20,"month":4,"name":"focal"}

We no longer segfault, and we can read rows from the mysqli cursor correctly.

The fixed package in -proposed solves the issue. Happy to mark Bionic as verified.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted php7.2 (7.2.24-0ubuntu0.18.04.9) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

mediawiki/1:1.27.4-3 (amd64)
php-horde-db/2.4.0-1ubuntu2 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#php7.2

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Matthew Ruffell (mruffell)]

I took a look at the autopkgtest regression for php-luasandbox on arm64.

https://autopkgtest.ubuntu.com/results/autopkgtest-focal/focal/arm64/p/php-luasandbox/20210908_213035_50d50@/log.gz

We fail on:

FAIL profiler sorting [tests/profiler-sorting.phpt]

I started a t4.medium arm64 instance on AWS, installed autopkgtest and ran:

$ sudo autopkgtest php-luasandbox_3.0.3-2build2.dsc --apt-pocket=proposed -- null

Each run, the test always passes:

PASS profiler sorting [tests/profiler-sorting.phpt]

Interesting. Full test output below:

https://paste.ubuntu.com/p/3kZ2Yj4nKt/

The software I was running to conduct the tests, clearly showing php7.4 from -proposed is installed:

$ uname -a
Linux ip-172-31-28-33 5.11.0-1017-aws #18~20.04.1-Ubuntu SMP Fri Aug 27 11:19:40 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

$ sudo apt-cache policy php7.4-cli
php7.4-cli:
  Installed: 7.4.3-4ubuntu2.6
  Candidate: 7.4.3-4ubuntu2.6
  Version table:
 *** 7.4.3-4ubuntu2.6 500
        500 http://ap-southeast-2c.clouds.ports.ubuntu.com/ubuntu-ports focal-proposed/main arm64 Packages
        100 /var/lib/dpkg/status

So why does the test consistently fail on autopkgtest infrastructure?

Looking at the test itself:

https://paste.ubuntu.com/p/kwZNrgb3fk/

We see why. The test itself is based on completing three busy loops counting from 0 to 1e6, 0 to 4e6 and 0 to 2e6, all within a 0.5s window. If the arm64 based autopkgtest server / VM is under high load, then it might get scheduled out and not complete within the timeframe, and cause the loop to break and the test to error.

The test is inherently flaky and based on VM performance. On AWS with just this running, the autopkgtest passes every time. This should not be a reason to prevent php7.4 7.4.3-4ubuntu2.6 from being released to -updates.

[Launchpad Janitor (janitor)]

This bug was fixed in the package php7.4 - 7.4.3-4ubuntu2.6

---------------
php7.4 (7.4.3-4ubuntu2.6) focal; urgency=medium

  * Fix a segmentation fault and implement support for using cursors
    on prepared statements in the mysqli database driver. (LP: #1939853)
    - d/p/lp-1939853-1-Fix-Segfault-with-get_result-and-PS-cursors.patch
    - d/p/lp-1939853-2-MySQLnd-Support-cursors-in-store-get-result.patch

 -- Matthew Ruffell <email address hidden> Fri, 13 Aug 2021 17:39:12 +1200

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for php7.4 has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package php7.2 - 7.2.24-0ubuntu0.18.04.9

---------------
php7.2 (7.2.24-0ubuntu0.18.04.9) bionic; urgency=medium

  * Fix a segmentation fault and implement support for using cursors
    on prepared statements in the mysqli database driver. (LP: #1939853)
    - d/p/lp-1939853-1-Fix-Segfault-with-get_result-and-PS-cursors.patch
    - d/p/lp-1939853-2-MySQLnd-Support-cursors-in-store-get-result.patch

 -- Matthew Ruffell <email address hidden> Mon, 16 Aug 2021 17:46:32 +1200

[Bryce Harrington (bryce)]

The test flaked again in jammy with php8.1 on armhf. I'm retriggering until it passes now that some of the armhf platform issues have been resolved but given comment #10's findings, wonder if disabling the test case (at least for 32-bit) might be a more sustainable solution.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "Debdiff for php on Focal" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]