Apache 2.4.x: mod_http2 sends empty response after MaxRequestsPerChild
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Apache2 Web Server |
Fix Released
|
Medium
|
|||
apache2 (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Bionic |
Fix Released
|
Undecided
|
Sergio Durigan Junior | ||
Focal |
Fix Released
|
Undecided
|
Sergio Durigan Junior | ||
Impish |
Fix Released
|
Undecided
|
Sergio Durigan Junior |
Bug Description
[ Impact ]
Apache2 users with mod_http2 enabled can face problems when the MaxRequestsPerChild limit is reached. In this scenario, apache2 will send a GOAWAY packet too early in the connection, which will cause no data to be transferred to the client (i.e., an empty HTTP response from the server).
[ Test Plan ]
This problem reproduces on Bionic, Focal and Impish. The instructions are almost the same for all 3, but on Bionic an extra step is necessary to make sure apache2 will use HTTP2.
$ lxc launch ubuntu-daily:focal bug1969629-focal
$ lxc shell bug1969629-focal
# apt update
# apt install -y apache2
# a2enmod http2
# [ $(lsb_release -sc) = "bionic" ] && (echo -e "\nProtocols h2 h2c http/1.1" >> /etc/apache2/
# cat > /etc/apache2/
<IfModule mpm_event_module>
</IfModule>
__EOF__
# systemctl restart apache2.service
# while true; do if [[ $(curl -o /dev/null -s -k --http2 http://
BUG DETECTED
The "while" command above should never stop, but as can be seen, it does stop because the size of the downloaded file (index.html, in this case) is zero. This can also be confirmed by looking at the server logs:
127.0.0.1 - - [26/Apr/
127.0.0.1 - - "GET / HTTP/2.0" 200 0 "-" "curl/7.58.0"
[ Where problems could occur ]
Even though the patch is relatively simple, it touches an area that can be a bit complex: session processing of an HTTP request. There are upstream tests that exercise this area and I believe we can be reasonably confident that this change does the right thing. If a regression is to occur, it would likely manifest in the form of a bug in the HTTP2 session handling, and the quicker solution here would be to revert the change while revisiting the patch to determine what went wrong (possibly with upstream help).
[ Original Description ]
We run into a bug in mod_http2 for Apache 2.4.x.
Every time MaxRequestsPerChild is reached, an empty response is sent to the client:
https:/
This bug was fixed and backported to v2.4.52:
https:/
Any chance that this fix get backported to the Bionic and Focal Package?
Related branches
- Bryce Harrington (community): Approve
- Canonical Server: Pending requested
-
Diff: 92 lines (+70/-0)3 files modifieddebian/changelog (+8/-0)
debian/patches/mod_http2-Don-t-send-GOAWAY-too-early-when-MaxReques.patch (+61/-0)
debian/patches/series (+1/-0)
- Bryce Harrington (community): Approve
- Canonical Server: Pending requested
-
Diff: 92 lines (+70/-0)3 files modifieddebian/changelog (+8/-0)
debian/patches/mod_http2-Don-t-send-GOAWAY-too-early-when-MaxReques.patch (+61/-0)
debian/patches/series (+1/-0)
- Bryce Harrington (community): Approve
- Canonical Server: Pending requested
-
Diff: 92 lines (+70/-0)3 files modifieddebian/changelog (+8/-0)
debian/patches/mod_http2-Don-t-send-GOAWAY-too-early-when-MaxReques.patch (+61/-0)
debian/patches/series (+1/-0)
summary: |
- mod_http2: fixes 2 regressions in server limit handling. + Apache 2.4.x: mod_http2 sends empty response after MaxRequestsPerChild |
description: | updated |
Changed in apache2 (Ubuntu Bionic): | |
assignee: | nobody → Sergio Durigan Junior (sergiodj) |
Changed in apache2 (Ubuntu Focal): | |
assignee: | nobody → Sergio Durigan Junior (sergiodj) |
Changed in apache2 (Ubuntu Impish): | |
assignee: | nobody → Sergio Durigan Junior (sergiodj) |
Changed in apache2 (Ubuntu Bionic): | |
status: | Triaged → In Progress |
Changed in apache2 (Ubuntu Focal): | |
status: | Triaged → In Progress |
Changed in apache2 (Ubuntu Impish): | |
status: | Triaged → In Progress |
Changed in apache2: | |
importance: | Unknown → Medium |
status: | Unknown → Fix Released |
When requesting a simple image file with HTTP2 every time MaxRequestsPerChild is reached, an empty response is sent to the client. This is even reproducible.
Server Version: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f
Server MPM: event
OS: Ubuntu 20.04
Server Version: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1
Server MPM: worker
OS: Ubuntu 18.04
Same issue exists with mpm_event and mpm_worker.
My configuration:
<IfModule mpm_event_module> PerChild 100
StartServers 8
MinSpareThreads 4
MaxSpareThreads 12
ThreadLimit 64
ThreadsPerChild 16
MaxRequests
ServerLimit 64
MaxClients 64
</IfModule>
mod_http2 no custom configuration in use.
Test command: /example. net/2101583. jpg?test= $i -w '%{size_download}') == 0 ]]; then break; fi; done
for i in {1..2000}; do if [[ $(curl -o /dev/null -s -k --http2 https:/
Access log output: 2021:18: 20:32 +0100] "GET /2101583. jpg?fakeVar= 33 HTTP/2.0" 200 17115 "-" "curl/7.68.0" 2021:18: 20:33 +0100] "GET /2101583. jpg?fakeVar= 34 HTTP/2.0" 200 17115 "-" "curl/7.68.0" 2021:18: 20:33 +0100] "GET /2101583. jpg?fakeVar= 35 HTTP/2.0" 200 0 "-" "curl/7.68.0" 2021:18: 20:33 +0100] "GET /2101583. jpg?fakeVar= 36 HTTP/2.0" 200 17115 "-" "curl/7.68.0" 2021:18: 20:33 +0100] "GET /2101583. jpg?fakeVar= 37 HTTP/2.0" 200 17115 "-" "curl/7.68.0" 2021:18: 20:33 +0100] "GET /2101583. jpg?fakeVar= 38 HTTP/2.0" 200 17115 "-" "curl/7.68.0" 2021:18: 20:33 +0100] "GET /2101583. jpg?fakeVar= 39 HTTP/2.0" 200 17115 "-" "curl/7.68.0" 2021:18: 20:37 +0100] "GET /2101583. jpg?fakeVar= 135 HTTP/2.0" 200 0 "-" "curl/7.68.0" 2021:18: 20:42 +0100] "GET /2101583. jpg?fakeVar= 235 HTTP/2.0" 200 0 "-" "curl/7.68.0" 2021:18: 20:47 +0100] "GET /2101583. jpg?fakeVar= 335 HTTP/2.0" 200 0 "-" "curl/7.68.0"
8.8.8.8 - - [08/Dec/
8.8.8.8 - - [08/Dec/
8.8.8.8 - - [08/Dec/
8.8.8.8 - - [08/Dec/
8.8.8.8 - - [08/Dec/
8.8.8.8 - - [08/Dec/
8.8.8.8 - - [08/Dec/
...
8.8.8.8 - - [08/Dec/
...
8.8.8.8 - - [08/Dec/
...
8.8.8.8 - - [08/Dec/