apache2 2.4.41-4ubuntu3.17 defaults to transfer-encoding=chunked where this is undesired

[Tim Andersson]

We (autopkgtest.ubuntu.com) had an unattended upgrade recently bump our apache version to 2.4.41-4ubuntu3.17.

This was problematic for us - we serve static files through apache and seemingly after this version bump, our transfer encodings are now ALWAYS set to be "chunked".

This is an issue, as we have external services which periodically download these static files I've mentioned, which need to check the Content-Length header whilst downloading these files.

I've manually installed 2.4.41-4ubuntu3.16 on our staging servers and verified that this patch is indeed the root cause of the issue.

with 2.4.41-4ubuntu3.16:
```
 wget -v -d https://autopkgtest.staging.ubuntu.com/static/autopkgtest.db
...
---request begin---
GET /static/autopkgtest.db HTTP/1.1
Host: autopkgtest.staging.ubuntu.com
User-Agent: Wget/1.21.3
Accept: */*
Accept-Encoding: identity
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... w
---response begin---
HTTP/1.1 200 OK
date: Tue, 16 Apr 2024 10:15:41 GMT
server: Apache/2.4.41 (Ubuntu)
cache-control: public, max-age=60
expires: Tue, 16 Apr 2024 10:16:41 GMT
accept-ranges: bytes
content-length: 1434976256
last-modified: Tue, 16 Apr 2024 10:14:27 GMT
etag: "1713262467.7848494-1434976256-79961951"
content-type: application/octet-stream
set-cookie: SRVNAME=S0; path=/

---response end---
200 OK

Stored cookie autopkgtest.staging.ubuntu.com -1 (ANY) / <session> <insecure> [expiry none] SRVNAME S0
Registered socket 3 for persistent reuse.
Length: 1434976256 (1.3G) [application/octet-stream]
Saving to: ‘autopkgtest.db’
```

with 2.4.41-4ubuntu3.17:
```
---request begin---
GET /static/autopkgtest.db HTTP/1.1
Host: autopkgtest.staging.ubuntu.com
User-Agent: Wget/1.21.3
Accept: */*
Accept-Encoding: identity
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
date: Tue, 16 Apr 2024 10:23:50 GMT
server: Apache/2.4.41 (Ubuntu)
cache-control: public, max-age=60
expires: Tue, 16 Apr 2024 10:24:51 GMT
accept-ranges: bytes
last-modified: Tue, 16 Apr 2024 10:23:27 GMT
etag: "1713263007.8384898-1434976256-79961951"
transfer-encoding: chunked
content-type: application/octet-stream
set-cookie: SRVNAME=S0; path=/

---response end---
200 OK

Stored cookie autopkgtest.staging.ubuntu.com -1 (ANY) / <session> <insecure> [expiry none] SRVNAME S0
Registered socket 3 for persistent reuse.
Length: unspecified [application/octet-stream]
Saving to: ‘autopkgtest.db’
```

And you can clearly see the content length isn't reported.

Is this the intended behaviour? Are there ways I can workaround this to not use chunked transfer encoding?

[Marc Deslauriers (mdeslaur)]

Thanks for filing this bug, I'll investigate the changes and will report back.

Have you seen this behaviour on anything other than focal?

[Marc Deslauriers (mdeslaur)]

I believe I've spotted the regression and will have a package to test soon.

[Marc Deslauriers (mdeslaur)]

I have uploaded a package with a possible fix to the security team PPA here:

https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages

Once it's finished building, could you please give it a try and see if it solves the issue for you? If so, I will publish it as a security regression fix. Thanks!

[Tim Andersson (andersson123)]

Thanks Marc! Wow that's great, unbelievably fast response! I'll test this right away and get back to you

[Tim Andersson (andersson123)]

Hi Marc,

I'm afraid the patch didn't work :( here's the same output as before:
```
---request begin---
GET /static/autopkgtest.db HTTP/1.1
Host: autopkgtest.staging.ubuntu.com
User-Agent: Wget/1.21.3
Accept: */*
Accept-Encoding: identity
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
date: Tue, 16 Apr 2024 12:24:15 GMT
server: Apache/2.4.41 (Ubuntu)
cache-control: public, max-age=60
expires: Tue, 16 Apr 2024 12:25:15 GMT
accept-ranges: bytes
last-modified: Tue, 16 Apr 2024 12:23:51 GMT
etag: "1713270231.1219802-1434976256-79961951"
transfer-encoding: chunked
content-type: application/octet-stream
set-cookie: SRVNAME=S0; path=/

---response end---
200 OK

Stored cookie autopkgtest.staging.ubuntu.com -1 (ANY) / <session> <insecure> [expiry none] SRVNAME S0
Registered socket 3 for persistent reuse.
Length: unspecified [application/octet-stream]
Saving to: ‘autopkgtest.db’
```

[Marc Deslauriers (mdeslaur)]

Thanks for testing, I'll keep digging...

[Marc Deslauriers (mdeslaur)]

I think this is actually the correct new behaviour for the security update...could you please try using ap_trust_cgilike_cl as instructed here:

https://bz.apache.org/bugzilla/show_bug.cgi?id=68872

[Tim Andersson (andersson123)]

Awesome, thanks so much mark! I added this line:

SetEnvIf Request_URI /static/autopkgtest.db ap_trust_cgilike_cl¬

And all is well. Thank you.

[Tim Andersson (andersson123)]

btw, feel free to mark this as invalid now :)

[Marc Deslauriers (mdeslaur)]

That's good to see!

Since this is a deliberate side-effect of the security change, I am marking this bug as "invalid". Thanks