Apache2: BalancerMember worker hostname (65.character.host.name) too long

Bug #1750356 reported by Graham Leggett on 2018-02-19
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Apache2 Web Server
Fix Released
Medium
apache2 (Ubuntu)
Undecided
Unassigned
Bionic
Low
Andreas Hasenack

Bug Description

[Impact]

If the BalancerMember directive contains a URL with a hostname longer than X characters, apache2 will fail to start with the following error:

BalancerMember worker hostname (65.character.host.name) too long

RFC1035 allows for longer hostnames, and apache upstream has this fix already.

[Test Case]

* Install the packages:
sudo apt install apache

* Edit /etc/apache2/sites-available/000-default.conf and add the following block inside the VirtualHost block:
  <Proxy "balancer://test">
    BalancerMember "http://xxxxxx-xx-xxxxxxxx-xxxxx-xxxxx.xx-xxxx-x.xxxxx-xxx.xxx.xxxxx.xxxx:90/"
    BalancerMember "http://xxxxxx-xx-xxxxxxxx-xxxxx-xxxxx.xx-xxxx-x.xxxxx-xxx.xxx.xxxxx.xxxx:91/"
  </Proxy>
  ProxyRequests Off
  <Proxy *>
    Order deny,allow
    Allow from all
  </Proxy>
  RewriteEngine On
  RewriteRule ^/foo balancer://test/foo [P,L]

* Enable the necessary apache modules:
sudo a2enmod proxy_balancer proxy lbmethod_byrequests proxy_http rewrite

* Restart apache2, which will fail:
sudo systemctl restart apache2

* Run the status action and expect an error like this:
sudo systemctl status apache2.service
...
Jun 27 18:31:16 bionic-apache-1750356 apachectl[2218]: BalancerMember worker hostname (xxxxxx-xx-xxxxxxxx-xxxxx-xxxxx.xx-xxxx-x.xxxxx-xxx.xxx.xxxxx.xxxx) too long

* Update the apache2 packages to the ones available in proposed. As part of the upgrade, apache2 will be restarted again, and in this time it will work. Confirm with systemctl status apache2 that there are no errors this time:
sudo systemctl status apache2

* Try to access http://localhost/foo to trigger the load balancer configuration. It will trigger a DNS error as we don't have an entry for the BalancerMember hostname, but it shows that the configuration worked:

ubuntu@bionic-apache-1750356:~$ wget http://localhost/foo
--2018-06-27 18:39:58-- http://localhost/foo
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 502 Proxy Error
2018-06-27 18:39:58 ERROR 502: Proxy Error.

ubuntu@bionic-apache-1750356:~$ tail /var/log/apache2/error.log -n 1
[Wed Jun 27 18:39:58.732097 2018] [proxy:error] [pid 3628:tid 139981565716224] [client 127.0.0.1:42508] AH00898: DNS lookup failure for: xxxxxx-xx-xxxxxxxx-xxxxx-xxxxx.xx-xxxx-x.xxxxx-xxx.xxx.xxxxx.xxxx returned by /foo

[Regression Potential]
This change is twofold: it allows for longer hostnames, and apache won't fail to start anymore if that length is exceeded. In the latter case, however, the hostname is truncated. With that in mind, here are some scenarios:
- hostnames larger than 65 characters but less than 255: before apache2 would fail to start, now it works.
- hostname larger than 255 characters. Before apache would fail to start; now, it starts but truncates that hostname, logging a warning. The configuration is likely to not work due to the truncation, which will lead to DNS errors. If the admin was only relying on (re)start errors to become aware of configuration problems, he/she might miss this until it's too late. But at least the log will be clear about what happened.
- third party modules that use apache's mod_proxy structure might not be aware of the new hostname_ex member which can hold the longer string, since we didn't update the MODULE_MAGIC_NUMBER_MINOR number with this patchset, and will probably remain exhibiting the problem described in this bug.

[Other Info]
The security team's regression test suite for apache2 (http://launchpad.net/qa-regression-testing) was run with the test packages from the PPA at https://launchpad.net/~ahasenack/+archive/ubuntu/apache-balance-member-hostname-1750356/+packages and passed: https://pastebin.ubuntu.com/p/nZ6GGHXgwQ/

== Original Description ==

If the BalancerMember directive contains a URL with a hostname longer than X characters, we fail as follows:

BalancerMember worker hostname (65.character.host.name) too long

The size of the hostname needs to be raised so it is RFC1035 compliant.

Bug fixed upstream at https://bz.apache.org/bugzilla/show_bug.cgi?id=62085, patches backported to v2.4.30:

http://svn.apache.org/r1824455
http://svn.apache.org/r1824504

(Both patches required, first is extended by second).

Related branches

If the BalancerMember directive contains a URL with a hostname longer than X characters, we fail as follows:

BalancerMember worker hostname (65.character.host.name) too long

The same fix for https://bz.apache.org/bugzilla/show_bug.cgi?id=53218 on the worker name needs to be applied to the hostname, and separately the size of the hostname needs to be raised so it is RFC1035 compliant in trunk.

Backported to v2.4.30:

http://svn.apache.org/r1824455
http://svn.apache.org/r1824504

(Both patches required, first is extended by second).

tags: added: xenial

Just triaging right now, but I agree.
And in Bionic we only have 2.4.29 at the moment.

If you have a test at hand could you also outline a steps to reproduce?
Those will eventually come handy to verify the fix.

tags: added: server-next
Changed in apache2 (Ubuntu):
status: New → Triaged
Graham Leggett (minfrin-y) wrote :

This breaks things for us:

<Proxy "balancer://test">
  BalancerMember "https://xxxxxx-xx-xxxxxxxx-xxxxx-xxxxx.xx-xxxx-x.xxxxx-xxx.xxx.xxxxx.xxxx:443/"
</Proxy>

Andreas Hasenack (ahasenack) wrote :

Confirmed broken in bionic, and working in cosmic.

Changed in apache2 (Ubuntu Bionic):
status: New → Triaged
Changed in apache2 (Ubuntu):
status: Triaged → Fix Released
Graham Leggett (minfrin-y) wrote :

Any chance of a fix for xenial? This is where we hit the issue.

Andreas Hasenack (ahasenack) wrote :

Hm, what should happen with this hunk of the patch?

--- 2.4.x/include/ap_mmn.h 2018/02/16 14:58:32 1824503
+++ 2.4.x/include/ap_mmn.h 2018/02/16 15:04:41 1824504
@@ -517,7 +518,7 @@
 #ifndef MODULE_MAGIC_NUMBER_MAJOR
 #define MODULE_MAGIC_NUMBER_MAJOR 20120211
 #endif
-#define MODULE_MAGIC_NUMBER_MINOR 74 /* 0...n */
+#define MODULE_MAGIC_NUMBER_MINOR 75 /* 0...n */

 /**
  * Determine if the server's current MODULE_MAGIC_NUMBER is at least a

Apache in bionic (2.4.29) has MODULE_MAGIC_NUMBER_MINOR set to 68

Andreas Hasenack (ahasenack) wrote :

I'm inclined to not change it, because 75 implies everything between 68 and 75, and 69 was probably used for something else. And it's a sequential number.

Graham Leggett (minfrin-y) wrote :

The module magic number gives code that depends on the apache API an indication of whether a particular function or variable is available or not.

I imagine Ubuntu already has a policy for the module magic number, and I suspect it's that the MMN doesn't change over the lifetime of a version of the distro. So the fields will be physically there, but code won't get any hint that they are, and nothing will break.

Changed in apache2:
importance: Unknown → Medium
status: Unknown → Fix Released
Andreas Hasenack (ahasenack) wrote :

I prepared test packages here: https://launchpad.net/~ahasenack/+archive/ubuntu/apache-balance-member-hostname-1750356/+packages

will propose a branch soon for bionic.

Changed in apache2 (Ubuntu Bionic):
assignee: nobody → Andreas Hasenack (ahasenack)
status: Triaged → In Progress
importance: Undecided → Low
description: updated
description: updated
description: updated
description: updated
Robie Basak (racb) wrote :

I'm not very keen on this from an SRU perspective. RFC or not, it's really a feature addition (the support for longer hostnames for BalancerMember) and the patch implements it as such (across nine source files). This makes the patch difficult to review comprehensively.

Do we really need this in an SRU? Can affected users use shorter hostnames instead?

Graham Leggett (minfrin-y) wrote :

> I'm not very keen on this from an SRU perspective. RFC or not, it's really a feature addition

This is a bug. RFC1035 describes how long a hostname must be, and httpd was not honouring the RFC. This doesn't implement "longer hostnames", this implements RFC compliant hostnames.

The httpd project has ABI guarantees in the v2.4 release series. This meant we had to support the older fields unchanged, while at the same time introducing new compliant fields. This is well established and well understood at httpd and is not out of the ordinary.

> Do we really need this in an SRU? Can affected users use shorter hostnames instead?

No.

The longer answer is "epically no". We have a huge estate of machines, all with names that are both descriptive and unique, and therefore long. There is no option to make them shorter. Instead of hacking around bugs, let's fix them.

Robie Basak (racb) wrote :

> This is a bug.

I'm not interested in arguments about semantics. My point is that from the point of view of the proposed code change it's the implementation of a new feature, with all the wide-ranging changes and regression risk that goes with it.

> This meant we had to support the older fields unchanged, while at the same time introducing new compliant fields. This is well established and well understood at httpd and is not out of the ordinary.

Ah - do you represent upstream? Part of my issue is in gaining confidence that the patch does the right thing in the absence of the rest of upstream microrelease changes. If someone familiar with the patch can vouch for it, that would help.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.