cloud-init

Bug #1868232
Comment #1

Comment 1 for bug 1868232

Revision history for this message

Dan Watkins (oddbloke) wrote on 2020-03-20:

The current behaviour is that cloud-init will use the patterns defined in /etc/cloud/cloud.cfg:

primary:
   - http://%(ec2_region)s.ec2.archive.ubuntu.com/ubuntu/
   - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/
   - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/

to determine the mirror to use. In this case, it will be one of the two latter patterns, depending on exactly how the data source in question presents "zone_01". Either way, the problem is the same.

Once this mirror URL is generated, cloud-init tests that it _resolves_ before using it. This is where the problem lies: *.clouds.archive.ubuntu.com will always resolve, but the newly-deployed Apache servers will no longer serve every domain that resolves. Arguably this is a misconfiguration of the archive servers (why resolve something that you can't serve?), but cloud-init should handle this case gracefully regardless.

There are (at least) a couple of ways in which we could address this issue in cloud-init:

(a) rewrite the generated URL (or the variables which we are substituting into the pattern) to only include valid URI characters
(b) modify cloud-init to check that mirrors are accessible via HTTP (rather than simply resolvable)

While both of these would address the immediate issue, only implementing (b) would mean that all instances in such zones would fallback to using archive.ubuntu.com, so I think we should do some form of (a) regardless.

One obvious downside to (b) is that it will introduce an additional HTTP request to each boot on a Debian/Ubuntu host; this could be a concern both from a client boot speed perspective, but perhaps more importantly from a server load perspective. (My gut feel is that the cost in both cases wouldn't be significantly noticeable: most Debian/Ubuntu instances that come up will perform many HTTP requests to the archive hosts, so one additional one isn't likely to be noticed. We should consider this more deeply before we implement this, however.)

(As an aside, we should do some research to confirm that the non-ASCII encoding described in the linked RFC 3986 section won't be affected by our filtering. For example, if we currently rely on the libraries we use to convert non-ASCII hostnames to the defined percent-encoding, then we would regress non-ASCII hostnames by applying a naive filter before we pass the name to those libraries.)

The current behaviour is that cloud-init will use the patterns defined in /etc/cloud/cloud.cfg:

primary:
   - http://%(ec2_region)s.ec2.archive.ubuntu.com/ubuntu/
   - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/
   - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/

to determine the mirror to use.  In this case, it will be one of the two latter patterns, depending on exactly how the data source in question presents "zone_01".  Either way, the problem is the same.

Once this mirror URL is generated, cloud-init tests that it _resolves_ before using it.  This is where the problem lies: *.clouds.archive.ubuntu.com will always resolve, but the newly-deployed Apache servers will no longer serve every domain that resolves.  Arguably this is a misconfiguration of the archive servers (why resolve something that you can't serve?), but cloud-init should handle this case gracefully regardless.

There are (at least) a couple of ways in which we could address this issue in cloud-init:

One obvious downside to (b) is that it will introduce an additional HTTP request to each boot on a Debian/Ubuntu host; this could be a concern both from a client boot speed perspective, but perhaps more importantly from a server load perspective.  (My gut feel is that the cost in both cases wouldn't be significantly noticeable: most Debian/Ubuntu instances that come up will perform many HTTP requests to the archive hosts, so one additional one isn't likely to be noticed.  We should consider this more deeply before we implement this, however.)

(As an aside, we should do some research to confirm that the non-ASCII encoding described in the linked RFC 3986 section won't be affected by our filtering.  For example, if we currently rely on the libraries we use to convert non-ASCII hostnames to the defined percent-encoding, then we would regress non-ASCII hostnames by applying a naive filter before we pass the name to those libraries.)