Swift Accept Header does not comply with RFC rules on quality values

[Bill Huber]

Swift is not complying with the RFC rules below on quality values. Using the Accept Header support, Swift is able to generate more than 3 digits after the decimal point instead of producing a status code of 406 - NOT_ACCEPTABLE. Example below after the description:

3.9 Quality Values

 HTTP content negotiation (section 12) uses short "floating point"
 numbers to indicate the relative importance ("weight") of various
 negotiable parameters. A weight is normalized to a real number in
 the range 0 through 1, where 0 is the minimum and 1 the maximum
 value. If a parameter has a quality value of 0, then content with
 this parameter is `not acceptable' for the client. HTTP/1.1
 applications MUST NOT generate more than three digits after the
 decimal point. User configuration of these values SHOULD also be
 limited in this fashion.

curl -X GET -i -H "X-Auth-Token: $token" http://127.0.0.1:8080/v1/AUTH_test/Tont -H "Accept: text/xml;q=0.2234"
HTTP/1.1 200 OK
Content-Length: 867
X-Container-Object-Count: 4
Accept-Ranges: bytes
X-Storage-Policy: gold
X-Container-Bytes-Used: 0
X-Timestamp: 1457655301.60066
Content-Type: text/xml; charset=utf-8
X-Trans-Id: txf92b4adc15774f8bb29b0-0056fa95bb
Date: Tue, 29 Mar 2016 14:48:27 GMT

<?xml version="1.0" encoding="UTF-8"?>
<container name="Tont"><object><name>c.zip</name><hash>d41d8cd98f00b204e9800998ecf8427e</hash><bytes>0</bytes><content_type>application/zip</content_type><last_modified>2016-03-29T14:31:56.509850</last_modified></object><object><name>e.zip</name><hash>d41d8cd98f00b204e9800998ecf8427e</hash><bytes>0</bytes><content_type>application/zip</content_type><last_modified>2016-03-15T18:33:00.386120</last_modified></object><object><name>sample.zip</name><hash>d41d8cd98f00b204e9800998ecf8427e</hash><bytes>0</bytes><content_type>application/zip</content_type><last_modified>2016-03-28T19:53:50.736990</last_modified></object><object><name>t.zip</name><hash>d41d8cd98f00b204e9800998ecf8427e</hash><bytes>0</bytes><content_type>application/zip</content_type><last_modified>2016-03-29T14:09:48.812170</last_modified></object></container>

[Christian Schwede (cschwede)]

I don't think this is a valid bug. Swift does not generate more than 3 digits after the decimal point; in the example above Swift _consumes_ a header with more than 3 digits. The error is on the client side in this case. I can't find a reference in the RFC (https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html) that an application should return a 406 in this case.

[Bill Huber (wbhuber)]

According to the 406 status code on RFC - please read the Note:section below. I take it that Swift is a user agent that inspects the field passed in in the Accept request header. In common/swob.py, we parse through the field but come short of deciphering the quality value (lines #682-#690) and limiting it to three digits after the decimal point.

https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

10.4.7 406 Not Acceptable

The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.

Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.

      Note: HTTP/1.1 servers are allowed to return responses which are
      not acceptable according to the accept headers sent in the
      request. In some cases, this may even be preferable to sending a
      406 response. User agents are encouraged to inspect the headers of
      an incoming response to determine if it is acceptable.

[Tim Burke (1-tim-z)]

Swift is a *server*; the *client* (whether python-swiftclient, a
browser, cURL, or some other HTTP client) is the user agent.

The note is clarifying that the client *should not assume that its
Accept header was respected*. Instead, it should inspect the headers
of the response (in particular, the Content-Type, Content-Encoding,
and Content-Language headers) to determine whether the response is
actually something it can handle. This reflects the server behavior
specified in the Accept header definition [1] (note the use of SHOULD
rather than MUST):

> If an Accept header field is present, and if the server cannot send
> a response which is acceptable according to the combined Accept
> field value, then the server SHOULD send a 406 (not acceptable)
> response.

RFC 7231 walks this back even further and explicitly lets the server
choose whether to respect the header [2]:

> If the header field is present in a request and none of the
> available representations for the response have a media type that
> is listed as acceptable, the origin server can either honor the
> header field by sending a 406 (Not Acceptable) response or disregard
> the header field by treating the response as if it is not subject
> to content negotiation.

So, the client (in this case, cURL) is misbehaving by sending a
q value with more than 3 decimal points. If we take a rather narrow
reading of the RFC, we've got two options: 400 because of the bad
header, or ignore the header entirely. A 406 would be inappropriate;
the problem is not that the server lacks an acceptable representation,
but rather that the client sent a malformed header. Swift is liberal
in what it accepts and takes a third option: as long as we can make
sense of the value, do what the client seemed to want.

That said, I think there *is* a valid (if low-priority) bug in
swob: any failure to parse the Accept header results in a 406,
even when a 400 may be more appropriate. In particular, when
performing an account or container listing, both

  Accept: text/xml;q=foo

and

  Accept: text/xml;q=0.8;q=0.5

...trigger a 406, despite the fact that Swift can send a text/xml
response.

[1] http://tools.ietf.org/html/rfc2616#section-14.1
[2] http://tools.ietf.org/html/rfc7231#section-5.3.2

[Tim Burke (1-tim-z)]

Kicked out https://bugs.launchpad.net/swift/+bug/1716509 as a separate issue; closing this as not-a-bug.