neutron

  1. Bug #1829042
  2. Comment #1

Comment 1 for bug 1829042

Revision history for this message
Nate Johnston (nate-johnston) wrote :

I believe we have conclusively determined in [1] that this is not a WebOb issue. I believe that there may be different codepaths between the neutron API methods that see this issue and those that do not. I will note that the error seen in the Neutron server.log when this happens tells me that it is not handling parsing the string correctly.

% curl -s -H "Accept: application/json;charset=utf-8" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/networks"
{
    "NeutronError": {
        "detail": "",
        "message": "The server could not comply with the request since it is either malformed or otherwise incorrect.",
        "type": "HTTPNotAcceptable"
    }
}

The logs:

2019-05-07 14:47:47.341 77 DEBUG neutron.wsgi [-] (77) accepted ('172.17.1.31', 36530) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:883
2019-05-07 14:47:47.345 77 ERROR pecan.core [req-58872c3a-e917-4e3e-82cf-d3dbd931b6b2 21ec7039924a46e9aec6a9fa490da066 fb53e9370fb0427fadb680238d90987a - default default] Controller 'index' defined does not support content_type 'None'. Supported type(s): ['application/json']
2019-05-07 14:47:47.346 77 INFO neutron.pecan_wsgi.hooks.translation [req-58872c3a-e917-4e3e-82cf-d3dbd931b6b2 21ec7039924a46e9aec6a9fa490da066 fb53e9370fb0427fadb680238d90987a - default default] GET failed (client error): The server could not comply with the request since it is either malformed or otherwise incorrect.
2019-05-07 14:47:47.347 77 INFO neutron.wsgi [req-58872c3a-e917-4e3e-82cf-d3dbd931b6b2 21ec7039924a46e9aec6a9fa490da066 fb53e9370fb0427fadb680238d90987a - default default] 172.17.1.31 "GET /v2.0/networks HTTP/1.1" status: 406 len: 360 time: 0.0057609

It gets weirder. This is only a problem for certain methods.

/v2.0/address-scopes is fine:

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;charset=utf-8" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/address-scopes" | python -mjson.tool
{
    "address_scopes": []
}

/v2.0/ports is no good:

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;charset=utf-8" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/ports" | python -mjson.tool
{
    "NeutronError": {
        "detail": "",
        "message": "The server could not comply with the request since it is either malformed or otherwise incorrect.",
        "type": "HTTPNotAcceptable"
    }
}

/v2.0/flavors is fine:

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;charset=utf-8" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/flavors" | python -mjson.tool
{
    "flavors": []
}

/v2.0/routers is fine:

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;charset=utf-8" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/routers" | python -mjson.tool
{
    "routers": []
}

/v2.0/subnets is a problem again:

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;charset=utf-8" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/subnets" | python -mjson.tool
{
    "NeutronError": {
        "detail": "",
        "message": "The server could not comply with the request since it is either malformed or otherwise incorrect.",
        "type": "HTTPNotAcceptable"
    }
}

And it's definitely a combination of "charset=UTF-8" and those methods. A semicolon followed by "hamburger" is fine.

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;hamburger" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/subnets" | python -mjson.tool | head -5
{
    "subnets": [
        {
            "allocation_pools": [
                {

Using "charset" on it's own is fine:

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;charset" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/subnets" | python -mjson.tool | head -5
{
    "subnets": [
        {
            "allocation_pools": [
                {

Using "hamburger=food" is an error:

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;hamburger=food" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/subnets" | python -mjson.tool | head -5
{
    "NeutronError": {
        "detail": "",
        "message": "The server could not comply with the request since it is either malformed or otherwise incorrect.",
        "type": "HTTPNotAcceptable"

And of course, the original request "charset=UTF-8" is an error:

[stack@undercloud-0 ~]$ curl -s -H "Accept: application/json;charset=UTF-8" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/subnets" | python -mjson.tool | head -5
{
    "NeutronError": {
        "detail": "",
        "message": "The server could not comply with the request since it is either malformed or otherwise incorrect.",
        "type": "HTTPNotAcceptable"

Interestingly enough, reversing the order and saying "charset=UTF-8;application/json" works just fine:

[stack@undercloud-0 ~]$ curl -s -H "Accept: charset=UTF-8;application/json" -H "X-Auth-Token: gAAAAABc0cHuHmvz0YFb0uuPv5ADSxi5Vda_pi-Z8KwIeqjfIu2MtjBLA6NaoRorAjS8HrPzbmrby0K28uajyYj5GiKIJcMCDDYRhN1Hs_2Bfb-4ORbBBj2_X7o2gnybxaBak-TVdrQWdWsHyYnBfTq5CCT-K_mCwPd4KgQfUtoM6_4IUzDPKnQ" "http://192.168.24.1:9696/v2.0/subnets" | python -mjson.tool | head -5
{
    "subnets": [
        {
            "allocation_pools": [
                {

Based on this I think that we can say that:

1. The issue only occurs for certain methods (/v2.0/subnets, /v2.0/networks) but not others.
2. The issue only occurs when there is a semicolon with an equals sign after it.

A workaround is to use separate headers (i.e. `curl -s -H "Accept: application/json" -H "Accept-Charset: UTF-8"`) but the original report of this issue was related to a VNF whose vendor refused to change it's code.