Nova sends an "X-Service-Token" header when "send_service_user_token" is disabled

[Lana Kaleif]

Description
===========
In the interaction between nova-api and cinder, it is possible to enable the required check of service_user tokens.
When I try to explicitly turn off the sending of the user’s service token and enable the mandatory check of its availability on the receiving side, I do not get the expected error because the X-Service-Token header is still sent by nova-api.

Steps to reproduce
==================
cinder includes required token checking:

[keystone_authtoken]
...
service_token_roles = admin
service_token_roles_required = true

in nova, token sending is explicitly disabled and the user service is not set:

[service_user]
send_service_user_token = false

verification is performed on the example of the operation of volume attach:
openstack server add volume 0801102f-f9ba-42c5-b32a-85b4a7465122 a7fd514c-c871-4f69-a89f-bd44864a5814 --device /dev/vdb

Expected result
===============
with this configuration, error 401 is expected

Actual result
=============
no errors occur and the attach operation is successful.
multiple checks were made including the option to completely restart the servers

Environment
===========
CentOS 7
release: train
nova: 15.1.0
cinder: 5.0.0

Logs & Configs
==============
we intercept requests that go to the cinder port (8776) and we see that 192.168.50.81.49226 (which is the nova-api process) sends requests with the X-Service-Token header (which we previously disabled in nova.conf). Full log (req/res) of adding volume in attachment.

[root@centos ~]# tcpdump -i enp2s0f0 -n -S -s 1024 -A 'tcp dst port 8776'

06:20:21.870330 IP 192.168.50.81.49226 > 192.168.50.80.8776: Flags [P.], seq 1696694971:1696695677, ack 922604522, win 58, options [nop,nop,TS val 3747722 ecr 3644290], length 706
E....5@.@.....2Q..2P.J"He!..6......:.......
.9/..7..GET /v3/27c47772a3f442e4a81decb6b68e8376/volumes/a7fd514c-c871-4f69-a89f-bd44864a5814 HTTP/1.1
Host: 192.168.50.80:8776
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: application/json
User-Agent: python-cinderclient
X-Service-Token: gAAAAABeMrv1SxDwllx8lrDYiP6o0kMlr1hJu34q2N7WgTcsQ15GXU7s2EDG91DX0XOUz0YTS9Q-_nxuRJ5lbZfWDBgk2spJr9csJ1VhNWQhrZUZWhkBk3KMU6GwgC0X3kO5o4dIw41rj1VmH4TfIdV9bSEWjA3qMLjVYKdryyUzhCN1gZQISl0
X-Auth-Token: gAAAAABeMrv0vSJ6P8pLF7BDLimZ_LxhvKYCvEzWk_TZ62hIaDNnlFHRMx1rrIvDNl6Z78mv6C-EYWRqvURHCP6x9FCuhd16-i25gO4AUVi2Qo2-N-wpxO6dnbOHuuI6G8gtYu3ZPMuaHkn3uk-aaf7zh3PZ__OQy4mtZir6Qt7b5xqPb4QJcGg
X-OpenStack-Request-ID: req-a4ad6f8b-3f49-44d0-99b4-b7fa026017c6

06:20:22.184182 IP 192.168.50.81.49226 > 192.168.50.80.8776: Flags [.], ack 922606483, win 65, options [nop,nop,TS val 3748036 ecr 3644604], length 0
E..4.6@.@.....2Q..2P.J"He!.}6......A.......
.90..7..
06:20:22.547486 IP 192.168.50.81.49226 > 192.168.50.80.8776: Flags [P.], seq 1696695677:1696696271, ack 922606483, win 65, options [nop,nop,TS val 3748399 ecr 3644604], length 594
E....7@.@..H..2Q..2P.J"He!.}6......AG......
.92/.7..GET / HTTP/1.1
Host: 192.168.50.80:8776
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: nova-api keystoneauth1/3.17.1 python-requests/2.21.0 CPython/2.7.5
X-Service-Token: gAAAAABeMrv1SxDwllx8lrDYiP6o0kMlr1hJu34q2N7WgTcsQ15GXU7s2EDG91DX0XOUz0YTS9Q-_nxuRJ5lbZfWDBgk2spJr9csJ1VhNWQhrZUZWhkBk3KMU6GwgC0X3kO5o4dIw41rj1VmH4TfIdV9bSEWjA3qMLjVYKdryyUzhCN1gZQISl0
X-Auth-Token: gAAAAABeMrv0vSJ6P8pLF7BDLimZ_LxhvKYCvEzWk_TZ62hIaDNnlFHRMx1rrIvDNl6Z78mv6C-EYWRqvURHCP6x9FCuhd16-i25gO4AUVi2Qo2-N-wpxO6dnbOHuuI6G8gtYu3ZPMuaHkn3uk-aaf7zh3PZ__OQy4mtZir6Qt7b5xqPb4QJcGg

06:20:22.553939 IP 192.168.50.81.49226 > 192.168.50.80.8776: Flags [.], ack 922607474, win 71, options [nop,nop,TS val 3748405 ecr 3644974], length 0
E..4.8@.@.....2Q..2P.J"He!..6..r...G.......
.925.7..
06:20:22.564940 IP 192.168.50.81.49226 > 192.168.50.80.8776: Flags [P.], seq 1696696271:1696697181, ack 922607474, win 71, options [nop,nop,TS val 3748416 ecr 3644974], length 910
E....9@.@..
..2Q..2P.J"He!..6..r...G.......
.92@.7..POST /v3/27c47772a3f442e4a81decb6b68e8376/attachments HTTP/1.1
Host: 192.168.50.80:8776
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: application/json
User-Agent: python-cinderclient
X-Service-Token: gAAAAABeMrv1SxDwllx8lrDYiP6o0kMlr1hJu34q2N7WgTcsQ15GXU7s2EDG91DX0XOUz0YTS9Q-_nxuRJ5lbZfWDBgk2spJr9csJ1VhNWQhrZUZWhkBk3KMU6GwgC0X3kO5o4dIw41rj1VmH4TfIdV9bSEWjA3qMLjVYKdryyUzhCN1gZQISl0
X-Auth-Token: gAAAAABeMrv0vSJ6P8pLF7BDLimZ_LxhvKYCvEzWk_TZ62hIaDNnlFHRMx1rrIvDNl6Z78mv6C-EYWRqvURHCP6x9FCuhd16-i25gO4AUVi2Qo2-N-wpxO6dnbOHuuI6G8gtYu3ZPMuaHkn3uk-aaf7zh3PZ__OQy4mtZir6Qt7b5xqPb4QJcGg
OpenStack-API-Version: volume 3.44
Content-Type: application/json
X-OpenStack-Request-ID: req-a4ad6f8b-3f49-44d0-99b4-b7fa026017c6
Content-Length: 147

[Balazs Gibizer (balazs-gibizer)]

I was able to reproduce the problem on current master. Regardless of the setting of send_service_user_token nova sends the service token.

with [service_user]/send_service_user_token = false

08:03:59.263531 IP aio.56768 > aio.http: Flags [P.], seq 3136635724:3136636434, ack 282222251, win 512, options [nop,nop,TS val 548278802 ecr 548278802], length 710: HTTP: GET /volume/v3/57abc7a114c341708f5dea1a65716e2d/volumes/f122d57f-8273-48f3-a2fb-5335fcd15b3f HTTP/1.1
E.....@.@..!..y...y....P..CL..^.....w@.....
 ... ...GET /volume/v3/57abc7a114c341708f5dea1a65716e2d/volumes/f122d57f-8273-48f3-a2fb-5335fcd15b3f HTTP/1.1
Host: 192.168.121.129
User-Agent: python-cinderclient
Accept-Encoding: gzip, deflate
Accept: application/json
Connection: keep-alive
X-OpenStack-Request-ID: req-e57eaa48-ff8e-4e50-a6fc-a054b899dd9d
X-Auth-Token: gAAAAABemBFu99R0nCxAoorJJxf0PhYq-siT8WrygdGzYg50Wg73kimG5khjP5FaUm4Un2yIcpw0NJtsNpuy5gVT7yT6T33AY9ZJNJh7JyKWoKxH3r4R16QpTsWISL54ctrYycQOd1X2AeMmk_rP7GvsxvihytwtWNLOXKJBf2rrXbbLl0C5X-I
X-Service-Token: gAAAAABemBFv2zK0OJ1IJe6GDPu7fyJEmGsm8P82L2N2Zek8BA5J0EF1v4iZsH2ljWXqC-r1-zF4tMf_QOmoBdD2o9rFgpuaoWlWwj0IyvgNlkc0nv8tn0RbGmpei00Lf17-CMsfOTIxkrXF7JUikDZGwvTNXzJ5CEycYDIUjqcDkbN9-6BsEx8

with [service_user]/send_service_user_token = true

09:02:06.437197 IP aio.57272 > aio.http: Flags [P.], seq 2567770992:2567771702, ack 287109253, win 512, options [nop,nop,TS val 551765985 ecr 551765985], length 710: HTTP: GET /volume/v3/57abc7a114c341708f5dea1a65716e2d/volumes/f122d57f-8273-48f3-a2fb-5335fcd15b3f HTTP/1.1
E...r&@.@.Q...y...y....P...p........w@.....
 .G. .G.GET /volume/v3/57abc7a114c341708f5dea1a65716e2d/volumes/f122d57f-8273-48f3-a2fb-5335fcd15b3f HTTP/1.1
Host: 192.168.121.129
User-Agent: python-cinderclient
Accept-Encoding: gzip, deflate
Accept: application/json
Connection: keep-alive
X-OpenStack-Request-ID: req-a77e52e6-b623-42c0-a6ec-e18691a1a4bc
X-Auth-Token: gAAAAABemB8N2IX9lI3ECL_zksQz1egSMytu4mG6vnWExq1HQBIFN4O60hEJAgLLXOpCgpKDKX4VmVCDl4qMgt8aBShGrC4eYmcjctIOuUbXlY9TzgUxZAs8z6bKgDq5OLbEgSgnCUIQAtvnY2f7vrmvFa2HAucckXv7JUbYNhU4UY15cFAR3U8
X-Service-Token: gAAAAABemB8OypdEvu4g3DJRerexgHEoiaOwHXPV6wgGY10iG3RFbFbENqcV1DNnFVUveX07i5ZltABJqx0EXfb4Jl1AsYzBjUS2zxFEKX8-niStTnEMsqpvQ2yfH9TPnldxh2Sl4Y9WgxlgrvQLaPMXMLe7loIJhhuj3HTkKAuGsmyouifOThI