LBaaS TLS with non-admin tenant is user unfriendly
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| Barbican |
Triaged
|
Wishlist
|
Douglas Mendizábal | |
| octavia |
Confirmed
|
High
|
Unassigned |
Bug Description
I went through https:/
If I set my user and tenant as "admin admin", the workflow passed.
But it failed if I set the user and tenant to "admin demo" and rerun all the steps.
Steps to reproduce:
1. source ~/devstack/openrc admin demo
2. barbican secret store --payload-
3. barbican secret store --payload-
4 .barbican secret container create --name=
5. neutron lbaas-loadbalan
6. neutron lbaas-listener-
The error msg I got is
$ neutron lbaas-listener-
WARNING:
DEBUG:stevedore
DEBUG:stevedore
DEBUG:stevedore
DEBUG:stevedore
DEBUG:stevedore
DEBUG:barbicanc
DEBUG:barbicanc
DEBUG:keystonec
INFO:requests.
Starting new HTTP connection (1): 192.168.100.148
DEBUG:requests.
DEBUG:keystonec
INFO:requests.
Starting new HTTP connection (1): 192.168.100.148
DEBUG:requests.
DEBUG:keystonec
RESP BODY: {"versions": {"values": [{"status": "stable", "updated": "2015-04-
DEBUG:keystonec
INFO:requests.
Resetting dropped connection: 192.168.100.148
DEBUG:requests.
DEBUG:keystonec
RESP BODY: {"total": 1, "containers": [{"status": "ACTIVE", "updated": "2016-06-
DEBUG:barbicanc
DEBUG:barbicanc
DEBUG:barbicanc
TLS container http://
Neutron server returns request_ids: ['req-82d53607-
=======
The related barbican-svc log:
2016-06-10 12:25:26.135 INFO barbican.
eb3b 9b07426f96574e2
2016-06-10 12:25:26.137 INFO barbican.
9b07426f96574e
{address space usage: 215629824 bytes/205MB} {rss usage: 100933632 bytes/96MB} [pid: 4671|app: 0|req: 117/117] 192.168.100.149 () {30 v
ars in 465 bytes} [Fri Jun 10 12:25:25 2016] GET /v1/containers?
headers in 172 bytes (1 switches on core 0)
2016-06-10 12:25:28.183 ERROR barbican.
4f00aff0b24f4ea
2016-06-10 12:25:28.183 TRACE barbican.
2016-06-10 12:25:28.183 TRACE barbican.
2016-06-10 12:25:28.183 TRACE barbican.
2016-06-10 12:25:28.183 TRACE barbican.
2699, in one
2016-06-10 12:25:28.183 TRACE barbican.
2016-06-10 12:25:28.183 TRACE barbican.
2016-06-10 12:25:28.183 TRACE barbican.
2016-06-10 12:25:28.184 ERROR barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.184 TRACE barbican.
2016-06-10 12:25:28.187 INFO barbican.
Jiahao liang (jiahao.liang) wrote : | #1 |
Adam Harwell (adam-harwell) wrote : | #2 |
This is actually caused by: https:/
Marked as duplicate, reviving that CR (which should be somewhat close to done).
Jiahao liang (jiahao.liang) wrote : | #3 |
Thank you Adam to confirm that.
Praveen Yalagandula (ypraveen-5) wrote : | #4 |
I ran into this too and this looks like a show-stopper for using barbican. Is there a workaround?
Jiahao liang (jiahao.liang) wrote : | #5 |
Hi guys,
This bug was previously marked as a duplicate of bug #1519170.
However, Lbaas TERMINATED_HTTPS is still not working for non-admin tenant even with bug #1519170 fixed.
I am going to reraise this bug and I assume that bug #1497410 and bug #1612588 will also have the same issue.
My test env is barbican(master), all other components are from stable/mitaka branch
The error I got was:
# source /home/stack/
# neutron lbaas-listener-
http://
TLS container http://
Neutron server returns request_ids: ['req-6116a004-
Related q-svc.log and barbican-svc.log will be attached on next comment.
I made a few breakpoint to traceback the issue and that's where I fount out the last error:
traceback neutron_
traceback neutron_
traceback neutron_
traceback python2.
traceback python2.
traceback python2.
traceback python2.
Processed request: 403 Forbidden - GET http://
=======
I dumped the request and found the X-Auth-Token header is actually the token for admin tenant instead of demo tenant.
Also in /etc/barbican/
I believe some work need to be done either on barbican client or the policy.json.
Jiahao liang (jiahao.liang) wrote : | #6 |
q-svc.log:
2016-09-20 18:16:07.544 25054 WARNING oslo_config.cfg [req-82cbc8b5-
2-b890-
2016-09-20 18:16:08.070 25054 INFO neutron.quota [req-9ca04dda-
2016-09-20 18:16:08.078 25054 DEBUG neutron.
2016-09-20 18:16:08.246 25054 DEBUG barbicanclient.
2016-09-20 18:16:08.247 25054 INFO neutron_
2016-09-20 18:16:08.248 25054 DEBUG barbicanclient.
2016-09-20 18:16:08.645 25054 DEBUG barbicanclient.
Jiahao liang (jiahao.liang) wrote : | #7 |
barbican-svc.log:
2016-09-20 18:16:08.589 4594 INFO barbican.
-b4e5-46e383e212d2 57600ccd1a31429
2016-09-20 18:16:08.643 4594 INFO barbican.
{address space usage: 220184576 bytes/209MB} {rss usage: 105312256 bytes/100MB} [pid: 4594|app: 0|req: 103/103] 192.168.200.43 () {34 vars in 609 bytes} [Tue Sep 20 18:16:08 2016] POST /v1/containers/
2016-09-20 18:16:08.682 4594 INFO barbican.
2016-09-20 18:16:08.684 4594 INFO barbican.
{address space usage: 220184576 bytes/209MB} {rss usage: 105312256 bytes/100MB} [pid: 4594|app: 0|req: 104/104] 192.168.200.43 () {30 vars in 541 bytes} [Tue Sep 20 18:16:08 2016] GET /v1/secrets/
2016-09-20 18:16:08.723 4594 ERROR barbican.
2016-09-20 18:16:08.724 4594 INFO barbican.
{address space usage: 220581888 bytes/210MB} {rss usage: 105684992 bytes/100MB} [pid: 4594|app: 0|req: 105/105] 192.168.200.43 () {30 vars in 551 bytes} [Tue Sep 20 18:16:08 2016] GET /v1/secrets/
2016-09-20 18:16:08.794 4594 INFO barbican.
2016-09-20 18:16:08.835 4594 INFO barbican.
summary: |
- TLS container could not be found + LBaaS TLS is not working with non-admin tenant |
tags: |
added: lbaas removed: lbaasv2 |
Hi,
I'm seeing the same thing as in comment #5 (https:/
Neutron fails with: ERROR neutron.
Neutron log can be found at http://
The barbican log can be found at http://
Using barbican 3.0.0.0rc2.dev5
Kind regards,
Joris
Jiahao liang (jiahao.liang) wrote : | #9 |
I believe this bug is related to two other bugs Octavia/Neutron LBaaS reported.
https:/
https:/
Johannes Grassler (jgr-launchpad) wrote : | #10 |
The problem happens in these three places:
https:/
common/
https:/
The first place is where the user's context is not handed down any more. The second place is where a hardwired admin context comes into play (i.e. the one returned by get_session()). The third link is the function that creates the hardwired context. As you can see from the attached stack trace (excerpted from neutron-lbaas.log), the path to that location is a bit long.
The LBaaS plugin is stuck with whatever credentials it is configured to use. Unless this configuration is changed from the default that's the admin user/admin tenant. For this to work sanely in a multi tenant environment, get_session() would have to return a session for the user who is currently talking to the Neutron API. This would render scary hacks (have the user explicitly grant access to their secret to the admin user) such as the one in the inital post on
https:/
entirely unneccessary.
A fix along those lines is probably a bit longish, since the user context would have to be relayed through every driver in the neutron_
If the fix as outlined is acceptable, I can work on it, but I probably won't get to it until a little after the summit (of course I'd be happy to backport the fix to stable/newton).
Dave McCowan (dave-mccowan) wrote : | #11 |
@Johannes, thanks for the debugging and detailed description. I like your approach, but I'm concerned about backwards compatibility. Your suggestion might be the best solution if we were starting from scratch. Since there might be existing users that count on LBaaS using the configured service or admin user, this change may break them. Maybe we can add a configuration option for Okata? Or maybe this can be a bug fix. Hopefully we can get some discussion going on this. I will bring it up at today's Barbican IRC meeting.
Johannes Grassler (jgr-launchpad) wrote : | #12 |
Sorry, I missed the comment and thus the IRC meeting. I had a look at the log[0] and I'd like to clarify some things, I probably didn't get across all that well:
I'm not proposing to save the user context. I propose to use it in neutron_lbaaas. If a user creates a secret container, and that same user then creates a load balancer, neutron_lbaas can use that user's authenticated context passed from the Neutron API/middleware to neutron_lbaas (rather than the admin user's context) to retrieve the container in question.
That being said, after I've read the log I realized that's not really feasible. It does not allow this kind of access the container long after the authenticated session's token has expired:
20:18:12 <sbalukoff> So, part of the problem here lies in the fact that LBaaS / Octavia may need
to access the secrets at a time when the user isn't actively deploying a load balancer that
requires a secret. That is to say, in a fail-over situation, LBaaS / Octavia will need to access
the secret.
So there is in fact a need to have some sort of long-term access to the container. That can be done without ACLs, though. The "standard" (as in: multiple projects use it) mechanism for this is Keystone trusts. Namely, Heat (see [1] for an in-depth description) and Magnum use these for deferred operations. They're not the ideal least-privilege solution (in my book, that ideal would be a token that only allows access to _one_ specific resource), but they are far more restrictive than granting the Neutron service user access to any or all of a user's secrets.
As for further discussion at the Summit: I'll be there from Tuesday morning until early Friday afternoon, and I'd be happy to meet up for further discussion. I'll give some thought to implementing this in a backwards compatible manner in the mean time.
[0] http://
[1] http://
Changed in octavia: | |
importance: | Undecided → High |
Michael Johnson (johnsom) wrote : | #13 |
At first pass reading through this it seems like the right answer is to use the token received with the "neutron lbaas-listener-
Does this sound right?
Does the barbican ACL API support this use case?
Dave McCowan (dave-mccowan) wrote : | #14 |
@michael, I like the idea, but it would be a change of behavior. This could be config option. barbican_user: (admin or passthrough). When "admin" is set then barbican_admin_user and barbican_
Johannes Grassler (jgr-launchpad) wrote : | #15 |
Using the token received with the "neutron lbaas-listener-
Changed in neutron: | |
status: | New → Confirmed |
importance: | Undecided → High |
Michael Johnson (johnsom) wrote : | #16 |
My proposal was not to store and continue to use the user keystone token, but to use the user keystone token at the time they create the LBaaS listener (provide us the container references) to grant the LBaaS service account ACL access to the conatiner(s) and content.
Using the user token beyond this initial setup would not work due to the expiration issues discussed above.
Subrahmanyam Ongole (osms69) wrote : | #17 |
Would there be any security violations, if we automatically grant ACL access to LBaaS service account on behalf of the user? If we do this, LBaaS account has access to all user secrets.
Stephen Balukoff (sbalukoff) wrote : | #18 |
That would put the control of access to the secrets in the hands of Octavia itself. Michael can speak to whether he thinks this is a good idea, though I don't see anything wrong with it. Note that in order for Octavia to ensure that secrets are not shared across projects, Octavia needs to know the secret's project_id. Presently the barbican API doesn't list the secret's project_id when the meta-data is accessed. I've opened an RFE bug which would solve this problem for us, and allow Octavia (and other 3rd party services) to ensure that secrets are not shared across projects: https:/
Michael Johnson (johnsom) wrote : | #19 |
To my knowledge we can grant ACL access to just the container the user is requesting we use for the listener creation, so we would not be granting the LBaaS service account access to all of the user's secrets, but just the ones that user is requesting we use for the listener.
Is that a mis-understanding?
Changed in octavia: | |
status: | New → Confirmed |
no longer affects: | neutron |
Dave McCowan (dave-mccowan) wrote : | #20 |
@michael: yes. it works this way, except you need to grant access to the container and each secret individually. (Three or four ACL post commands in total.) The only complication is that the user needs to know the UUID of the listener to make the post commands.
Changed in barbican: | |
assignee: | nobody → Douglas Mendizábal (dougmendizabal) |
Dave McCowan (dave-mccowan) wrote : | #21 |
Cascading ACL permissions would make this easier to use.
Changed in barbican: | |
status: | New → Triaged |
importance: | Undecided → Wishlist |
summary: |
- LBaaS TLS is not working with non-admin tenant + LBaaS TLS with non-admin tenant is user unfriendly |
Ahmed Ezzat Douban (doubando) wrote : | #22 |
@Dave, i have the same issue, how do i get the UUID of the listener, is it the ID of the project user used to create the listener ?
thanks in advance
For your reference, my local.conf is as following:
[[local|localrc]]
# The name of the RECLONE environment variable is a bit misleading. It doesn't actually
# reclone repositories, rather it uses git fetch to make sure the repos are current.
RECLONE=True
# Load the external LBaaS plugin.
enable_plugin neutron-lbaas https:/ /git.openstack. org/openstack/ neutron- lbaas stable/mitaka /git.openstack. org/openstack/ octavia stable/mitaka /git.openstack. org/openstack/ barbican stable/mitaka lbaas-dashboard https:/ /git.openstack. org/openstack/ neutron- lbaas-dashboard stable/mitaka
enable_plugin octavia https:/
#enable_plugin octavia /opt/stack/octavia hot-fix
enable_plugin barbican https:/
enable_plugin neutron-
GLANCE_ BRANCH= stable/ mitaka BRANCH= stable/ mitaka BRANCH= stable/ mitaka BRANCH= stable/ mitaka stable/ mitaka BRANCH= stable/ mitaka BRANCH= stable/ mitaka stable/ mitaka BRANCH= stable/ mitaka stable/ mitaka BRANCH= stable/ mitaka
HORIZON_
KEYSTONE_
KESYTONECLIENT_
NOVA_BRANCH=
NOVACLIENT_
NEUTRON_
HEAT_BRANCH=
CEILOMETER_
SWIFT_BRANCH=
CINDER_
LIBS_FROM_ GIT+=python- neutronclient PASSWORD= password password PASSWORD= password TOKEN=password PASSWORD= password
DATABASE_
ADMIN_PASSWORD=
SERVICE_
SERVICE_
RABBIT_
# Enable Logging $DEST/logs/ stack.sh. log LOGDIR= $DEST/logs
LOGFILE=
VERBOSE=True
LOG_COLOR=True
SCREEN_
# Pre-requisites
enable_service rabbit
enable_service mysql
enable_service key
# Horizon
enable_service horizon
# Nova
enable_service n-api
enable_service n-crt
enable_service n-cpu
enable_service n-cond
enable_service n-sch
# Glance
enable_service g-api
enable_service g-reg
# Neutron
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
# Cinder
enable_service c-api
enable_service c-vol
enable_service c-sch
# LBaaS V2 and Octavia MGMT_SUBNET= "192.168. 26.0/24" MGMT_SUBNET_ START=" 192.168. 26.2" MGMT_SUBNET_ END="192. 168.26. 200"
enable_service q-lbaasv2
enable_service octavia
enable_service o-cw
enable_service o-hm
enable_service o-hk
enable_service o-api
OCTAVIA_
OCTAVIA_
OCTAVIA_
# enable DVR
Q_PLUGIN=ml2 NETWORK_ TYPE=vxlan
Q_ML2_TENANT_
Q_DVR_MODE=dvr_snat
IMAGE_URLS+=",http:// download. cirros- cloud.net/ 0.3.4/cirros- 0.3.4-x86_ 64-disk. img"
LOGFILE= $DEST/logs/ stack.sh. log
# Old log files are automatically removed after 7 days to keep things neat. Change
# the number of days by setting ``LOGDAYS``.
LOGDAYS=2