murano-keystonev3-domain-user

[Robert Duncan]

Fuel 6.1 on CentOS 6.5

With keystone v3 configured with multi-domain support, user ids are now 64 characters
when these users try to launch an application from murano catalog we get:
Data too long for column 'user_id' at row 1"

because it is set to VARCHAR(36)

 Policy check succeeded for rule 'get_package' on target {'pa ckage_id': u'4fc0da1b40b648e7b9120ba93f4f356e'}
<134>Sep 16 12:51:34 node-30 murano-api 2015-09-16 12:51:34.297 3697 INFO eventl et.wsgi [-] 127.0.0.1 - - [16/Sep/2015 12:51:34] "GET /v1/catalog/packages/4fc0d a1b40b648e7b9120ba93f4f356e HTTP/1.1" 200 976 0.034964
<134>Sep 16 12:51:34 node-30 murano-api 2015-09-16 12:51:34.308 3697 INFO eventl et.wsgi [-] (3697) accepted ('127.0.0.1', 41312)
<131>Sep 16 12:51:34 node-30 murano-api 2015-09-16 12:51:34.337 3697 ERROR oslo. db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (DataError) (140 6, "Data too long for column 'user_id' at row 1") 'INSERT INTO session (created, updated, id, environment_id, user_id, state, description, version) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)' (datetime.datetime(2015, 9, 16, 12, 51, 34, 335489) , datetime.datetime(2015, 9, 16, 12, 51, 34, 335505), '85206a26a5aa4ec59baf778ad b00b7d3', '3c5b5d6697f044f9ae99e6741e5bd99d', '7d74072841592382362f7288367cbb3bd ce2243223e99c5ed11c02d1050380dd', 'opened', '{"Attributes": [], "Objects": {"def aultNetworks": {"environment": {"name": "quick-env-1-network", "?": {"type": "io .murano.resources.NeutronNetwork", "id": "91200bc59cc841faadd001be5917d61f"}}, " flat": null}, "name": "quick-env-1", "?": {"type": "io.murano.Environment", "id" : "32b248bfcb89429fa37fdbd7cade861a"}}}', 0L)
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters Traceback (mos t recent call last):
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters File "/usr/l ib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_c ontext
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters context)
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters File "/usr/l ib64/python2.6/site-packages/sqlalchemy/engine/default.py", line 436, in do_exec ute
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters cursor.exe cute(statement, parameters)
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters File "/usr/l ib64/python2.6/site-packages/MySQLdb/cursors.py", line 205, in execute
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters self.error handler(self, exc, value)
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters File "/usr/l ib64/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorha ndler
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters raise erro rclass, errorvalue
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters DataError: (14 06, "Data too long for column 'user_id' at row 1")
2015-09-16 12:51:34.337 3697 TRACE oslo.db.sqlalchemy.exc_filters
<134>Sep 16 12:51:34 node-30 murano-api 2015-09-16 12:51:34.344 3697 INFO eventl et.wsgi [-] 127.0.0.1 - - [16/Sep/2015 12:51:34] "POST /v1/environments/3c5b5d66 97f044f9ae99e6741e5bd99d/configure HTTP/1.1" 500 1170 0.034662
<134>Sep 16 12:51:34 node-30 murano-api 2015-09-16 12:51:34.350 3697 INFO eventl et.wsgi [-] (3697) accepted ('127.0.0.1', 41313)
<132>Sep 16 12:51:34 node-30 murano-api 2015-09-16 12:51:34.356 3697 AUDIT muran o.common.policy [-] Policy check succeeded for rule 'delete_environment' on targ et {'environment_id': u'52e48a86bbff4587a554aa5a1be46c11'}
<131>Sep 16 12:51:34 node-30 murano-api 2015-09-16 12:51:34.381 3697 ERROR oslo. db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (DataError) (140 6, "Data too long for column 'user_id' at row 1") 'INSERT INTO session (created, updated, id, environment_id, user_id, state, description, version) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)' (datetime.datetime(2015, 9, 16, 12, 51, 34, 379943) , datetime.datetime(2015, 9, 16, 12, 51, 34, 379959), '04e11c428ebd462981aa989a1 23205eb', '52e48a86bbff4587a554aa5a1be46c11', '7d74072841592382362f7288367cbb3bd ce2243223e99c5ed11c02d1050380dd', 'opened', '{"Attributes": [], "Objects": {"def aultNetworks": {"environment": {"name": "quick-env-2-network", "?": {"type": "io .murano.resources.NeutronNetwork", "id": "634da54e1d2f4337bfc63b67524f5ff0"}}, " flat": null}, "name": "quick-env-2", "?": {"type": "io.murano.Environment", "id" : "d51508e68b944c0293c9809326d9f39d"}}}', 0L)
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters Traceback (mos t recent call last):
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters File "/usr/l ib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_c ontext
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters context)
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters File "/usr/l ib64/python2.6/site-packages/sqlalchemy/engine/default.py", line 436, in do_exec ute
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters cursor.exe cute(statement, parameters)
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters File "/usr/l ib64/python2.6/site-packages/MySQLdb/cursors.py", line 205, in execute
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters self.error handler(self, exc, value)
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters File "/usr/l ib64/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorha ndler
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters raise erro rclass, errorvalue
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters DataError: (14 06, "Data too long for column 'user_id' at row 1")
2015-09-16 12:51:34.381 3697 TRACE oslo.db.sqlalchemy.exc_filters
<134>Sep 16 12:51:34 node-30 murano-api 2015-09-16 12:51:34.388 3697 INFO eventl et.wsgi [-] 127.0.0.1 - - [16/Sep/2015 12:51:34] "DELETE /v1/environments/52e48a 86bbff4587a554aa5a1be46c11 HTTP/1.1" 500 1170 0.036852
[root@node-30 keystone]# "Data too long for column 'user_id' at row 1"

mysql> use murano
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> describe session;
+----------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+-------+
| created | datetime | NO | | NULL | |
| updated | datetime | NO | | NULL | |
| id | varchar(36) | NO | PRI | NULL | |
| environment_id | varchar(255) | YES | MUL | NULL | |
| user_id | varchar(36) | NO | | NULL | |
| state | varchar(36) | NO | | NULL | |
| description | text | NO | | NULL | |
| version | bigint(20) | NO | | NULL | |
+----------------+--------------+------+-----+---------+-------+
8 rows in set (0.00 sec)

bug in upstream: https://bugs.launchpad.net/murano/+bug/1521104

[Robert Duncan (rduncan-t)]

for quick work around ,updating the column makes the error go away and domain users can launch apps from Murano

[Dmytro Dovbii (ddovbii)]

Hi Robert! If you apply suggested workaround, everything works fine?

[Robert Duncan (rduncan-t)]

Yes, simply updating the column to varchar(64) made the error go away. this is true for Horizon and API. tested on Fuel 6.1 Juno and Fuel 7.0 Kilo

here is Kilo:
describe session;

| user_id | varchar(36) | NO | | NULL | |

after the update domain users with a 64bit id can run all environments imported from community app catalog.

thanks,
Robert.

[Robert Duncan (rduncan-t)]

Hi,
correction, because murano uses a versioned url for keystone auth e.g.

# Complete public Identity API endpoint (string value)
#auth_uri=<None>
auth_uri=http://172.25.60.2:5000/v2.0

means domain users cannot use the murano API
murano should support keystone version 3 and use and unversioned auth URL, keystone presents http 300 'multiple choices' on the unversioned endpoint.

also there are no murano commands in the openstack common client such as 'openstack environment list'

thanks,
Rob.

[Kirill Zaitsev (kzaitsev)]

> also there are no murano commands in the openstack common client such as 'openstack environment list'

This is a separate bug/request. python-muranoclient currently does not have a plugin for openstackclien in upstream, so there are no commands for murano available throught openstack command.

see https://blueprints.launchpad.net/python-muranoclient/+spec/openstack-client-plugin-support

[Dmytro Dovbii (ddovbii)]

this bug in upstream: https://bugs.launchpad.net/murano/+bug/1521104

[Dmytro Dovbii (ddovbii)]

will fix it there first

[Vitaly Sedelnik (vsedelnik)]

Won't Fix for 7.0-updates per comment in upstream bug - https://bugs.launchpad.net/murano/+bug/1521104/comments/1

[Robert Duncan (rduncan-t)]

As a wokaround for api clients when using keystone domain specific drivers = true
(i.e domain scoped tokens), create v2.0 and v3 endpoints in the openstack catalog and export run commands:
OS_AUTH_URL=http://keystone:35357/v2.0
&
OS_URL=http://keystone:35357/v3

now we can use openstack common client and murano client

messy but this works when the external keystone domain is in ldap, but probably won't work if it's in OpenID or SAML

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Stan Lagun <email address hidden>
Review: https://review.fuel-infra.org/15592

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Stan Lagun <email address hidden>
Review: https://review.fuel-infra.org/15593

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/15592
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: dc61c69e0e18f04655a1d035bf21dbbee73f4c4b
Author: Stan Lagun <email address hidden>
Date: Mon Dec 28 20:55:28 2015

user_id column was widened to support domain users

The length of keystone domain users is 64 chars length
but the columns that hold user_id were VARCHAR(36)

Change-Id: Ibb167f60cbab3820c8c59c819dbe0e247cbef244
Closes-Bug: #1496389

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Stan Lagun <email address hidden>
Review: https://review.fuel-infra.org/16007

[Roman Podoliaka (rpodolyaka)]

Lowering this to High per https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Confirm_and_triage_bugs

[Kirill Zaitsev (kzaitsev)]

For 7.0 we probably need to add a release not or something similar with explanation that this update requires applying a migration and instructions about how to apply the migration.

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/16007
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: 78af5548a4be0cb9bb82e9ae542694b92d7289a1
Author: Stan Lagun <email address hidden>
Date: Fri Jan 22 15:25:09 2016

Resolve alembic migrations conflict

Makes migration 011 to be a standalone alembic head
to avoid conflicts with different migration 011 that is
present in upstream. This is an addition to commit
Ibb167f60cbab3820c8c59c819dbe0e247cbef244

Change-Id: I1867ed71614bbf745f53b904468a77b397890fbf
Closes-Bug: #1496389

[Anastasia Kuznetsova (akuznetsova)]

Fix was successfully landed and column length was increased
Verified on:
mysql> use murano;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> describe session;
+----------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+-------+
| created | datetime | NO | | NULL | |
| updated | datetime | NO | | NULL | |
| id | varchar(36) | NO | PRI | NULL | |
| environment_id | varchar(255) | YES | MUL | NULL | |
| user_id | varchar(64) | NO | | NULL | |
| state | varchar(36) | NO | | NULL | |
| description | text | NO | | NULL | |
| version | bigint(20) | NO | | NULL | |
+----------------+--------------+------+-----+---------+-------+
8 rows in set (0.02 sec)

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "529"
  build_id: "529"
  fuel-nailgun_sha: "baec8643ca624e52b37873f2dbd511c135d236d9"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "658be72c4b42d3e1436b86ac4567ab914bfb451b"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "e2d79330d5d708796330fac67722c21f85569b87"
  fuel-ostf_sha: "3bc76a63a9e7d195ff34eadc29552f4235fa6c52"
  fuel-mirror_sha: "fb45b80d7bee5899d931f926e5c9512e2b442749"
  fuelmenu_sha: "e071216cb214e34b4d861478033425ee6a54a3be"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "a43cf96cd9532f10794dce736350bf5bed350e9d"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "a365f05b903368225da3fea9aa42afc1d50dc9b4"

[Roman Podoliaka (rpodolyaka)]

Release notes:

When using Keystone V3 API configured with multi-domain support, a launch of a Murano application no longer fails with an error.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 9.0/mitaka
Change author: Stan Lagun <email address hidden>
Review: https://review.fuel-infra.org/18566

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Nikolay Starodubtsev <email address hidden> on branch: 9.0/mitaka
Review: https://review.fuel-infra.org/18578
Reason: Already in 9.0 branch

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Kirill Zaitsev <email address hidden> on branch: 9.0/mitaka
Review: https://review.fuel-infra.org/18566
Reason: In stable mitaka this is done with a 012 migration

[Fuel Devops McRobotson (fuel-devops-robot)]

Change restored by Serg Melikyan <email address hidden> on branch: 9.0/mitaka
Review: https://review.fuel-infra.org/18566

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/18566
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: 9cf8ffd2673973ec7eb36ade69b2bf4bc77125ce
Author: Stan Lagun <email address hidden>
Date: Tue Apr 5 14:04:46 2016

Resolve alembic migrations conflict

Makes migration 011 to be a standalone alembic head
to avoid conflicts with different migration 011 that is
present in upstream. This is an addition to commit
Ibb167f60cbab3820c8c59c819dbe0e247cbef244

Change-Id: I1867ed71614bbf745f53b904468a77b397890fbf
Closes-Bug: #1496389

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Serg Melikyan <email address hidden> on branch: openstack-ci/fuel-7.0/2015.1.0
Review: https://review.fuel-infra.org/15593
Reason: Is not accepted by maintenance team

[Vitaly Sedelnik (vsedelnik)]

Won't Fix for 7.0-updates as we don't accept db schema changes to stable branch.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/newton
Change author: Stan Lagun <email address hidden>
Review: https://review.fuel-infra.org/33557

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 11.0/ocata
Change author: Stan Lagun <email address hidden>
Review: https://review.fuel-infra.org/33982

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/ocata
Change author: Stan Lagun <email address hidden>
Review: https://review.fuel-infra.org/34712

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Roman Podoliaka <email address hidden> on branch: 11.0/ocata
Review: https://review.fuel-infra.org/33982
Reason: we do not need 11.0/ocata anymore - use mcp/ocata instead

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ihor Kalnytskyi <email address hidden> on branch: mcp/ocata
Review: https://review.fuel-infra.org/34712
Reason: This patch was a workaround for upgrading MOS, and is not required for MCP.

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ihor Kalnytskyi <email address hidden> on branch: mcp/newton
Review: https://review.fuel-infra.org/33557
Reason: This patch was a workaround for upgrading MOS, and is not required for MCP.