Comment 0 for bug 1496747

Revision history for this message
Szymon Datko (sdatko) wrote :

I was having fun with Cinder, when I have spotted a bug in a mechanism that is being used to grant access for projects (aka 'tenants') to volume types. To be more specific: the problem is related to removing project access.

How to reproduce the bug:

1. In a loop, about 150 times:
a) create non-public volume type ( is_public=False )
b) grant access for project to newly created type
2. Try to remove project access to one of recently created type
3. You can expect from python-cinderclient an error 500 (~ Server is incapable of performing operation)
4. On Cinder's log there is an error: "Out of range value for column 'deleted' at row 1"

Packages' versions on my host:

cinder-api 1:2015.1.1-0ubuntu2~cloud2
cinder-common 1:2015.1.1-0ubuntu2~cloud2
cinder-scheduler 1:2015.1.1-0ubuntu2~cloud2
cinder-volume 1:2015.1.1-0ubuntu2~cloud2
python-cinder 1:2015.1.1-0ubuntu2~cloud2
python-cinderclient 1:1.3.1-2
python-oslo-db 1.7.1-0ubuntu2~cloud0

Description:

The problem is related to database - column 'deleted' on table 'volume_type_projects'.

Here you can see that it is created as *Boolean* type:
https://github.com/openstack/cinder/blob/master/cinder/db/sqlalchemy/migrate_repo/versions/032_add_volume_type_projects.py#L36

During conversion process, the result of db-sync in database is type *tinyint(1)*:
mysql> desc volume_type_projects;
+----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| deleted_at | datetime | YES | | NULL | |
| volume_type_id | varchar(36) | YES | MUL | NULL | |
| project_id | varchar(255) | YES | | NULL | |
| deleted | tinyint(1) | YES | | NULL | |
+----------------+--------------+------+-----+---------+----------------+

Type *tinyint(1)* is, in fact, a reason why the bug does not occur before 128 access rules.

Now, what in fact causes bug is usage of soft_delete() function during access removing. See it here:
https://github.com/openstack/cinder/blob/master/cinder/db/sqlalchemy/api.py#L2776

The mentioned function is definied in *oslo.db* as:
https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/orm.py#L28

So, what happens when project access is removed? To the column 'deleted' in table 'volume_type_projects' the *row id* is assigned instead of *Boolean* value. Due to conversion from *Boolean* to *tinyint(1)* on db-sync, the problem is not spotted on fresh installation.

Possible solutions:

a) replace soft_delete() function with just update() - like it is done in rest of the code, see example here:
https://github.com/openstack/cinder/blob/master/cinder/db/sqlalchemy/api.py#L2710

b) upgrade the db schema