Cannot provide list of IDs to filter snapshots

[Avishay Traeger]

If I pass the filter {'id': [id1, id2, id3]} to db/sqlalchemy/api.py:volume_get_all(), it works as expected - returning only volumes whose ID is in the list. The same filter does not work for snapshot_get_all(), even though the code looks almost identical. Here is the stack trace (the error message in that mess is "Operand should contain 1 column(s)"):

2016-02-28 15:45:06.460 ERROR oslo_messaging._drivers.common [req-760caf24-40cd-45d8-96d7-12e660edf12a demo] ['Traceback (most recent call last
):\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n incoming.messag
e))\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch\n return self._do_dispatch
(endpoint, method, ctxt, args)\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
\n result = func(ctxt, **new_args)\n', ' File "/opt/stack/cinder/cinder/volume/manager.py", line 3538, in get_manageable_snapshots\n ret
urn self._get_manageable_resources(ctxt, \'snapshot\')\n', ' File "/opt/stack/cinder/cinder/volume/manager.py", line 2308, in _get_manageable_
resources\n LOG.exception(_LE("Listing manageable volumes failed, due to "\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_utils/ex
cutils.py", line 220, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 1
96, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/opt/stack/cinder/cinder/volume/manager.py", line 2301, in
_get_manageable_resources\n ctxt, filters={\'id\': potential_ids})\n', ' File "/opt/stack/cinder/cinder/db/api.py", line 284, in snapshot_g
et_all\n sort_dirs, offset)\n', ' File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 175, in wrapper\n return f(*args, **kwargs)
\n', ' File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 2140, in snapshot_get_all\n return query.all()\n', ' File "/usr/local/li
b/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2588, in all\n return list(self)\n', ' File "/usr/local/lib/python2.7/dist-package
s/sqlalchemy/orm/query.py", line 2736, in __iter__\n return self._execute_and_instances(context)\n', ' File "/usr/local/lib/python2.7/dist-
packages/sqlalchemy/orm/query.py", line 2751, in _execute_and_instances\n result = conn.execute(querycontext.statement, self._params)\n', '
 File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute\n return meth(self, multiparams, params)\n',
' File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection\n return connection._execut
e_clauseelement(self, multiparams, params)\n', ' File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execu
te_clauseelement\n compiled_sql, distilled_params\n', ' File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146,
 in _execute_context\n context)\n', ' File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_
exception\n util.raise_from_cause(newraise, exc_info)\n', ' File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 2
00, in raise_from_cause\n reraise(type(exception), exception, tb=exc_tb, cause=cause)\n', ' File "/usr/local/lib/python2.7/dist-packages/sq
lalchemy/engine/base.py", line 1139, in _execute_context\n context)\n', ' File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/de
fault.py", line 450, in do_execute\n cursor.execute(statement, parameters)\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in execute\n result = self._query(query)\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query\n conn.query(q)\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 819, in query\n self._affected_rows = self._read_query_result(unbuffered=unbuffered)\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1001, in _read_query_result\n result.read()\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1285, in read\n first_packet = self.connection._read_packet()\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 965, in _read_packet\n packet.check_error()\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 394, in check_error\n err.raise_mysql_exception(self._data)\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception\n _check_mysql_exception(errinfo)\n', ' File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 115, in _check_mysql_exception\n raise InternalError(errno, errorvalue)\n', "DBError: (pymysql.err.InternalError) (1241, u'Operand should contain 1 column(s)') [SQL: u'SELECT snapshots.created_at AS snapshots_created_at, snapshots.updated_at AS snapshots_updated_at, snapshots.deleted_at AS snapshots_deleted_at, snapshots.deleted AS snapshots_deleted, snapshots.id AS snapshots_id, snapshots.user_id AS snapshots_user_id, snapshots.project_id AS snapshots_project_id, snapshots.volume_id AS snapshots_volume_id, snapshots.cgsnapshot_id AS snapshots_cgsnapshot_id, snapshots.status AS snapshots_status, snapshots.progress AS snapshots_progress, snapshots.volume_size AS snapshots_volume_size, snapshots.display_name AS snapshots_display_name, snapshots.display_description AS snapshots_display_description, snapshots.encryption_key_id AS snapshots_encryption_key_id, snapshots.volume_type_id AS snapshots_volume_type_id, snapshots.provider_location AS snapshots_provider_location, snapshots.provider_id AS snapshots_provider_id, snapshots.provider_auth AS snapshots_provider_auth, snapshot_metadata_1.created_at AS snapshot_metadata_1_created_at, snapshot_metadata_1.updated_at AS snapshot_metadata_1_updated_at, snapshot_metadata_1.deleted_at AS snapshot_metadata_1_deleted_at, snapshot_metadata_1.deleted AS snapshot_metadata_1_deleted, snapshot_metadata_1.id AS snapshot_metadata_1_id, snapshot_metadata_1.`key` AS snapshot_metadata_1_key, snapshot_metadata_1.value AS snapshot_metadata_1_value, snapshot_metadata_1.snapshot_id AS snapshot_metadata_1_snapshot_id \\nFROM snapshots LEFT OUTER JOIN snapshot_metadata AS snapshot_metadata_1 ON snapshot_metadata_1.snapshot_id = snapshots.id AND snapshot_metadata_1.deleted = false \\nWHERE snapshots.deleted = false AND snapshots.id = %(id_1)s ORDER BY snapshots.created_at DESC, snapshots.id DESC'] [parameters: {u'id_1': ['05f459c4-bc19-45f6-b98f-33ccdc2e659e', '29ae5f66-a11e-4eae-abb1-1a1556381824']}]\n"]

[Avishay Traeger (avishay-il)]

For reference, the query generated for volumes:

SELECT volumes.created_at AS
volumes_created_at, volumes.updated_at AS volumes_updated_at, volumes.deleted_at AS volumes_deleted_at, volumes.deleted AS volumes_deleted, vol
umes.id AS volumes_id, volumes._name_id AS volumes__name_id, volumes.ec2_id AS volumes_ec2_id, volumes.user_id AS volumes_user_id, volumes.proj
ect_id AS volumes_project_id, volumes.snapshot_id AS volumes_snapshot_id, volumes.host AS volumes_host, volumes.size AS volumes_size, volumes.a
vailability_zone AS volumes_availability_zone, volumes.status AS volumes_status, volumes.attach_status AS volumes_attach_status, volumes.migrat
ion_status AS volumes_migration_status, volumes.scheduled_at AS volumes_scheduled_at, volumes.launched_at AS volumes_launched_at, volumes.termi
nated_at AS volumes_terminated_at, volumes.display_name AS volumes_display_name, volumes.display_description AS volumes_display_description, vo
lumes.provider_location AS volumes_provider_location, volumes.provider_auth AS volumes_provider_auth, volumes.provider_geometry AS volumes_prov
ider_geometry, volumes.provider_id AS volumes_provider_id, volumes.volume_type_id AS volumes_volume_type_id, volumes.source_volid AS volumes_so
urce_volid, volumes.encryption_key_id AS volumes_encryption_key_id, volumes.consistencygroup_id AS volumes_consistencygroup_id, volumes.bootabl
e AS volumes_bootable, volumes.multiattach AS volumes_multiattach, volumes.replication_status AS volumes_replication_status, volumes.replicatio
n_extended_status AS volumes_replication_extended_status, volumes.replication_driver_data AS volumes_replication_driver_data, volumes.previous_
status AS volumes_previous_status, consistencygroups_1.created_at AS consistencygroups_1_created_at, consistencygroups_1.updated_at AS consiste
ncygroups_1_updated_at, consistencygroups_1.deleted_at AS consistencygroups_1_deleted_at, consistencygroups_1.deleted AS consistencygroups_1_de
leted, consistencygroups_1.id AS consistencygroups_1_id, consistencygroups_1.user_id AS consistencygroups_1_user_id, consistencygroups_1.projec
t_id AS consistencygroups_1_project_id, consistencygroups_1.host AS consistencygroups_1_host, consistencygroups_1.availability_zone AS consiste
ncygroups_1_availability_zone, consistencygroups_1.name AS consistencygroups_1_name, consistencygroups_1.description AS consistencygroups_1_des
cription, consistencygroups_1.volume_type_id AS consistencygroups_1_volume_type_id, consistencygroups_1.status AS consistencygroups_1_status, c
onsistencygroups_1.cgsnapshot_id AS consistencygroups_1_cgsnapshot_id, consistencygroups_1.source_cgid AS consistencygroups_1_source_cgid, volu
me_attachment_1.created_at AS volume_attachment_1_created_at, volume_attachment_1.updated_at AS volume_attachment_1_updated_at, volume_attachme
nt_1.deleted_at AS volume_attachment_1_deleted_at, volume_attachment_1.deleted AS volume_attachment_1_deleted, volume_attachment_1.id AS volume
_attachment_1_id, volume_attachment_1.volume_id AS volume_attachment_1_volume_id, volume_attachment_1.instance_uuid AS volume_attachment_1_inst
ance_uuid, volume_attachment_1.attached_host AS volume_attachment_1_attached_host, volume_attach...

[zhangguoqing (474751729-o)]

vim /usr/lib/python2.7/site-packages/cinder/db//sqlalchemy/api.py

2137 import pdb;pdb.set_trace()
2138 filters = {'id': ['49fcc84c-79aa-42ab-84af-d1128eb7c69d', '887154ce-134d-483d-ac79-391afe986d41', '49fcc84c-79aa-42ab-84af-d1128eb7c69e']}

cinder snapshot-list --all-tenant

diff str(query) [the operator '=' and 'IN']
 VOLUME:
WHERE volumes.deleted = 0 AND volumes.id IN (:id_1, :id_2, :id_3) ORDER BY volumes.created_at DESC, volumes.id DESC
SNAPSHOT:
WHERE snapshots.deleted = 0 AND snapshots.id = :id_1 ORDER BY snapshots.created_at DESC, snapshots.id DESC

[zhangguoqing (474751729-o)]

Hi Avishay Traeger,
     I have fix it by hard coding the filters, but I don't know your command line to filter a list of snapshot-ids?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/292298

[OpenStack Infra (hudson-openstack)]

Change abandoned by Eric Harney (<email address hidden>) on branch: master
Review: https://review.openstack.org/292298
Reason: No update since March. Please re-submit if you would like to continue work on this patch. Thanks!

[Sean McGinnis (sean-mcginnis)]

Unassigning due to no activity for > 6 months.