Nova DB archiving not working in rh1

[Ben Nemec]

[root@overcloud-controller-0 mysql]# nova-manage db archive_deleted_rows --verbose --max_rows 100000
Option "notification_topics" from group "DEFAULT" is deprecated. Use option "topics" from group "oslo_messaging_notifications".
error: (pymysql.err.IntegrityError) (1062, u"Duplicate entry '1' for key 'PRIMARY'") [SQL: u'INSERT INTO shadow_instance_type_extra_specs (created_at, updated_at, deleted_at, deleted, id, `key`, value, instance_type_id) SELECT instance_type_extra_specs.created_at, instance_type_extra_specs.updated_at, instance_type_extra_specs.deleted_at, instance_type_extra_specs.deleted, instance_type_extra_specs.id, instance_type_extra_specs.`key`, instance_type_extra_specs.value, instance_type_extra_specs.instance_type_id \nFROM instance_type_extra_specs \nWHERE instance_type_extra_specs.deleted != %s ORDER BY instance_type_extra_specs.id \n LIMIT %s'] [parameters: (0, 99582)]

This appears to be the problem melwitt describes in https://lists.gt.net/openstack/operators/63495?page=last I will see about cleaning up the problematic shadow entries.

I believe this is causing extreme slowness in the cloud and is probably contributing to the large number of timeouts that we're seeing.

[Ben Nemec (bnemec)]

Fixed by running "delete from shadow_instance_type_extra_specs;" against the nova db. I'm going to leave this open until I've finished archiving the nova db so that hopefully performance in rh1 will be back to normal.

[Ben Nemec (bnemec)]

DB is cleaned up, rh1 performance seems to be better again. There were other issues (namely rabbit) in addition to this that also had to be resolved but they have been.