Don't allow upstream projects to monopolise resources

[Iain Lane]

At the minute we have a large (> 2000 item) distro queue. The items are being processed slowly though, because of the following-

ubuntu@juju-prod-ues-proposed-migration-machine-3:~> nova list
+--------------------------------------+------------------------------------------+--------+------------+-------------+-----------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------------------------------------+--------+------------+-------------+-----------------------------------------+
| 069799a4-8f67-4b4e-bdf9-24ff30853645 | adt-artful-amd64-snapd-20170518-151451 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.72 |
| afc56bde-96be-4861-a2e7-b313b6698b7e | adt-artful-amd64-snapd-20170518-152516 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.83 |
| 8029ed39-2ee8-413b-aa86-a4200ac6d0be | adt-artful-amd64-snapd-20170518-152657 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.85 |
| c4bdfe21-2506-4169-93c6-52a82ff68185 | adt-artful-amd64-snapd-20170518-154019 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.97 |
| ab7a818c-0485-42e0-84c2-027664617de4 | adt-artful-i386-binutils-20170518-132701 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.3 |
| 97776bda-2652-4c3c-9522-19a2ac8885bf | adt-artful-i386-linux-20170518-153335 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.82 |
| 2d043848-1940-445e-88f2-9cf5cc67e89c | adt-xenial-amd64-snapd-20170518-151534 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.73 |
| bb4137eb-37f1-4c75-9ef6-84bdaba3663f | adt-xenial-amd64-snapd-20170518-153950 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.101 |
| 5f29277c-d6f6-4046-8047-f4c5b54ba3e7 | adt-xenial-amd64-snapd-20170518-154932 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.98 |
| c7a1c02f-3bd1-40f6-bc4d-88fa6d6a449c | adt-yakkety-amd64-snapd-20170518-145708 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.92 |
| 9908fc7d-1d66-4ef6-9750-05cf147a9206 | adt-yakkety-amd64-snapd-20170518-155016 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.99 |
| 8083cf47-b927-4523-b5d0-0172c1b42e22 | adt-yakkety-amd64-snapd-20170518-155058 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.100 |
| 6e0a3b49-9f68-4b21-8622-a71d4ca5fffc | adt-yakkety-amd64-snapd-20170518-155416 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.102 |
| 8f602c54-9a88-4f6a-90ad-118029bdf526 | adt-zesty-amd64-snapd-20170518-152804 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.86 |
| 2e701ea0-658b-4876-a33c-3c4c6a156a47 | adt-zesty-amd64-snapd-20170518-154322 | ACTIVE | - | Running | net_ues_proposed_migration=10.42.41.103 |
+--------------------------------------+------------------------------------------+--------+------------+-------------+-----------------------------------------+
ubuntu@juju-prod-ues-proposed-migration-machine-3:~> . lgw01.rc
ubuntu@juju-prod-ues-proposed-migration-machine-3:~> nova list
+--------------------------------------+------------------------------------------------------+--------+------------+-------------+------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------------------------------------------------+--------+------------+-------------+------------------------------------------+
| d5f2e40f-a760-4928-8e6d-2e6f7f7f7476 | adt-artful-amd64-linux-20170518-113404 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.126 |
| 939e8cfe-e321-4835-b335-0d0ba8c5e1f7 | adt-artful-amd64-linux-20170518-150022 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.110 |
| a0ae6af7-b87c-4985-904a-edd051fb42e9 | adt-artful-amd64-snapd-20170518-150914 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.118 |
| 9cc6a86d-8522-40b4-b96d-6b7aa9c85e8a | adt-artful-i386-libkdegames-20170518-153632 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.127 |
| 9f9d742e-d946-4e40-af95-c0ed01582d7e | adt-artful-i386-libkeduvocdocument-20170518-154052 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.130 |
| 142c1cdc-edf9-4703-a8c5-c815456c1b31 | adt-artful-i386-r-bioc-annotationdbi-20170518-144326 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.104 |
| 4fe944f9-b2d5-4045-b675-b7e8cc4e8375 | adt-xenial-amd64-snapd-20170518-152128 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.124 |
| 74b75361-8586-4fdc-a9de-15d8efaef0b7 | adt-xenial-i386-snapd-20170518-134609 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.94 |
| bd831c3a-9285-4370-92d8-da871cdc4469 | adt-xenial-i386-snapd-20170518-143152 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.105 |
| 609b20a5-1f7d-44e8-b0c1-a9c6b817ea76 | adt-yakkety-amd64-snapd-20170518-150801 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.119 |
| 68ba861d-8ad8-48e4-bd1e-b192557d2244 | adt-yakkety-amd64-snapd-20170518-154912 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.129 |
| 832f259d-5f0a-4aa5-8b27-761a69bced42 | adt-zesty-amd64-snapd-20170518-140157 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.100 |
| 90512bee-10c8-4838-b3e5-8d5c56a00890 | adt-zesty-amd64-snapd-20170518-151441 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.121 |
| 680ac016-b6d2-489d-8a9d-6adb07b359c2 | adt-zesty-amd64-snapd-20170518-151922 | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.123 |
| b9eb663c-ff7b-4dd9-b7e0-770239c508ad | laney-test | ACTIVE | - | Running | net_ues_proposed_migration=10.220.44.65 |
+--------------------------------------+------------------------------------------------------+--------+------------+-------------+------------------------------------------+

snapd is taking up the majority of our quota.

I think we should consider limiting the amount of concurrent tests any one project can run.

[Iain Lane (laney)]

A related issue is that in the case of repeated pushes to a PRed branch, we'll request tests for each commit without removing the old ones.

[Iain Lane (laney)]

I pushed a change which should help with this, but isn't a fundamental solution

  https://git.launchpad.net/autopkgtest-cloud/commit/?id=8682e173e4a26449a6095afc2786b45938d5ae04