Jujud needs an acceptance test set that covers "normal" database sizes and write counts.

[Mark Shuttleworth]

We' just dug into #1344940 which was filed after observing a huge database with large numbers of unnecessary writes. This bug is to ask the QA team to ensure that we have automated acceptance tests that bound the size of database and number of writes and reads expected for a set of normal spinups and teardowns and quiet periods.

I would ask that we design 5 test environments (bundles) covering a range of scales; small environments, large but simple environments, and large and complex environments.

I would ask that on a regular (weekly?) basis we:

 * spin up those environments on local, EC2, OpenStack and MAAS, and measure:
      - database size before deploy, after 5 minutes, after 30 minutes, and then for a further 30 minutes of quiescent time
      - total number of database reads and writes in each of those
      - total number of API calls in each period
      - time till quiescent (i.e. time to deploy)

 * automate this in a way that it can be reproduced by any developer (or at least one with n machines handy)
     - therefore make it possible to spin up on any substrate (MAAS, EC2, OpenStack, local)

 * set thresholds that would trigger a high priority bug and report to the list weekly on current results

Thanks,
Mark

[Torsten Baumann (torbaumann)]

Core is not addressing nor specifying what is "normal" However we will take these ideas as part of the scale initiative