Comment 3 for bug 1209421

I spent a few hours looking into this one; adding my 2 cents below..

I do agree with Daniel's assessment above in that the hypervisor support matrix (https://wiki.openstack.org/wiki/HypervisorSupportMatrix) can / should not be generated via unit test based analysis and that likely the correct place for such automated analysis is tempest / live tests or similar. In fact I don't think the compute driver UT methods are enough to determine / build the full hypervisor matrix even if we wanted to.

Note -- I suggest if we agree the UT class should not generate the hyper matrix that we at least remove the comments in test_virt_drivers.py which indicates it does for clarity sake.

However I do believe some potentially useful information can be gleamed on a per hypervisor basis using static analysis. As part of my investigation of this bug, I implemented a quick python script which effectively does the following:

- Analyzes the nova ComputeDriver (base) class and determines the list of methods required for the driver "contract". Additionally methods are marked as required or optional based on if the base driver raises NotImplementedError or just does a pass signifying optional.
- Analyzes the hypervisor virt driver (concrete) driver implementations (classes) and determines which methods the virt driver class implements.
- Outputs a matrix which contains the compute driver methods and their implemented status within the virt driver.

For example see the output from the vmwareapi compute driver here: http://paste.openstack.org/show/112285/

This is fairly low level information at the driver method level, however IMO it provides information which I was unable to extract using UT coverage or any other means.

That said I propose the following:
[1] I will submit a patch to remove the comments in test_virt_drivers which indicates it generates a matrix and also remove the dead code which tracks the not implemented errors.
[2] I will start an email thread to determine if there's any interest / resources to work on a means to generate the hyper matrix outside the UTs (e.g. tempest or elsewhere).
[3] I can clean up my simple static analysis script and make it avail on github if anyone thinks its useful?

Thoughts from others are appreciated.