Comment 3 for bug 1542892

We actually have this problem with all optional API extensions. When deciding if to run a test you either skip by checking if the API extension is loaded via 'neutron ext-list', or by checking the current tempest.conf. The Tempest way is the second option, and by default Devstack enables all neutron extensions in tempest.conf. This means that if you install Neutron without any advanced services, QoS, ml2 port security etc, and you use the default tempest.conf that comes from Devstack, a ton of tests will fail instead of be skipped.