Automation is required to check not only for successful deployment, but also that puppet log doesn't contain warnings.
In ISO #180 I see the following warning messages, which must be fixed as part of this bug as well:
2014-02-23 22:12:22 WARNING
tag is a metaparam; this value will inherit to all contained resources
2014-02-23 22:12:22 NOTICE
(Scope(Class[L23network::Hosts_file])) Generating host entry node-7 192.168.0.3 node-7.domain.tld
2014-02-23 22:12:22 NOTICE
(Scope(Class[L23network::Hosts_file])) Generating host entry node-7 192.168.0.3 node-7.domain.tld
2014-02-23 22:12:22 NOTICE
(Scope(Class[L23network::Hosts_file])) Generating host entry node-5 192.168.0.2 node-5.domain.tld
2014-02-23 22:12:16 WARNING
You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 37 in file /etc/puppet/modules/nova/manifests/db/mysql.pp
2014-02-23 22:12:16 WARNING
You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 255 in file /etc/puppet/modules/nova/manifests/init.pp
2014-02-23 22:12:16 WARNING
You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 234 in file /etc/puppet/modules/nova/manifests/init.pp
2014-02-23 22:12:16 WARNING
You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 224 in file /etc/puppet/modules/nova/manifests/init.pp
2014-02-23 22:12:14 WARNING
You cannot collect without storeconfigs being set on line 131 in file /etc/puppet/modules/openstack/manifests/nova/controller.pp
2014-02-23 22:12:14 WARNING
You cannot collect without storeconfigs being set on line 128 in file /etc/puppet/modules/openstack/manifests/nova/controller.pp
2014-02-23 22:12:14 WARNING
You cannot collect without storeconfigs being set on line 125 in file /etc/puppet/modules/openstack/manifests/nova/controller.pp
2014-02-23 22:12:13 WARNING
(Scope(Class[Osnailyfacter::Apache_api_proxy])) Could not look up qualified variable 'osnailyfacter::params::apache_site_dir'; class osnailyfacter::params could not be found at /etc/puppet/modules/osnailyfacter/manifests/apache_api_proxy.pp:4
2014-02-23 22:12:13 WARNING
(Scope(Class[Osnailyfacter::Apache_api_proxy])) Could not look up qualified variable 'osnailyfacter::params::apache_site_dir'; class osnailyfacter::params could not be found at /etc/puppet/modules/osnailyfacter/manifests/apache_api_proxy.pp:4
Yes, some of these warnings are realy nasty and should be somehow loged and viewed by developers.
We could just grep logs for warning keyword and output it somewhere if test log.