In cases of that the magnum api or keystone are busy we should try to contact them again to fetch the certificates.
http://git.openstack.org/cgit/openstack/magnum/tree/magnum/drivers/common/templates/kubernetes/fragments/make-cert.sh#n98
Bug watches keep track of this bug in other bug trackers.