Baremetal: Invalid credentials

[Jon-Paul Sullivan]

After a successful run of devtest_overcloud.sh it was observed that no nova or keystone commands would work due to credential problems.
root@overcloud-notcompute0-zdcmpw47hksu:~# . ~/stackrc
root@overcloud-notcompute0-zdcmpw47hksu:~# nova list
ERROR: Invalid OpenStack Nova credentials.
root@overcloud-notcompute0-zdcmpw47hksu:~#

The various admin passwords and tokens were listed in the configuration files, but none of these worked. There was uncertainty around the validity of the build as no logs were collected from the run of devtest_overcloud.sh to validate the deployment. Specifically the running of init-keystone correctly could not be verified.

This was performing a cached run (export USE_CACHE=1).

[Robert Collins (lifeless)]

You can check if init_keystone was run by running keystone tenant-list tc with a service_endpoint + service_token vs user credentials.

[Jon-Paul Sullivan (jonpaul-sullivan)]

The system has been torn down now, so no further debug can be gleaned from it.

I think a medium to high priority should be improving the logging of devtest as a result of this.

It would be helpful if the dib logs were saved to disk but not to console, and the devtest.sh subcommands also logged via tee to their own log files. Might make it easier to see what is going on, because at the moment the logs are imho needlessly dominated by pip builds.

[Robert Collins (lifeless)]

redirecting output would make interactive debugging much harder. I don't think thats a good tradeoff.

[jan grant (jan-grant)]

https://review.openstack.org/#/c/83250/ may go a long way to sorting out the confusion over old credential files.

[Ben Nemec (bnemec)]

I'm going to mark this bug won't-fix because the original problem reported is impossible to follow-up on. For the logging discussion a new bug should be opened. There may be some ways we could make the logs easier to consume - say adding easily parseable markers around sections so they can be filtered on. But this is not the place for it. :-)