devtest_ scripts often assume they are running as root

[vlowther]

I am working on adding support to tripleo-incubator enable wrapping devtest.sh in a Docker container to make it easier to do local testing without trashing your dev box, and potentially to speed up the Jenkins test gates.

Right now, devtest.sh gets as far as trying to bring up the undercloud and failing.

[vlowther (victor-lowther)]

set -x trace from failing code snippet:

/opt/openstack/tripleo-incubator/scripts/devtest_undercloud.sh@153: heat stack-create -f /opt/openstack/tripleo-heat-emplates/undercloud-vm.yaml -P AdminToken=6b63304c07871b180854edec71e79dc722a68f5f -P AdminPassword=8fe8abffd18561e71c6eeed2c212e452ef57aac -P CeilometerPassword=4c15304308e82cb19ec0a9f56e59da75556faca9 -P GlancePassword=f4f4b4e2821ba88c282f0fb3dc012745dc1c725 -P HeatPassword=e4df8db92525886359b1244b9fba74a8856a5684 -P NeutronPassword=72e2bac494a4038c2efab28fce0a2b2537941b0 -P NovaPassword=371e01e6d1c10f4e7cd9963da15018991a97fad0 -P BaremetalArch=i386 -P underclouImage=0c7dcbb6-70de-4032-ae1c-bcc97734aa43 -P 'PowerSSHPrivateKey=-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA2JsKe01JdRGRAw3ZRfeRZgrZBa80kZwkWgl91WEO3KQXCHnN
8HeJWxgeIvnsK0w1LzN4EN0W7mMNNr8O7RXNTlHgJHZFGOrkqYKDbCTNyzOK82M3
yccobgA5zlfO1A+fsjuaaRilhiQxETVbJb41+pOch682b7rvRGcPahuhuJAAHcw6
y1/pbIXucL4Oe86XqOmp3hM6JGxXl6qlOeuiu4dV1hTO/ttIUqr5ZlSim1obmjFF
/yvQuD675agsdSeZLbLqwVMNkaaRbGMOSRIecu5bo7szwfQu8LaaQPL9YpJm4xR9
lSWJKeFWD3oGGPQATHuiz3AlBp7JPiXRa9yE0wIBIwKCAQEAs3kl8SLH1gc9o3Hf
6YP8IVlx/WVI0Gtuk764uB04Myjf4nONYNgSuTE9irHST8LM+zlGOdvCiwGdO/1k
I4cX0y3lmo3hewvExvccb4w1fHsp/N0uOXkhf7cLUzLP8YlJzjFirtnoO/IaBvGx
7BKhwP32uZEtGr91yvZOmb8fmOrRSpS4LcUQ/Q/mPZqpR3etHJS5p269zWaUR6E7
ZWVDuAUZlc4/9MTncBqOLQsSGoFpM2koxOOjim3yeHi8vppnBCh54wmATNRe14ra
Ay8vcCNmEwh+U/G31zFWdVSeUNimmERmKtQOfhK4gCRUjPF3eZsmuM1LZh9pcbKJ
PS18gwKBgQD1sa7yO6wjsnuidz08zvlBFNKx4SuqT9biRTVfCk+mzHc7MhGqFM0Y
JA6ktK9a/HmTsVM5na/3ekuZhuLn5WJnEUZi7UARNyW2hUDsKOk3mo9Hpwds1MeH
Qk1Tl61TULKKKfP4lB5L+PC/5kdM3FMCNcAyUjS2fzoUhUNn/se5HQKBgQDhsP/d
CQO6DQjDVuCkROygIiWt/d1+nuSwtWM8S03n/1CR1LrGEhe3wcu6wedd6NGWLzvS
WWBI1Kw7hsDKso9DBcAYCK8+Mr5WyINhoLr9CmlSLFoT+p7+xS27AS78p3lIoEC6
QBj2vBfJJJzyFvBTaXvGK2/M6NTdL6nVcrHCrwKBgQCFYHTp5eEasVkPDYekqt8U
s4hR70OW+CQxsIqv76eVD+j0P8ByRc5sMNS/z8zoP+o6O7DWImbWzVw9ZnstbeT2
H1IYcioX+V2dmMt44wJCwZ481wQH4TkdimR91f8BV7Fg8jP7+KK7hyObX8BOS7gI
g5Q4kwasNm/8gtty+AYFZwKBgBnLFe1gHa7c6w8CnVSaKavmpTh0yNtFW/bwKJkt
LXmZhY0CXn0JYcvbokE6u1swjPs4mR9aq+sQ/b2pABcqWYQAqD1C0jL+fCcs29Ck
p6fj7seQCkttGXwzyrZJRzLRT7CHV9rFfzIkH/m7Cp9TFCbKOggE9tWW88GBykRH
n0lzAoGBAOCVVINZscPKEm8+RDUcx3D/8EvlE1jPD2usIC7IFpJpWgU66BALePwz
8bp2i0a4F1/gujLUnrhzS0EUiulsp3V12kQv9LnsPATtQReQCWMgHkbqe0jZwUyz
fOzMBRwZ0UkFbSh4l8N4pTuiq9U05WQKe3+TX7kZZNQMhXFJB0rx
-----END RSA PRIVATE KEY-----' -P NeutronPublicInterface=eth0 -P PowerSSHHost=192.168.122.1 -P PowerManager=nova.virtbaremetal.virtual_power_driver.VirtualPowerManager -P PowerUserName=openstack undercloud
+--------------------------------------+------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+--------------------+----------------------+
| 4a09478f-4c2a-4ec2-84b8-38e0aa2e6f1f | undercloud | CREATE_IN_PROGRESS | 2014-04-29T14:22:25Z |
+--------------------------------------+------------+--------------------+----------------------+
/opt/openstack/tripleo-incubator/scripts/devtest_undercloud.sh@175: echo 'Waiting for the undercloud stack to be read'
Waiting for the undercloud stack to be ready
/op...

[vlowther (victor-lowther)]

nova-compute logs from seed vm

[vlowther (victor-lowther)]

libvirt nodes along with mac addresses in the Docker container:

bash-4.1# while read node; do printf "%s: " "$node"; virsh dumpxml "$node" |grep "mac address"; done < <(virsh list --
all | tail -n +2 | awk -F" " '{print $2}')
: seed: <mac address='52:54:00:2a:2b:40'/>
      <mac address='00:8c:8b:a7:77:8f'/>
baremetal_0: <mac address='00:27:dc:38:cd:93'/>
baremetal_1: <mac address='00:b2:64:5b:ba:56'/>
baremetal_2: <mac address='00:b9:6b:35:4c:73'/>
baremetal_3: <mac address='00:f7:74:35:39:e1'/>

[vlowther (victor-lowther)]

tripleo-incubator with Docker as a dev/test env support: https://github.com/VictorLowther/tripleo-incubator

To use the docker container, make sure you have the openvswitch kernel module installed, and run scripts/docker_deploy.sh (ubuntu|fedora|opensuse)

docker_deploy runs everything in a centos 6.5 container due to init system hilariousness. There are bugfixes to other scripts and templates to enable everything to function.

[vlowther (victor-lowther)]

nova show for the offending node:

root@ubuntu:~# nova show undercloud-undercloud-flrf2sb6lwkd
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | seed |
| OS-EXT-SRV-ATTR:hypervisor_hostname | c4fec417-f25f-4da9-833a-f637984a7829 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2014-04-29T14:22:29Z |
| fault | {"message": "Node undercloud-undercloud-flrf2sb6lwkd could not be found.", "code": 404, "created": "2014-04-29T14:23:13Z"} |
| flavor | baremetal (dd10146a-f4f5-434a-93b6-01f7c7441735) ...

[vlowther (victor-lowther)]

Root cause is that devtest.sh and friends assume they are running as root some of the time, and I am not doing that in Docker to make it easier to use it as a dev env. Patch that fixes this assumption forthcoming.

[James Polley (tchaypo)]

Victor, I've assigned this to you since I believe you're working on fixing this.

I believe this is still something you're actively working on, so we provably want to keep this open?

My personal feeling is that this is more of a feature request than a bug - I think I'd like to see it designed in a spec rather than being addressed here.

[Brent Eagles (beagles)]

Where we don't rely on the devtest scripts any longer and the significance of this bug reported is probably mitigated by the quickstart capability I'm going to mark as incomplete. I containerizing the bootstrapping environment for the undercloud is something that interests somebody, a blueprint/spec is more approriate. Marking as Won'tFix as the "world has moved on"