Cannot schedule instance on the 2nd compute node

[Anshuman Manral]

Hi

I have installed one controller and one compute node in my setup using devstack.

Both of the nodes seem to have installed properly. Pasted below are the nova.conf output for the controller and compute node. You can see that in compute node configuration all servers are correctly pointing to controller (10.10.25.136). Multi host is enabled on both. The scheduler is set to SimpleScheduler as mentioned in the devstack documentation.

The follwoing output also shows the controller can see the compute node properly:
root@compute-node-1:~# nova-manage service list
Binary Host Zone Status State Updated_At
nova-cert compute-node-1.ipinfusion.com nova enabled :-) 2012-12-03 22:47:53
nova-scheduler compute-node-1.ipinfusion.com nova enabled :-) 2012-12-03 22:47:53
nova-consoleauth compute-node-1.ipinfusion.com nova enabled :-) 2012-12-03 22:47:53
nova-conductor compute-node-1.ipinfusion.com nova enabled :-) 2012-12-03 22:47:52
nova-compute compute-node-1.ipinfusion.com nova enabled :-) 2012-12-03 22:47:51
nova-compute compute-node-3.ipinfusion.com nova enabled :-) 2012-12-03 22:47:55 ====> other compute node seen

(compute-node-1 (10.10.25.136) is controller and compute-node-3 (10.10.25.140) is the extra compute node)

However when I configure a new instance in nova, it is always getting created only on the compute node which is part of the controller. The instances never get created on the compute only node. Attached file shows the output for two instances. Same is case for all instances

Kindly let me know if everything looks ok, its a bug or some changes are required.

Regards
Anshuman

nova.conf for controller:
=======================
[DEFAULT]
verbose=True
auth_strategy=keystone
allow_resize_to_same_host=True
api_paste_config=/etc/nova/api-paste.ini
rootwrap_config=/etc/nova/rootwrap.conf
compute_scheduler_driver=nova.scheduler.simple.SimpleScheduler
dhcpbridge_flagfile=/etc/nova/nova.conf
force_dhcp_release=True
fixed_range=10.0.0.0/24
s3_host=10.10.25.136
s3_port=3333
osapi_compute_extension=nova.api.openstack.compute.contrib.standard_extensions
my_ip=10.10.25.136
sql_connection=mysql://root:openstack@localhost/nova?charset=utf8
libvirt_type=kvm
libvirt_cpu_mode=none
instance_name_template=instance-%08x
enabled_apis=ec2,osapi_compute,metadata
volume_api_class=nova.volume.cinder.API
state_path=/opt/stack/data/nova
lock_path=/opt/stack/data/nova
instances_path=/opt/stack/data/nova/instances
multi_host=True
send_arp_for_ha=True
logging_context_format_string=%(asctime)s %(color)s%(levelname)s %(name)s [%(request_id)s %(user_name)s %(project_name)s%(color)s] %(instance)s%(color)s%(message)s
logging_default_format_string=%(asctime)s %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s
logging_debug_format_suffix=from (pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d
logging_exception_prefix=%(color)s%(asctime)s TRACE %(name)s %(instance)s
network_api_class=nova.network.quantumv2.api.API
quantum_admin_username=quantum
quantum_admin_password=openstack
quantum_admin_auth_url=http://10.10.25.136:35357/v2.0
quantum_auth_strategy=keystone
quantum_admin_tenant_name=service
quantum_url=http://localhost:9696
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
linuxnet_interface_driver=
novncproxy_base_url=http://10.10.25.136:6080/vnc_auto.html
xvpvncproxy_base_url=http://10.10.25.136:6081/console
vncserver_listen=127.0.0.1
vncserver_proxyclient_address=127.0.0.1
ec2_dmz_host=10.10.25.136
rabbit_host=localhost
rabbit_password=openstack
glance_api_servers=10.10.25.136:9292
compute_driver=libvirt.LibvirtDriver
firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver

nov.conf for compute node
=======================
[DEFAULT]
verbose=True
auth_strategy=keystone
allow_resize_to_same_host=True
api_paste_config=/etc/nova/api-paste.ini
rootwrap_config=/etc/nova/rootwrap.conf
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler
dhcpbridge_flagfile=/etc/nova/nova.conf
force_dhcp_release=True
fixed_range=10.0.0.0/24
s3_host=10.10.25.136
s3_port=3333
osapi_compute_extension=nova.api.openstack.compute.contrib.standard_extensions
my_ip=10.10.25.140
sql_connection=mysql://root:openstack@10.10.25.136/nova?charset=utf8
libvirt_type=kvm
libvirt_cpu_mode=none
instance_name_template=instance-%08x
enabled_apis=ec2,osapi_compute,metadata
state_path=/opt/stack/data/nova
lock_path=/opt/stack/data/nova
instances_path=/opt/stack/data/nova/instances
multi_host=True
send_arp_for_ha=True
logging_context_format_string=%(asctime)s %(color)s%(levelname)s %(name)s [%(request_id)s %(user_name)s %(project_name)s%(color)s] %(instance)s%(color)s%(message)s
logging_default_format_string=%(asctime)s %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s
logging_debug_format_suffix=from (pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d
logging_exception_prefix=%(color)s%(asctime)s TRACE %(name)s %(instance)s
network_api_class=nova.network.quantumv2.api.API
quantum_admin_username=quantum
quantum_admin_password=openstack
quantum_admin_auth_url=http://10.10.25.136:35357/v2.0
quantum_auth_strategy=keystone
quantum_admin_tenant_name=service
quantum_url=http://10.10.25.136:9696
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
linuxnet_interface_driver=
novncproxy_base_url=http://10.10.25.136:6080/vnc_auto.html
xvpvncproxy_base_url=http://10.10.25.136:6081/console
vncserver_listen=127.0.0.1
vncserver_proxyclient_address=127.0.0.1
ec2_dmz_host=10.10.25.136
rabbit_host=10.10.25.136
rabbit_password=openstack
glance_api_servers=10.10.25.136:9292
compute_driver=libvirt.LibvirtDriver
firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver

[Yaguang Tang (heut2008)]

if you use multi_host mode , you need to running nova-compute,nova-network nova-api(metadata api) on the compute node .

[Anshuman Manral (anshuman-manral)]

Hi

I guess we can use quantum instead of nova-network here? My controller with compute node is working fine with quantum

Regards
Anshuman

[Anshuman Manral (anshuman-manral)]

Hi Yaguang

My localrc file looks like this on the compute node:

ENABLED_SERVICES=n-cpu,quantum,q-agt,n-api,n-vol,rabbit
SERVICE_HOST=10.10.25.136
MULTI_HOST=1
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
Q_HOST=$SERVICE_HOST

So I have the n-cpu and n-api enabled and using quantum instead of nova-network

But still we can see only the compute service coming up.

Infact I can see that the nova-api and quantum-agt are both running fine on the compute node:

root@compute-node-3:~# ps wax | grep nova
 4168 pts/1 S+ 0:00 python /opt/stack/nova/bin/nova-api
 4173 pts/1 S+ 0:00 python /opt/stack/nova/bin/nova-api
 4174 pts/1 S+ 0:00 python /opt/stack/nova/bin/nova-api
 4175 pts/1 S+ 0:00 python /opt/stack/nova/bin/nova-api
 5859 pts/6 S+ 0:00 grep --color=auto nova
 8285 pts/5 S+ 0:00 sg libvirtd /opt/stack/nova/bin/nova-compute
 8300 pts/5 Sl+ 34:55 python /opt/stack/nova/bin/nova-compute
root@compute-node-3:~# ps wax | grep quantum
 5976 pts/6 S+ 0:00 grep --color=auto quantum
27312 pts/4 S+ 1:28 python /opt/stack/quantum/bin/quantum-openvswitch-agent --config-file /etc/quantum/quantum.conf --config-file /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini

Please let me know what may be going wrong here

Regards
Anshuman

[Robbie Harwood (rharwood)]

This is working for me with compute's conf:

[[local|localrc]]
MULTI_HOST=True
DATABASE_PASSWORD=$ADMIN_PASSWORD
DATABASE_TYPE=mysql
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=token
ENABLED_SERVICES=n-cpu,n-acpi,c-vol,rabbit,neutron,q-agt
SERVICE_HOST=$CONTROLLER
DATABASE_HOST=$SERVICE_HOST
MYSQL_HOST=$DATABASE_HOST
RABBIT_HOST=$SERVICE_HOST
Q_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=${SERVICE_HOST}:9292
Q_PLUGIN=ml2

If you have further issues with getting multinode to work, you might have better luck on IRC than a bug report (I can help some, but you're probably better off in #openstack).