Activity log for bug #1008978

Date Who What changed Old value New value Message
2012-06-05 12:35:11 wuzhenle bug added bug
2012-06-07 17:49:43 Russell Bryant nova: status New Incomplete
2012-06-08 03:51:20 wuzhenle description Hello: i deploy openstak with multi node. control node: rabbitmq; keystone; mysql; horizon; nova-api nova-scheduler nova-novncproxy nova-consoleauth and glance network node: nove-network (flatdhcp) nova-api (only metadata service) compute node: nova-compute After i launch an instance,i try to view the log of vm through horizon.But it take nearly 20 seconds ....then i try again by cli command (nova console-log XXX) and get the same result. this is api.log in control node 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] GET http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/3 2012-06-05 19:53:52 DEBUG nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Unrecognized Content-Type provided in request from (pid=6335) get_body /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/wsgi.py:697 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] HTTP exception thrown: The resource could not be found. 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/3 returned with HTTP 404 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] GET http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/detail 2012-06-05 19:53:52 DEBUG nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Unrecognized Content-Type provided in request from (pid=6335) get_body /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/wsgi.py:697 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'deleted': False, 'project_id': u'6cd20a72ec4a455a8648d7728975f32e'} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Generated ACTIVE from vm_state=active task_state=None. from (pid=6335) status_from_state /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:96 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Converting nw_info: [VIF({'network': Network({'bridge': u'br100', 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'version': 4, 'type': u'fixed', 'floating_ips': [], 'address': u'10.168.3.153'})], 'version': 4, 'meta': {u'dhcp_server': u'10.168.3.114'}, 'dns': [IP({'meta': {}, 'version': 4, 'type': u'dns', 'address': u'8.8.4.4'})], 'routes': [], 'cidr': u'10.168.0.0/19', 'gateway': IP({'meta': {}, 'version': 4, 'type': u'gateway', 'address': u'10.168.3.114'})}), Subnet({'ips': [], 'version': None, 'meta': {u'dhcp_server': None}, 'dns': [], 'routes': [], 'cidr': None, 'gateway': IP({'meta': {}, 'version': None, 'type': u'gateway', 'address': None})})], 'meta': {u'tenant_id': None, u'should_create_bridge': True, u'bridge_interface': u'eth0'}, 'id': u'b4041965-63b7-45e0-a506-6cfbc1386fb6', 'label': u'private'}), 'meta': {}, 'id': u'2a2fd9f4-0dcc-47e6-a440-49b6caf10632', 'address': u'fa:16:3e:03:4a:55'})] from (pid=6335) get_networks_for_instance_from_nw_info /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:291 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Converted networks: {u'private': {'ips': [FixedIP({'meta': {}, 'version': 4, 'type': u'fixed', 'floating_ips': [], 'address': u'10.168.3.153'})], 'floating_ips': []}} from (pid=6335) get_networks_for_instance_from_nw_info /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:301 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'uuid': [u'2e6c8b3a-961b-48de-a568-b8ea799661e4']} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'uuid': [u'2e6c8b3a-961b-48de-a568-b8ea799661e4']} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/detail returned with HTTP 200 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] POST http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/2e6c8b3a-961b-48de-a568-b8ea799661e4/action 2012-06-05 19:53:53 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Making asynchronous call on compute.nb447.corp.yodao.com ... from (pid=6335) multicall /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/amqp.py:321 2012-06-05 19:53:53 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] MSG_ID is 060a9fe9ff474762959dcc24bff14eb4 from (pid=6335) multicall /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/amqp.py:324 2012-06-05 19:54:09 INFO nova.api.openstack.wsgi [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/2e6c8b3a-961b-48de-a568-b8ea799661e4/action returned with HTTP 200 this the compute.log in compute node 2012-06-05 19:54:02 DEBUG nova.manager [-] Running periodic task ComputeManager._poll_unconfirmed_resizes from (pid=15794) periodic_tasks /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/manager.py:152 2012-06-05 19:54:02 DEBUG nova.rpc.amqp [-] received {u'_context_roles': [u'KeystoneServiceAdmin', u'admin', u'KeystoneAdmin'], u'_msg_id': u'060a9fe9ff474762959dcc24bff14eb4', u'_context_read_deleted': u'no', u'_context_request_id': u'req-1f6e439b-5de7-41af-a847-e7471cfc7ec4', u'args': {u'instance_uuid': u'2e6c8b3a-961b-48de-a568-b8ea799661e4', u'tail_length': None}, u'_context_auth_token': '<SANITIZED>', u'_context_is_admin': True, u'_context_project_id': u'6cd20a72ec4a455a8648d7728975f32e', u'_context_timestamp': u'2012-06-05T11:53:52.961339', u'_context_user_id': u'ac148194334b475e8820334b156a532b', u'method': u'get_console_output', u'_context_remote_address': u'10.168.3.113'} from (pid=15794) _safe_log /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/common.py:160 2012-06-05 19:54:02 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] unpacked context: {'user_id': u'ac148194334b475e8820334b156a532b', 'roles': [u'KeystoneServiceAdmin', u'admin', u'KeystoneAdmin'], 'timestamp': '2012-06-05T11:53:52.961339', 'auth_token': '<SANITIZED>', 'remote_address': u'10.168.3.113', 'is_admin': True, 'request_id': u'req-1f6e439b-5de7-41af-a847-e7471cfc7ec4', 'project_id': u'6cd20a72ec4a455a8648d7728975f32e', 'read_deleted': u'no'} from (pid=15794) _safe_log /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/common.py:160 2012-06-05 19:54:02 AUDIT nova.compute.manager [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Get console output for instance 2e6c8b3a-961b-48de-a568-b8ea799661e4 2012-06-05 19:54:08 DEBUG nova.utils [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Running cmd (subprocess): sudo chown 2006 /var/lib/nova/instances/instance-00000017/console.log from (pid=15794) execute /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/utils.py:219s what was worse, it may appear the timeot error.like this bug https://bugs.launchpad.net/nova/+bug/966767 Hello:     i deploy openstak with multi node.     control node: rabbitmq; keystone; mysql; horizon; nova-api nova-scheduler nova-novncproxy nova-consoleauth and glance     network node: nove-network (flatdhcp) nova-api (only metadata service)     compute node: nova-compute     After i launch an instance,i try to view the log of vm through horizon.But it take nearly 20 seconds ....then i try again by cli command (nova console-log XXX) and get the same result.     this is api.log in control node 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] GET http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/3 2012-06-05 19:53:52 DEBUG nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Unrecognized Content-Type provided in request from (pid=6335) get_body /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/wsgi.py:697 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] HTTP exception thrown: The resource could not be found. 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/3 returned with HTTP 404 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] GET http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/detail 2012-06-05 19:53:52 DEBUG nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Unrecognized Content-Type provided in request from (pid=6335) get_body /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/wsgi.py:697 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'deleted': False, 'project_id': u'6cd20a72ec4a455a8648d7728975f32e'} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Generated ACTIVE from vm_state=active task_state=None. from (pid=6335) status_from_state /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:96 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Converting nw_info: [VIF({'network': Network({'bridge': u'br100', 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'version': 4, 'type': u'fixed', 'floating_ips': [], 'address': u'10.168.3.153'})], 'version': 4, 'meta': {u'dhcp_server': u'10.168.3.114'}, 'dns': [IP({'meta': {}, 'version': 4, 'type': u'dns', 'address': u'8.8.4.4'})], 'routes': [], 'cidr': u'10.168.0.0/19', 'gateway': IP({'meta': {}, 'version': 4, 'type': u'gateway', 'address': u'10.168.3.114'})}), Subnet({'ips': [], 'version': None, 'meta': {u'dhcp_server': None}, 'dns': [], 'routes': [], 'cidr': None, 'gateway': IP({'meta': {}, 'version': None, 'type': u'gateway', 'address': None})})], 'meta': {u'tenant_id': None, u'should_create_bridge': True, u'bridge_interface': u'eth0'}, 'id': u'b4041965-63b7-45e0-a506-6cfbc1386fb6', 'label': u'private'}), 'meta': {}, 'id': u'2a2fd9f4-0dcc-47e6-a440-49b6caf10632', 'address': u'fa:16:3e:03:4a:55'})] from (pid=6335) get_networks_for_instance_from_nw_info /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:291 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Converted networks: {u'private': {'ips': [FixedIP({'meta': {}, 'version': 4, 'type': u'fixed', 'floating_ips': [], 'address': u'10.168.3.153'})], 'floating_ips': []}} from (pid=6335) get_networks_for_instance_from_nw_info /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:301 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'uuid': [u'2e6c8b3a-961b-48de-a568-b8ea799661e4']} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'uuid': [u'2e6c8b3a-961b-48de-a568-b8ea799661e4']} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/detail returned with HTTP 200 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] POST http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/2e6c8b3a-961b-48de-a568-b8ea799661e4/action 2012-06-05 19:53:53 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Making asynchronous call on compute ... from (pid=6335) multicall /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/amqp.py:321 2012-06-05 19:53:53 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] MSG_ID is 060a9fe9ff474762959dcc24bff14eb4 from (pid=6335) multicall /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/amqp.py:324 2012-06-05 19:54:09 INFO nova.api.openstack.wsgi [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/2e6c8b3a-961b-48de-a568-b8ea799661e4/action returned with HTTP 200 this the compute.log in compute node 2012-06-05 19:54:02 DEBUG nova.manager [-] Running periodic task ComputeManager._poll_unconfirmed_resizes from (pid=15794) periodic_tasks /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/manager.py:152 2012-06-05 19:54:02 DEBUG nova.rpc.amqp [-] received {u'_context_roles': [u'KeystoneServiceAdmin', u'admin', u'KeystoneAdmin'], u'_msg_id': u'060a9fe9ff474762959dcc24bff14eb4', u'_context_read_deleted': u'no', u'_context_request_id': u'req-1f6e439b-5de7-41af-a847-e7471cfc7ec4', u'args': {u'instance_uuid': u'2e6c8b3a-961b-48de-a568-b8ea799661e4', u'tail_length': None}, u'_context_auth_token': '<SANITIZED>', u'_context_is_admin': True, u'_context_project_id': u'6cd20a72ec4a455a8648d7728975f32e', u'_context_timestamp': u'2012-06-05T11:53:52.961339', u'_context_user_id': u'ac148194334b475e8820334b156a532b', u'method': u'get_console_output', u'_context_remote_address': u'10.168.3.113'} from (pid=15794) _safe_log /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/common.py:160 2012-06-05 19:54:02 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] unpacked context: {'user_id': u'ac148194334b475e8820334b156a532b', 'roles': [u'KeystoneServiceAdmin', u'admin', u'KeystoneAdmin'], 'timestamp': '2012-06-05T11:53:52.961339', 'auth_token': '<SANITIZED>', 'remote_address': u'10.168.3.113', 'is_admin': True, 'request_id': u'req-1f6e439b-5de7-41af-a847-e7471cfc7ec4', 'project_id': u'6cd20a72ec4a455a8648d7728975f32e', 'read_deleted': u'no'} from (pid=15794) _safe_log /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/common.py:160 2012-06-05 19:54:02 AUDIT nova.compute.manager [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Get console output for instance 2e6c8b3a-961b-48de-a568-b8ea799661e4 2012-06-05 19:54:08 DEBUG nova.utils [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Running cmd (subprocess): sudo chown 2006 /var/lib/nova/instances/instance-00000017/console.log from (pid=15794) execute /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/utils.py:219s  what was worse, it may appear the timeot error.like this bug https://bugs.launchpad.net/nova/+bug/966767
2012-06-08 08:32:20 wuzhenle attachment added api.log https://bugs.launchpad.net/nova/+bug/1008978/+attachment/3180495/+files/api-log.png
2012-06-08 08:32:55 wuzhenle attachment added compute.log https://bugs.launchpad.net/nova/+bug/1008978/+attachment/3180496/+files/compute-log.png
2012-06-08 08:34:11 wuzhenle description Hello:     i deploy openstak with multi node.     control node: rabbitmq; keystone; mysql; horizon; nova-api nova-scheduler nova-novncproxy nova-consoleauth and glance     network node: nove-network (flatdhcp) nova-api (only metadata service)     compute node: nova-compute     After i launch an instance,i try to view the log of vm through horizon.But it take nearly 20 seconds ....then i try again by cli command (nova console-log XXX) and get the same result.     this is api.log in control node 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] GET http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/3 2012-06-05 19:53:52 DEBUG nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Unrecognized Content-Type provided in request from (pid=6335) get_body /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/wsgi.py:697 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] HTTP exception thrown: The resource could not be found. 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-2269fc51-b05c-4c36-bc1c-a6a37928da84 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/3 returned with HTTP 404 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] GET http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/detail 2012-06-05 19:53:52 DEBUG nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Unrecognized Content-Type provided in request from (pid=6335) get_body /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/wsgi.py:697 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'deleted': False, 'project_id': u'6cd20a72ec4a455a8648d7728975f32e'} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Generated ACTIVE from vm_state=active task_state=None. from (pid=6335) status_from_state /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:96 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Converting nw_info: [VIF({'network': Network({'bridge': u'br100', 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'version': 4, 'type': u'fixed', 'floating_ips': [], 'address': u'10.168.3.153'})], 'version': 4, 'meta': {u'dhcp_server': u'10.168.3.114'}, 'dns': [IP({'meta': {}, 'version': 4, 'type': u'dns', 'address': u'8.8.4.4'})], 'routes': [], 'cidr': u'10.168.0.0/19', 'gateway': IP({'meta': {}, 'version': 4, 'type': u'gateway', 'address': u'10.168.3.114'})}), Subnet({'ips': [], 'version': None, 'meta': {u'dhcp_server': None}, 'dns': [], 'routes': [], 'cidr': None, 'gateway': IP({'meta': {}, 'version': None, 'type': u'gateway', 'address': None})})], 'meta': {u'tenant_id': None, u'should_create_bridge': True, u'bridge_interface': u'eth0'}, 'id': u'b4041965-63b7-45e0-a506-6cfbc1386fb6', 'label': u'private'}), 'meta': {}, 'id': u'2a2fd9f4-0dcc-47e6-a440-49b6caf10632', 'address': u'fa:16:3e:03:4a:55'})] from (pid=6335) get_networks_for_instance_from_nw_info /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:291 2012-06-05 19:53:52 DEBUG nova.api.openstack.common [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Converted networks: {u'private': {'ips': [FixedIP({'meta': {}, 'version': 4, 'type': u'fixed', 'floating_ips': [], 'address': u'10.168.3.153'})], 'floating_ips': []}} from (pid=6335) get_networks_for_instance_from_nw_info /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/api/openstack/common.py:301 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'uuid': [u'2e6c8b3a-961b-48de-a568-b8ea799661e4']} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 DEBUG nova.compute.api [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Searching by: {'uuid': [u'2e6c8b3a-961b-48de-a568-b8ea799661e4']} from (pid=6335) get_all /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/compute/api.py:1057 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-8b70379a-00f5-4b7c-a903-a526e87c02d6 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/detail returned with HTTP 200 2012-06-05 19:53:52 INFO nova.api.openstack.wsgi [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] POST http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/2e6c8b3a-961b-48de-a568-b8ea799661e4/action 2012-06-05 19:53:53 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Making asynchronous call on compute ... from (pid=6335) multicall /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/amqp.py:321 2012-06-05 19:53:53 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] MSG_ID is 060a9fe9ff474762959dcc24bff14eb4 from (pid=6335) multicall /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/amqp.py:324 2012-06-05 19:54:09 INFO nova.api.openstack.wsgi [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] http://10.168.3.113:8774/v2/6cd20a72ec4a455a8648d7728975f32e/servers/2e6c8b3a-961b-48de-a568-b8ea799661e4/action returned with HTTP 200 this the compute.log in compute node 2012-06-05 19:54:02 DEBUG nova.manager [-] Running periodic task ComputeManager._poll_unconfirmed_resizes from (pid=15794) periodic_tasks /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/manager.py:152 2012-06-05 19:54:02 DEBUG nova.rpc.amqp [-] received {u'_context_roles': [u'KeystoneServiceAdmin', u'admin', u'KeystoneAdmin'], u'_msg_id': u'060a9fe9ff474762959dcc24bff14eb4', u'_context_read_deleted': u'no', u'_context_request_id': u'req-1f6e439b-5de7-41af-a847-e7471cfc7ec4', u'args': {u'instance_uuid': u'2e6c8b3a-961b-48de-a568-b8ea799661e4', u'tail_length': None}, u'_context_auth_token': '<SANITIZED>', u'_context_is_admin': True, u'_context_project_id': u'6cd20a72ec4a455a8648d7728975f32e', u'_context_timestamp': u'2012-06-05T11:53:52.961339', u'_context_user_id': u'ac148194334b475e8820334b156a532b', u'method': u'get_console_output', u'_context_remote_address': u'10.168.3.113'} from (pid=15794) _safe_log /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/common.py:160 2012-06-05 19:54:02 DEBUG nova.rpc.amqp [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] unpacked context: {'user_id': u'ac148194334b475e8820334b156a532b', 'roles': [u'KeystoneServiceAdmin', u'admin', u'KeystoneAdmin'], 'timestamp': '2012-06-05T11:53:52.961339', 'auth_token': '<SANITIZED>', 'remote_address': u'10.168.3.113', 'is_admin': True, 'request_id': u'req-1f6e439b-5de7-41af-a847-e7471cfc7ec4', 'project_id': u'6cd20a72ec4a455a8648d7728975f32e', 'read_deleted': u'no'} from (pid=15794) _safe_log /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/rpc/common.py:160 2012-06-05 19:54:02 AUDIT nova.compute.manager [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Get console output for instance 2e6c8b3a-961b-48de-a568-b8ea799661e4 2012-06-05 19:54:08 DEBUG nova.utils [req-1f6e439b-5de7-41af-a847-e7471cfc7ec4 ac148194334b475e8820334b156a532b 6cd20a72ec4a455a8648d7728975f32e] Running cmd (subprocess): sudo chown 2006 /var/lib/nova/instances/instance-00000017/console.log from (pid=15794) execute /usr/lib/python2.6/site-packages/nova-2012.1-py2.6.egg/nova/utils.py:219s  what was worse, it may appear the timeot error.like this bug https://bugs.launchpad.net/nova/+bug/966767 Hello:     i deploy openstak with multi node.     control node: rabbitmq; keystone; mysql; horizon; nova-api nova-scheduler nova-novncproxy nova-consoleauth and glance     network node: nove-network (flatdhcp) nova-api (only metadata service)     compute node: nova-compute     After i launch an instance,i try to view the log of vm through horizon.But it take nearly 20 seconds ....then i try again by cli command (nova console-log XXX) and get the same result.   what was worse, it may appear the timeot error.like this bug https://bugs.launchpad.net/nova/+bug/966767
2012-07-12 13:32:34 Thierry Carrez nova: status Incomplete New
2012-08-14 21:44:31 Russell Bryant nova: status New Incomplete
2012-11-23 21:24:04 Davanum Srinivas (DIMS) nova: status Incomplete Invalid