vmware: error trying to store hypervisor_version as string in using postgresql

[Yaguang Tang]

Hi,

I am trying to use VMware hypervisor ESXi 5.0.0 as a compute resource in OpenStack 2013.1.1. The driver being used is "•vmwareapi.VMwareVCDriver". The vCenter is contains a single cluster (two nodes) running ESXi version 5.0.

In the compute node, I am seeing the below error

2013-06-26 17:45:27.532 10253 AUDIT nova.compute.resource_tracker [-] Free ram (MB): 146933
2013-06-26 17:45:27.532 10253 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 55808
2013-06-26 17:45:27.533 10253 AUDIT nova.compute.resource_tracker [-] Free VCPUS: 24
2013-06-26 17:45:27.533 10253 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:583
2013-06-26 17:45:27.534 10253 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is ece63342e4254910be13ee92b948ace6 multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:586
2013-06-26 17:45:27.534 10253 DEBUG nova.openstack.common.rpc.amqp [-] UNIQUE_ID is c85fb461be22468b846368a6b8608ac2. _add_unique_id /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:337
2013-06-26 17:45:27.566 10253 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:583
2013-06-26 17:45:27.567 10253 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is 3f5d065d1c8a4989b4d36415d45abe8b multicall /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:586
2013-06-26 17:45:27.567 10253 DEBUG nova.openstack.common.rpc.amqp [-] UNIQUE_ID is 1421bebfd9744271be63dcef74551c93. _add_unique_id /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:337
2013-06-26 17:45:27.597 10253 CRITICAL nova [-] Remote error: DBError (DataError) invalid input syntax for integer: "5.0.0"
LINE 1: ..., 7, 24, 147445, 55808, 0, 512, 0, 'VMware ESXi', '5.0.0', '...
                                                             ^
 'INSERT INTO compute_nodes (created_at, updated_at, deleted_at, deleted, service_id, vcpus, memory_mb, local_gb, vcpus_used, memory_mb_used, local_gb_used, hypervisor_type, hypervisor_version, hypervisor_hostname, free_ram_mb, free_disk_gb, current_workload, running_vms, cpu_info, disk_available_least) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(service_id)s, %(vcpus)s, %(memory_mb)s, %(local_gb)s, %(vcpus_used)s, %(memory_mb_used)s, %(local_gb_used)s, %(hypervisor_type)s, %(hypervisor_version)s, %(hypervisor_hostname)s, %(free_ram_mb)s, %(free_disk_gb)s, %(current_workload)s, %(running_vms)s, %(cpu_info)s, %(disk_available_least)s) RETURNING compute_nodes.id' {'local_gb': 55808, 'vcpus_used': 0, 'deleted': 0, 'hypervisor_type': u'VMware ESXi', 'created_at': datetime.datetime(2013, 6, 26, 12, 15, 34, 804731), 'local_gb_used': 0, 'updated_at': None, 'hypervisor_hostname': u'10.100.10.42', 'memory_mb': 147445, 'current_workload': 0, 'vcpus': 24, 'free_ram_mb': 146933, 'running_vms': 0, 'free_disk_gb': 55808, 'service_id': 7, 'hypervisor_version': u'5.0.0', 'disk_available_least': None, 'deleted_at': None, 'cpu_info': u'{"model": "Intel(R) Xeon(R) CPU L5640 @ 2.27GHz", "vendor": "HP", "topology": {"cores": 12, "threads": 24, "sockets": 2}}', 'memory_mb_used': 512}
[u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 430, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 133, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 350, in compute_node_create\n result = self.db.compute_node_create(context, values)\n', u' File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 192, in compute_node_create\n return IMPL.compute_node_create(context, values)\n', u' File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 96, in wrapper\n return f(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 498, in compute_node_create\n compute_node_ref.save()\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/models.py", line 54, in save\n session.flush()\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py", line 437, in _wrap\n raise exception.DBError(e)\n', u'DBError: (DataError) invalid input syntax for integer: "5.0.0"\nLINE 1: ..., 7, 24, 147445, 55808, 0, 512, 0, \'VMware ESXi\', \'5.0.0\', \'...\n ^\n \'INSERT INTO compute_nodes (created_at, updated_at, deleted_at, deleted, service_id, vcpus, memory_mb, local_gb, vcpus_used, memory_mb_used, local_gb_used, hypervisor_type, hypervisor_version, hypervisor_hostname, free_ram_mb, free_disk_gb, current_workload, running_vms, cpu_info, disk_available_least) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(service_id)s, %(vcpus)s, %(memory_mb)s, %(local_gb)s, %(vcpus_used)s, %(memory_mb_used)s, %(local_gb_used)s, %(hypervisor_type)s, %(hypervisor_version)s, %(hypervisor_hostname)s, %(free_ram_mb)s, %(free_disk_gb)s, %(current_workload)s, %(running_vms)s, %(cpu_info)s, %(disk_available_least)s) RETURNING compute_nodes.id\' {\'local_gb\': 55808, \'vcpus_used\': 0, \'deleted\': 0, \'hypervisor_type\': u\'VMware ESXi\', \'created_at\': datetime.datetime(2013, 6, 26, 12, 15, 34, 804731), \'local_gb_used\': 0, \'updated_at\': None, \'hypervisor_hostname\': u\'10.100.10.42\', \'memory_mb\': 147445, \'current_workload\': 0, \'vcpus\': 24, \'free_ram_mb\': 146933, \'running_vms\': 0, \'free_disk_gb\': 55808, \'service_id\': 7, \'hypervisor_version\': u\'5.0.0\', \'disk_available_least\': None, \'deleted_at\': None, \'cpu_info\': u\'{"model": "Intel(R) Xeon(R) CPU L5640 @ 2.27GHz", "vendor": "HP", "topology": {"cores": 12, "threads": 24, "sockets": 2}}\', \'memory_mb_used\': 512}\n'].
2013-06-26 17:45:27.597 10253 TRACE nova Traceback (most recent call last):
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/bin/nova-compute", line 85, in <module>
2013-06-26 17:45:27.597 10253 TRACE nova service.wait()
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 689, in wait
2013-06-26 17:45:27.597 10253 TRACE nova _launcher.wait()
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 209, in wait
2013-06-26 17:45:27.597 10253 TRACE nova super(ServiceLauncher, self).wait()
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 179, in wait
2013-06-26 17:45:27.597 10253 TRACE nova service.wait()
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait
2013-06-26 17:45:27.597 10253 TRACE nova return self._exit_event.wait()
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
2013-06-26 17:45:27.597 10253 TRACE nova return hubs.get_hub().switch()
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
2013-06-26 17:45:27.597 10253 TRACE nova return self.greenlet.switch()
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
2013-06-26 17:45:27.597 10253 TRACE nova result = function(*args, **kwargs)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 147, in run_server
2013-06-26 17:45:27.597 10253 TRACE nova server.start()
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 446, in start
2013-06-26 17:45:27.597 10253 TRACE nova self.manager.pre_start_hook(rpc_connection=self.conn)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 612, in pre_start_hook
2013-06-26 17:45:27.597 10253 TRACE nova self.update_available_resource(nova.context.get_admin_context())
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3877, in update_available_resource
2013-06-26 17:45:27.597 10253 TRACE nova rt.update_available_resource(context)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line 242, in inner
2013-06-26 17:45:27.597 10253 TRACE nova retval = f(*args, **kwargs)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 272, in update_available_resource
2013-06-26 17:45:27.597 10253 TRACE nova self._sync_compute_node(context, resources)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 293, in _sync_compute_node
2013-06-26 17:45:27.597 10253 TRACE nova self._create(context, resources)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 293, in _sync_compute_node
2013-06-26 17:45:27.597 10253 TRACE nova self._create(context, resources)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 307, in _create
2013-06-26 17:45:27.597 10253 TRACE nova values)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 617, in compute_node_create
2013-06-26 17:45:27.597 10253 TRACE nova return self.conductor_rpcapi.compute_node_create(context, values)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 349, in compute_node_create
2013-06-26 17:45:27.597 10253 TRACE nova return self.call(context, msg, version='1.33')
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", line 80, in call
2013-06-26 17:45:27.597 10253 TRACE nova return rpc.call(context, self._get_topic(topic), msg, timeout)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py", line 140, in call
2013-06-26 17:45:27.597 10253 TRACE nova return _get_impl().call(CONF, context, topic, msg, timeout)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 798, in call
2013-06-26 17:45:27.597 10253 TRACE nova rpc_amqp.get_connection_pool(conf, Connection))
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 612, in call
2013-06-26 17:45:27.597 10253 TRACE nova rv = list(rv)
2013-06-26 17:45:27.597 10253 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 561, in __iter__
2013-06-26 17:45:27.597 10253 TRACE nova raise result
2013-06-26 17:45:27.597 10253 TRACE nova RemoteError: Remote error: DBError (DataError) invalid input syntax for integer: "5.0.0"
2013-06-26 17:45:27.597 10253 TRACE nova LINE 1: ..., 7, 24, 147445, 55808, 0, 512, 0, 'VMware ESXi', '5.0.0', '...
2013-06-26 17:45:27.597 10253 TRACE nova ^
2013-06-26 17:45:27.597 10253 TRACE nova 'INSERT INTO compute_nodes (created_at, updated_at, deleted_at, deleted, service_id, vcpus, memory_mb, local_gb, vcpus_used, memory_mb_used, local_gb_used, hypervisor_type, hypervisor_version, hypervisor_hostname, free_ram_mb, free_disk_gb, current_workload, running_vms, cpu_info, disk_available_least) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(service_id)s, %(vcpus)s, %(memory_mb)s, %(local_gb)s, %(vcpus_used)s, %(memory_mb_used)s, %(local_gb_used)s, %(hypervisor_type)s, %(hypervisor_version)s, %(hypervisor_hostname)s, %(free_ram_mb)s, %(free_disk_gb)s, %(current_workload)s, %(running_vms)s, %(cpu_info)s, %(disk_available_least)s) RETURNING compute_nodes.id' {'local_gb': 55808, 'vcpus_used': 0, 'deleted': 0, 'hypervisor_type': u'VMware ESXi', 'created_at': datetime.datetime(2013, 6, 26, 12, 15, 34, 804731), 'local_gb_used': 0, 'updated_at': None, 'hypervisor_hostname': u'10.100.10.42', 'memory_mb': 147445, 'current_workload': 0, 'vcpus': 24, 'free_ram_mb': 146933, 'running_vms': 0, 'free_disk_gb': 55808, 'service_id': 7, 'hypervisor_version': u'5.0.0', 'disk_available_least': None, 'deleted_at': None, 'cpu_info': u'{"model": "Intel(R) Xeon(R) CPU L5640 @ 2.27GHz", "vendor": "HP", "topology": {"cores": 12, "threads": 24, "sockets": 2}}', 'memory_mb_used': 512}
2013-06-26 17:45:27.597 10253 TRACE nova [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 430, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 133, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 350, in compute_node_create\n result = self.db.compute_node_create(context, values)\n', u' File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 192, in compute_node_create\n return IMPL.compute_node_create(context, values)\n', u' File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 96, in wrapper\n return f(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 498, in compute_node_create\n compute_node_ref.save()\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/models.py", line 54, in save\n session.flush()\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py", line 437, in _wrap\n raise exception.DBError(e)\n', u'DBError: (DataError) invalid input syntax for integer: "5.0.0"\nLINE 1: ..., 7, 24, 147445, 55808, 0, 512, 0, \'VMware ESXi\', \'5.0.0\', \'...\n ^\n \'INSERT INTO compute_nodes (created_at, updated_at, deleted_at, deleted, service_id, vcpus, memory_mb, local_gb, vcpus_used, memory_mb_used, local_gb_used, hypervisor_type, hypervisor_version, hypervisor_hostname, free_ram_mb, free_disk_gb, current_workload, running_vms, cpu_info, disk_available_least) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(service_id)s, %(vcpus)s, %(memory_mb)s, %(local_gb)s, %(vcpus_used)s, %(memory_mb_used)s, %(local_gb_used)s, %(hypervisor_type)s, %(hypervisor_version)s, %(hypervisor_hostname)s, %(free_ram_mb)s, %(free_disk_gb)s, %(current_workload)s, %(running_vms)s, %(cpu_info)s, %(disk_available_least)s) RETURNING compute_nodes.id\' {\'local_gb\': 55808, \'vcpus_used\': 0, \'deleted\': 0, \'hypervisor_type\': u\'VMware ESXi\', \'created_at\': datetime.datetime(2013, 6, 26, 12, 15, 34, 804731), \'local_gb_used\': 0, \'updated_at\': None, \'hypervisor_hostname\': u\'10.100.10.42\', \'memory_mb\': 147445, \'current_workload\': 0, \'vcpus\': 24, \'free_ram_mb\': 146933, \'running_vms\': 0, \'free_disk_gb\': 55808, \'service_id\': 7, \'hypervisor_version\': u\'5.0.0\', \'disk_available_least\': None, \'deleted_at\': None, \'cpu_info\': u\'{"model": "Intel(R) Xeon(R) CPU L5640 @ 2.27GHz", "vendor": "HP", "topology": {"cores": 12, "threads": 24, "sockets": 2}}\', \'memory_mb_used\': 512}\n'].
2013-06-26 17:45:27.597 10253 TRACE nova

Is this a bug with OpenStack?

Thank you,
Pragadees

[Yaguang Tang (heut2008)]

reported by impoundbull

[Kiran Kumar Vaddi (kiran-kumar-vaddi)]

This is caused since the hypervisor_version in the compute_nodes table is an Integer field and we are trying to push 5.0.0. This fails is some DBs like postgresql. Needs a fix in the VMware driver to remove the period and put back the version as 500 or 5.00

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/34685

[David Jia (jiataotj)]

I think field "hypervisor_hostname" in table compute_nodes should defined as string is more reasonable

[Shawn Hartsock (hartsock)]

I've opened this bug instead:
https://bugs.launchpad.net/nova/+bug/1207058

I would rather the database table definition were changed to match working data types.

[Gary Kotton (garyk)]

https://review.openstack.org/#/c/53109/

[Joe Gordon (jogo)]

Gary, now that I have re-read everything I see that this was mistakenly set to won't fix. But when fixing that can you please explain why.

Also I don't think this is worth a critical level, it may be critical to vmware but its not critical to nova (but I won't touch the importance anyway).

So tell me if I am wrong: the won't fix was to the original solution, of changing the DB entry into a string. Instead you just converted the value that was being added from a string to an int. Which is the correct fix.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/53109
Committed: http://github.com/openstack/nova/commit/e2efa57ab2fb03b92f80abbf612a257c28475140
Submitter: Jenkins
Branch: master

commit e2efa57ab2fb03b92f80abbf612a257c28475140
Author: Gary Kotton <email address hidden>
Date: Tue Oct 22 05:51:09 2013 -0700

    VMware: enable driver to work with postgres database

    The hypervisor version is expected to be an integer and not
    a string.

    Change-Id: Ifac9ca3480191413e3a02f0dae94e452707ae82f
    Closes-Bug: #1195139

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/72978

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/72978
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=00d18827aaf780e419c5b335bf2651b17ea34e30
Submitter: Jenkins
Branch: stable/havana

commit 00d18827aaf780e419c5b335bf2651b17ea34e30
Author: Gary Kotton <email address hidden>
Date: Tue Oct 22 05:51:09 2013 -0700

    VMware: enable driver to work with postgres database

    The hypervisor version is expected to be an integer and not
    a string. The nova/utils.py change is part of the following
    patch but is not being backported as it adds new feature
    functionality and is therefore not acceptable for backport

    https://review.openstack.org/#/c/36073/

    Closes-Bug: #1195139
    (cherry picked from commit e2efa57ab2fb03b92f80abbf612a257c28475140)

    Conflicts:

     nova/tests/virt/vmwareapi/test_vmwareapi.py
     nova/virt/vmwareapi/host.py

    Change-Id: Ifac9ca3480191413e3a02f0dae94e452707ae82f