Use of BIGINT column type affects testing with sqlite

Bug #1147879 reported by Kaushik Chandrashekar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Low
Kaushik Chandrashekar

Bug Description

The problem is sqlalchemy doesn't recognize the type BigInt

Step to reproduce this issue:

Run the unit tests,

$./run_tests.sh api

Then check the logs,

$head -100 subunit.log
time: 2013-03-05 21:47:33.002802Z
test: nova.tests.api.ec2.test_cinder_cloud.CinderCloudTestCase.test_create_snapshot
time: 2013-03-05 21:47:34.087936Z
tags: worker-1
successful: nova.tests.api.ec2.test_cinder_cloud.CinderCloudTestCase.test_create_snapshot [ multipart
Content-Type: text/plain;charset=utf8
pythonlogging:'nova'
2A1
Loading network driver 'nova.network.linux_net'
Starting conductor node (version 2013.1)
Loading compute driver 'nova.virt.fake.FakeDriver'
Starting compute node (version 2013.1)
Updating host status
Auditing locally available compute resources
Free ram (MB): 7680
Free disk (GB): 1028
Free VCPUS: 1
Compute_service record created for 07698146867b4ee98105daf0c0b6a9b3:fake-mini
Starting scheduler node (version 2013.1)
Loading network driver 'nova.network.linux_net'
Starting network node (version 2013.1)
Create volume of 1 GB
creating volume 2a22fa5a-07d3-4dbc-95d5-704645348577
Create snapshot of volume vol-00000001
deleting volume 2a22fa5a-07d3-4dbc-95d5-704645348577
0
Content-Type: text/plain;charset=utf8
stderr
AC5
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bw_in'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bw_out'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'last_ctr_in'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'last_ctr_out'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'tot_reads'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'tot_read_bytes'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'tot_writes'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'tot_write_bytes'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'curr_reads'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'curr_read_bytes'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'curr_writes'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'curr_write_bytes'
  default, primary_key))
/home/kaushikc/code/nova/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/expression.py:1927: SAWarning: The IN-predicate on "instances.uuid" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance.
  return self._in_impl(operators.in_op, operators.notin_op, other)
0
Content-Type: text/plain;charset=utf8
stdout
0
]

The same problem happens while running tests with other tools like nosetests

$ nosetests

nosetests nova/tests/api/test_auth.py
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bw_in'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bw_out'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'last_ctr_in'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'last_ctr_out'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'tot_reads'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'tot_read_bytes'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'tot_writes'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'tot_write_bytes'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'curr_reads'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'curr_read_bytes'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'curr_writes'
  default, primary_key))
/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'curr_write_bytes'
  default, primary_key))
..........
----------------------------------------------------------------------
Ran 10 tests in 1.493s

OK

Since the warnings are in stderr, some editor plugins (like pyunit) to run unit tests in place, are not working.

Changed in nova:
assignee: nobody → Kaushik Chandrashekar (kaushikc)
tags: added: low-hanging-fruit
aeva black (tenbrae)
Changed in nova:
status: New → Confirmed
Revision history for this message
aeva black (tenbrae) wrote :

 I'm fairly sure that sqla is just converting these to INT type, which in sqlite can accommodate up to 8 bytes already. I don't think the tests are adversely affected, it's just the warning gives the impression something is wrong when it isn't. This is really an issue in sqla, but it would be nice to not see those warnings all the time.

Changed in nova:
importance: Undecided → Low
tags: added: db
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/23883

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/23883
Committed: http://github.com/openstack/nova/commit/4ed4c848b66bc5fdd42e05c8b8c726ace2c25704
Submitter: Jenkins
Branch: master

commit 4ed4c848b66bc5fdd42e05c8b8c726ace2c25704
Author: Kaushik Chandrashekar <email address hidden>
Date: Thu Mar 7 19:17:00 2013 -0600

    Compile BigInteger to INTEGER for sqlite

    Ensure BigInteger does not map to BIGINT for an auto-created
    sqlite registry DB, as this type is not supported by sqlite.

    Refactoring a test to use generic unsupported column type

    Fixes bug 1147879

    Change-Id: I590c798dd470def286802bc6e00bb4b621332b8d

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-rc1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.