Glance Image Limit is 2^32 on 32bit Systems

[Donal Lafferty]

Glance uses plain integers to record the image size. On 32-bit machines, this limits the max image size to approximately 2 gigs. The image size may be larger on 64-bit systems.

E.g. in http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/api.py

def _image_update(context, values, image_id):
...

            if 'size' in values:
                values['size'] = int(values['size'])

[John Dickinson (notmyname)]

Python converts integers to long integers internally as they get bigger. Obviously there could be issues if this values is used in a c-style data structure or by a client that cannot handle larger values, but the referenced lines don't break.

For example, in a python console:
>>> int(2**100)
1267650600228229401496703205376L

If there were limits to the integer representation, that would have overflowed them.

[Jay Pipes (jaypipes)]

Hey John, Donal,

So, are we saying this is, or is not, a bug?

-jay

[Rick Harris (rconradharris)]

> Python converts integers to long integers internally as they get bigger.

True, but ultimately the `size` value is persisted as a MySQL `Integer`, so, AFAIK we're capped at 2 billion.

> So, are we saying this is, or is not, a bug?

2 gig seems too low. I'd say we should make it a low priority bug to switch to using `sa.BigInteger` as the column type for `size`.

[Vish Ishaya (vishvananda)]

This seems like a high priority bug to me. Windows images are almost always going to be larger than 2G, even compressed.
On Mar 21, 2011, at 9:52 AM, Rick Harris wrote:

>> Python converts integers to long integers internally as they get
> bigger.
>
> True, but ultimately the `size` value is persisted as a MySQL `Integer`,
> so, AFAIK we're capped at 2 billion.
>
>> So, are we saying this is, or is not, a bug?
>
> 2 gig seems too low. I'd say we should make it a low priority bug to
> switch to using `sa.BigInteger` as the column type for `size`.
>
> --
> You received this bug notification because you are a member of Glance
> Bug Team, which is subscribed to Glance.
> https://bugs.launchpad.net/bugs/739433
>
> Title:
> Glance Image Limit is 2^32 on 32bit Systems
>
> Status in OpenStack Image Registry and Delivery Service (Glance):
> New
>
> Bug description:
> Glance uses plain integers to record the image size. On 32-bit
> machines, this limits the max image size to approximately 2 gigs. The
> image size may be larger on 64-bit systems.
>
> E.g. in http://bazaar.launchpad.net/~glance-core/glance/cactus-
> trunk/view/head:/glance/registry/db/api.py
>
> def _image_update(context, values, image_id):
> ...
>
> if 'size' in values:
> values['size'] = int(values['size'])

[Donal Lafferty (donal-lafferty)]

Is there a work around?

E.g. is there an alternative to 'Integer' in http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/migrate_repo/versions/001_add_images_table.py

[Jay Pipes (jaypipes)]

Agreed that this should probably be a high priority bug, since there is no workaround and many images are going to be > 2GB.

Rick, changing Integer to BigInteger should fix it I believe?

Unfortunately, here comes another sa-migrate script that won't pass our test suite (because SQLite will drop the column and sa-migrate will try to drop non-existing indexes after dropping the original table schema)

[Jay Pipes (jaypipes)]

Before making any changes to the database schema, I created this functional test:

    def test_size_greater_2G(self):
        """
        A test against the actual datastore backend for the registry
        to ensure that the image size property is not truncated.

        :see https://bugs.launchpad.net/glance/+bug/739433
        """

        self.cleanup()
        api_port, reg_port, conf_file = self.start_servers()

        # 1. POST /images with public image named Image1
        # attribute and a size of 5G. Use the HTTP engine with an
        # X-Image-Meta-Location attribute to make Glance forego
        # "adding" the image data.
        # Verify a 200 OK is returned
        cmd = ("curl -i -X POST "
               "-H 'Expect: ' " # Necessary otherwise sends 100 Continue
               "-H 'X-Image-Meta-Location: http://example.com/fakeimage' "
               "-H 'X-Image-Meta-Size: %d' "
               "-H 'X-Image-Meta-Name: Image1' "
               "-H 'X-Image-Meta-Is-Public: True' "
               "http://0.0.0.0:%d/images") % (FIVE_GB, api_port)

        exitcode, out, err = execute(cmd)
        self.assertEqual(0, exitcode)

        lines = out.split("\r\n")
        status_line = lines[0]

        self.assertEqual("HTTP/1.1 200 OK", status_line)

        # 2. HEAD /images
        # Verify image size is what was passed in, and not truncated
        cmd = "curl -i -X HEAD http://0.0.0.0:%d/images/1" % api_port

        exitcode, out, err = execute(cmd)

        self.assertEqual(0, exitcode)

        lines = out.split("\r\n")
        status_line = lines[0]

        self.assertEqual("HTTP/1.1 200 OK", status_line)
        self.assertTrue("X-Image-Meta-Name: Image1" in out)
        self.assertTrue("X-Image-Meta-Size: %d" % FIVE_GB in out)

I got a pass on this test without any changes to the schema. Perhaps it's due to me being on a 64-bit machine, though, which makes SQLite's Integer columns 64-bit wide?

Donal, could you add that above test case to /tests/functional/test_curl_api.py and run on your machine? Let me know the results. Thanks!

[Donal Lafferty (donal-lafferty)]

Hi Jay,

I'm going to have to ask Ewan about how to execute /tests/functional/test_curl_api.py. Doesn't look like we're running these locally.

[Jay Pipes (jaypipes)]

Hi Donal,

What version of Glance's trunk are you running? The functional tests were added post-Bexar, in Cactus, r94.

Cheers!
jay

[Jay Pipes (jaypipes)]

Oh, and running ./run_tests.sh will execute the functional tests automatically...

[Donal Lafferty (donal-lafferty)]

Test passes with the default connection string, which attaches to sqlite.

In the process of checking MySQL, which is the database that we're using.

[Donal Lafferty (donal-lafferty)]

Three things. 1) sqlite INT is bigger than 32bits on a 32bit Ubuntu10.10. 2) mysql INT is not 3) can't get functional test to work with mysql connection string. E.g. with GLANCE_SQL_CONNECTION=mysql://root:mypassword@127.0.0.1/glance

To verify MySQL will not work with schema use mysql tool to add table with one column of type INT. E.g. start mysql tool

mysql -u"root" -p

Next, create a new database, and create a table 'test' with a column called 'size' fo type 'INT'. Add two rows, one with the value '1' and another with the value 5368709120 (5gig in bytes). The commands are below, and result in a wrong value being stored and returned for 5368709120

CREATE DATABASE foo;
USE foo;
CREATE TABLE test ( size INT);
INSERT INTO test (size) VALUES( 1 );
INSERT INTO test (size) VALUES( 5368709120 );
SELECT * FROM test;

E.g.

mysql> CREATE DATABASE foo;
Query OK, 1 row affected (0.00 sec)

mysql> USE foo;
Database changed
mysql> CREATE TABLE test (size INT)
    -> ;
Query OK, 0 rows affected (0.01 sec)

mysql> INSERT INTO test (size) VALUES (1);
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO test (size) VALUES (5368709120);
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> SELECT * FROM test;
+------------+
| size |
+------------+
| 1 |
| 2147483647 |
+------------+
2 rows in set (0.00 sec)

mysql>

[Jay Pipes (jaypipes)]

Thanks for the info, Donal. Much appreciated. I'll update the test case to allow for setting the sql_connection to a mysql database.

[Donal Lafferty (donal-lafferty)]

Besides testcase, source code needs fixing in two areas. The schema used to create new databases must use BIGINT as the 'size' column datatype. This is a two liner in the branch at https://code.launchpad.net/~donal-lafferty/glance/fix739433

The second issue is the migration script. I'm not familiar with the downgrade/upgrade mechanism. On the face of it, I would guess that all references to the 'size' column should use BIGINT. However, I'd have to test this theory. Does Jay Pipes already have a solution?

[Jay Pipes (jaypipes)]

Ran into some issues with the fix. Working through them locally. Just wanted to give a heads up.