Glance generates a non-reproducible config file

[Thomas Goirand]

When generating glance-api.conf and glance-registry.conf, the workers directive, even though commented, is populated with a value which depends on the number of CPU (or core?) of the machine generating the file. This means that the build of Glance isn't reproducible (ie: building the package on 2 different machine will not produce byte for byte identical packages).

If you didn't hear about the Debian reproducible build effort, please read these wiki entries:
https://wiki.debian.org/ReproducibleBuilds
https://wiki.debian.org/ReproducibleBuilds/About

and please consider removing non-deterministic bits when generating the configuration files.

[Stuart McLaren (stuart-mclaren)]

Is this a Glance bug, or is it a bug in the oslo config library which generates the config?

[Thomas Goirand (thomas-goirand)]

It's a bug in Glance. See glance/common/wsgi.py:

eventlet_opts = [
    cfg.IntOpt('workers', default=processutils.get_worker_count(),
               help=_('The number of child process workers that will be '
                      'created to service requests. The default will be '
                      'equal to the number of CPUs available.')),

While it's very much OK to have this kind of default (it's very good), what is not is having this default printed within the generated config file, even if the directive is commented-out. Indeed, processutils.get_worker_count() is in fact in use when using oslo-config-generator.

I would have happily write "2" instead of what's above to fix the config file, but that's not really what we want, we really want the described default.

Moving forward, what we could do is removing default=processutils.get_worker_count() and replace it by None, then if the value is None, then Glance would use processutils.get_worker_count() as value. That part, I don't know how to write it (and I'm too busy with the 320 OpenStack related packages I maintain in Debian to investigate).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/280587

[Cyril Roelandt (cyril-roelandt)]

Hey Thomas!

I hope this patch fixes your issue. Being a part of GNU Guix during my free time, I'm really interested in reproducibility issues, so do not hesitate to add me to such bug reports.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/281165
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=f89bbc5d879ab098134a1b3214669526e30a5d89
Submitter: Jenkins
Branch: master

commit f89bbc5d879ab098134a1b3214669526e30a5d89
Author: Stuart McLaren <email address hidden>
Date: Wed Feb 17 09:57:11 2016 +0000

    Add unit test for default number of workers

    The default number of workers should match the cpu count.

    Change-Id: Idfa1fea88e4a2e45ea2f888097bf82cb0005ab05
    Partial-bug: #1542855

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/280587
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=24ae73b157aeb7e4725fdc9496aee47d716d4a54
Submitter: Jenkins
Branch: master

commit 24ae73b157aeb7e4725fdc9496aee47d716d4a54
Author: Cyril Roelandt <email address hidden>
Date: Tue Feb 16 11:08:14 2016 +0100

    Make sure the generated glance-api.conf.sample is always the same

    Currently, it depends on the machine on which it has been generated, since the
    result of processutils.get_worker_count() is written in this file when running
    "tox -egenconfig". This prevents us from having reproducible builds.

    Change-Id: If34ac601dab65c567138a9c421b681ebebdeb7e2
    Closes-Bug: #1542855

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/glance 12.0.0.0b3 development milestone.