help text string on config fails translations and breaks glance docs

[Nikhil Komawar]

When glance_store 0.14.0 released and was updated in global requirements (upper constraints), the glance-docs gate failed.

Some of the reviews which failed on jenkins around july 20 are https://review.openstack.org/#/c/341168/ , https://review.openstack.org/#/c/345016/ and a few others.

Example failure is http://logs.openstack.org/28/326128/8/check/gate-glance-docs/edd5b69/console.html#_2016-07-20_21_34_30_530628 (copied below).

We pinned down glance_store on u-c and g-r to keep the glance review pipe flowing but until this issue is fixed glance_store release is blocked.

Temporary fix:
[1] requirements patch and [2] resolved the issue with glance-docs.

[1] https://review.openstack.org/#/c/345821/
[2] https://review.openstack.org/#/c/345951/

Full discussion is at: http://lists.openstack.org/pipermail/openstack-docs/2016-July/008899.html ( thread: http://lists.openstack.org/pipermail/openstack-docs/2016-July/thread.html#8899 ).

Some further digging is required as to where exactly is the translated string failing (a sphinx extension as Doug pointed out).

---

2016-07-20 21:34:30.506393 | found extension EntryPoint.parse('glance.store.filesystem.Store = glance_store._drivers.filesystem:Store')
2016-07-20 21:34:30.506567 | found extension EntryPoint.parse('cinder = glance_store._drivers.cinder:Store')
2016-07-20 21:34:30.506743 | found extension EntryPoint.parse('glance.store.rbd.Store = glance_store._drivers.rbd:Store')
2016-07-20 21:34:30.506919 | found extension EntryPoint.parse('no_conf = glance_store.tests.fakes:UnconfigurableStore')
2016-07-20 21:34:30.507092 | found extension EntryPoint.parse('swift = glance_store._drivers.swift:Store')
2016-07-20 21:34:30.507272 | found extension EntryPoint.parse('rbd = glance_store._drivers.rbd:Store')
2016-07-20 21:34:30.507444 | found extension EntryPoint.parse('http = glance_store._drivers.http:Store')
2016-07-20 21:34:30.507625 | found extension EntryPoint.parse('glance.store.s3.Store = glance_store._drivers.s3:Store')
2016-07-20 21:34:30.507805 | found extension EntryPoint.parse('vmware = glance_store._drivers.vmware_datastore:Store')
2016-07-20 21:34:30.508004 | found extension EntryPoint.parse('glance.store.swift.Store = glance_store._drivers.swift:Store')
2016-07-20 21:34:30.508473 | found extension EntryPoint.parse('glance.store.sheepdog.Store = glance_store._drivers.sheepdog:Store')
2016-07-20 21:34:30.508701 | found extension EntryPoint.parse('s3 = glance_store._drivers.s3:Store')
2016-07-20 21:34:30.508903 | found extension EntryPoint.parse('sheepdog = glance_store._drivers.sheepdog:Store')
2016-07-20 21:34:30.509114 | found extension EntryPoint.parse('glance.store.vmware_datastore.Store = glance_store._drivers.vmware_datastore:Store')
2016-07-20 21:34:30.509311 | found extension EntryPoint.parse('glance.store.http.Store = glance_store._drivers.http:Store')
2016-07-20 21:34:30.509506 | found extension EntryPoint.parse('glance.store.cinder.Store = glance_store._drivers.cinder:Store')
2016-07-20 21:34:30.509692 | found extension EntryPoint.parse('file = glance_store._drivers.filesystem:Store')
2016-07-20 21:34:30.525121 | [oslo.config] None DEFAULT
2016-07-20 21:34:30.528561 | [oslo.config] None cors
2016-07-20 21:34:30.528773 | [oslo.config] None cors.subdomain
2016-07-20 21:34:30.528976 | [oslo.config] None database
2016-07-20 21:34:30.530043 | [oslo.config] None glance_store
2016-07-20 21:34:30.530628 | /home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/oslo_i18n/_message.py:176: UserWarning: Failed to insert replacement values into translated message Override service catalog lookup with template for cinder endpoint e.g. http://localhost:8776/v2/%(tenant)s (Original: 'Override service catalog lookup with template for cinder endpoint e.g. http://localhost:8776/v2/%(tenant)s'): u'tenant'
2016-07-20 21:34:30.530671 | warnings.warn(msg % (translated_message, self.msgid, err))
2016-07-20 21:34:30.530871 | Failed to insert replacement values into translated message Override service catalog lookup with template for cinder endpoint e.g. http://localhost:8776/v2/%(tenant)s (Original: 'Override service catalog lookup with template for cinder endpoint e.g. http://localhost:8776/v2/%(tenant)s'): u'tenant'
2016-07-20 21:34:30.532955 | Traceback (most recent call last):
2016-07-20 21:34:30.533007 | File "setup.py", line 29, in <module>
2016-07-20 21:34:30.533080 | pbr=True)
2016-07-20 21:34:30.533208 | File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
2016-07-20 21:34:30.533245 | dist.run_commands()
2016-07-20 21:34:30.533278 | File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
2016-07-20 21:34:30.533409 | self.run_command(cmd)
2016-07-20 21:34:30.533445 | File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
2016-07-20 21:34:30.533571 | cmd_obj.run()
2016-07-20 21:34:30.533625 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/pbr/builddoc.py", line 198, in run
2016-07-20 21:34:30.533650 | setup_command.BuildDoc.run(self)
2016-07-20 21:34:30.533699 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/sphinx/setup_command.py", line 161, in run
2016-07-20 21:34:30.533730 | app.build(force_all=self.all_files)
2016-07-20 21:34:30.533780 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/sphinx/application.py", line 216, in build
2016-07-20 21:34:30.533800 | self.builder.build_all()
2016-07-20 21:34:30.533849 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 177, in build_all
2016-07-20 21:34:30.533884 | self.build(None, summary='all source files', method='all')
2016-07-20 21:34:30.533933 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 234, in build
2016-07-20 21:34:30.533950 | purple, length):
2016-07-20 21:34:30.534001 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 134, in status_iterator
2016-07-20 21:34:30.534024 | for item in iterable:
2016-07-20 21:34:30.534079 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/sphinx/environment.py", line 478, in update_generator
2016-07-20 21:34:30.534100 | self.read_doc(docname, app=app)
2016-07-20 21:34:30.534149 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/sphinx/environment.py", line 628, in read_doc
2016-07-20 21:34:30.534165 | pub.publish()
2016-07-20 21:34:30.534211 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/core.py", line 217, in publish
2016-07-20 21:34:30.534228 | self.settings)
2016-07-20 21:34:30.534275 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/readers/__init__.py", line 72, in read
2016-07-20 21:34:30.534291 | self.parse()
2016-07-20 21:34:30.534339 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/readers/__init__.py", line 78, in parse
2016-07-20 21:34:30.534361 | self.parser.parse(self.input, document)
2016-07-20 21:34:30.534410 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/__init__.py", line 172, in parse
2016-07-20 21:34:30.534439 | self.statemachine.run(inputlines, document, inliner=self.inliner)
2016-07-20 21:34:30.534487 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 170, in run
2016-07-20 21:34:30.534508 | input_source=document['source'])
2016-07-20 21:34:30.534554 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
2016-07-20 21:34:30.534573 | context, state, transitions)
2016-07-20 21:34:30.534621 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
2016-07-20 21:34:30.534645 | return method(match, context, next_state)
2016-07-20 21:34:30.534693 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2961, in text
2016-07-20 21:34:30.534787 | self.section(title.lstrip(), source, style, lineno + 1, messages)
2016-07-20 21:34:30.534843 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
2016-07-20 21:34:30.534868 | self.new_subsection(title, lineno, messages)
2016-07-20 21:34:30.534922 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
2016-07-20 21:34:30.534954 | node=section_node, match_titles=True)
2016-07-20 21:34:30.535007 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
2016-07-20 21:34:30.535029 | node=node, match_titles=match_titles)
2016-07-20 21:34:30.535077 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
2016-07-20 21:34:30.535105 | results = StateMachineWS.run(self, input_lines, input_offset)
2016-07-20 21:34:30.535152 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
2016-07-20 21:34:30.535171 | context, state, transitions)
2016-07-20 21:34:30.535219 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
2016-07-20 21:34:30.535242 | return method(match, context, next_state)
2016-07-20 21:34:30.535294 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2299, in explicit_markup
2016-07-20 21:34:30.535507 | nodelist, blank_finish = self.explicit_construct(match)
2016-07-20 21:34:30.535566 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2311, in explicit_construct
2016-07-20 21:34:30.535874 | return method(self, expmatch)
2016-07-20 21:34:30.535929 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2054, in directive
2016-07-20 21:34:30.536277 | directive_class, match, type_name, option_presets)
2016-07-20 21:34:30.536334 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2103, in run_directive
2016-07-20 21:34:30.536617 | result = directive_instance.run()
2016-07-20 21:34:30.536671 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/oslo_config/sphinxext.py", line 265, in run
2016-07-20 21:34:30.536704 | for line in _format_option_help(app, namespaces, split_namespaces):
2016-07-20 21:34:30.536757 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/oslo_config/sphinxext.py", line 227, in _format_option_help
2016-07-20 21:34:30.536774 | for line in lines:
2016-07-20 21:34:30.536824 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/oslo_config/sphinxext.py", line 173, in _format_group
2016-07-20 21:34:30.536855 | yield _indent(' :Reason: ' + opt.deprecated_reason)
2016-07-20 21:34:30.536902 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/oslo_i18n/_message.py", line 233, in __radd__
2016-07-20 21:34:30.536921 | return self.__add__(other)
2016-07-20 21:34:30.536968 | File "/home/jenkins/workspace/gate-glance-docs/.tox/venv/local/lib/python2.7/site-packages/oslo_i18n/_message.py", line 230, in __add__
2016-07-20 21:34:30.536985 | raise TypeError(msg)
2016-07-20 21:34:30.537010 | TypeError: Message objects do not support addition.
2016-07-20 21:34:30.936055 | ERROR: InvocationError: '/home/jenkins/workspace/gate-glance-docs/.tox/venv/bin/python setup.py build_sphinx'
2016-07-20 21:34:30.936147 | ___________________________________ summary ____________________________________
2016-07-20 21:34:30.936172 | ERROR: venv: commands failed
2016-07-20 21:34:30.944612 | [Zuul] Task exit code: 1
2016-07-20 21:34:34.117908 | [Zuul] Job complete, result: FAILURE

[Nikhil Komawar (nikhil-komawar)]

I will attempt to unblock the glance_store releases with a temporary fix using https://bugs.launchpad.net/glance-store/+bug/1606746

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/347595
Committed: https://git.openstack.org/cgit/openstack/glance_store/commit/?id=10f50da5fcd9257090ab137f4d257940bb01b44a
Submitter: Jenkins
Branch: master

commit 10f50da5fcd9257090ab137f4d257940bb01b44a
Author: Nikhil Komawar <email address hidden>
Date: Tue Jul 26 22:08:28 2016 -0400

    Fix cinder config string as per current i18n state

    As described in the bug 1605648 , the translations being done on the
    config string are not permitting additions so, the glance docs
    generation is failing.

    The glance_store 0.14.0 had to be pinned down to let glance review queue
    not break. see updates on this thread
    http://lists.openstack.org/pipermail/openstack-docs/2016-July/thread.html#8899

    The config strings need to be non additive for translations and will
    enable future releases of glance_store. Once, a appropriate fix is
    published for bug 1605648 the config strings can be more flexible.

    Change-Id: Ia6ed5883458f2e81e861cda02068956728e76357
    Closes-Bug: 1606746
    Partial-Bug: 1605648

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/347907

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/347907
Committed: https://git.openstack.org/cgit/openstack/oslo.config/commit/?id=b40925328e5ba9e87be42bab1ec3a41bf3d81f78
Submitter: Jenkins
Branch: master

commit b40925328e5ba9e87be42bab1ec3a41bf3d81f78
Author: Doug Hellmann <email address hidden>
Date: Wed Jul 27 11:35:04 2016 -0400

    disable lazy translation in sphinx extension

    The sphinx extension depends on all of the help text associated with an
    option being regular strings. When oslo_i18n's lazy translation feature
    is turned on, the marker functions return Message objects that do not
    support all of the features of strings. Add a step to turn off lazy
    evaluation when the oslo_context.sphinxext extension is activated.

    Change-Id: Icaec4691135cd07d96994f01498e0f9d2e09af58
    Closes-Bug: #1605648
    Signed-off-by: Doug Hellmann <email address hidden>

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/oslo.config 3.14.0 release.

[Nikhil Komawar (nikhil-komawar)]

It was discovered that glance_store 0.15.0 also was affected and the latest release of oslo.config should resolve this.

[Ian Cordasco (icordasc)]

This no longer seems to be an issue. It seems that this has been fixed. Marking this as fix released