tox -e docs doesn't build module index

[Matt Riedemann]

With this change https://review.openstack.org/#/c/121737/ autodoc_index_modules was set to 0 in the [pbr] section of setup.cfg so that you can no longer get to git/nova/doc/build/html/py-modindex.html locally to verify that module docstrings are correct.

We should enable autodoc_index_modules for the docs tox target again. The original point of the previous change was to not list the module index in the home page, which is fine.

[Matt Riedemann (mriedem)]

You can also see this here:

http://docs-draft.openstack.org/36/198836/1/check/gate-nova-docs/e4ca4b6//doc/build/html/py-modindex.html

[Matt Riedemann (mriedem)]

Hit bug 1472276 with pbr, which I was able to hack around, but then hit another error:

checking consistency... 2015-07-07 08:48:42.435 24115 CRITICAL nova [req-7340a971-0f5c-47dc-871b-7ef7ee54415a fake fake - - -] SphinxWarning: /home/mriedem/git/nova/doc/source/api/autoindex.rst:: WARNING: document isn't included in any toctree

2015-07-07 08:48:42.435 24115 ERROR nova Traceback (most recent call last):
2015-07-07 08:48:42.435 24115 ERROR nova File "setup.py", line 29, in <module>
2015-07-07 08:48:42.435 24115 ERROR nova pbr=True)
2015-07-07 08:48:42.435 24115 ERROR nova File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
2015-07-07 08:48:42.435 24115 ERROR nova dist.run_commands()
2015-07-07 08:48:42.435 24115 ERROR nova File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
2015-07-07 08:48:42.435 24115 ERROR nova self.run_command(cmd)
2015-07-07 08:48:42.435 24115 ERROR nova File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
2015-07-07 08:48:42.435 24115 ERROR nova cmd_obj.run()
2015-07-07 08:48:42.435 24115 ERROR nova File "/home/mriedem/git/nova/.tox/docs/local/lib/python2.7/site-packages/pbr/builddoc.py", line 198, in run
2015-07-07 08:48:42.435 24115 ERROR nova self._sphinx_run()
2015-07-07 08:48:42.435 24115 ERROR nova File "/home/mriedem/git/nova/.tox/docs/local/lib/python2.7/site-packages/pbr/builddoc.py", line 152, in _sphinx_run
2015-07-07 08:48:42.435 24115 ERROR nova app.build(force_all=self.all_files)
2015-07-07 08:48:42.435 24115 ERROR nova File "/home/mriedem/git/nova/.tox/docs/local/lib/python2.7/site-packages/sphinx/application.py", line 216, in build
2015-07-07 08:48:42.435 24115 ERROR nova self.builder.build_all()
2015-07-07 08:48:42.435 24115 ERROR nova File "/home/mriedem/git/nova/.tox/docs/local/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 177, in build_all
2015-07-07 08:48:42.435 24115 ERROR nova self.build(None, summary='all source files', method='all')
2015-07-07 08:48:42.435 24115 ERROR nova File "/home/mriedem/git/nova/.tox/docs/local/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 261, in build
2015-07-07 08:48:42.435 24115 ERROR nova self.env.check_consistency()
2015-07-07 08:48:42.435 24115 ERROR nova File "/home/mriedem/git/nova/.tox/docs/local/lib/python2.7/site-packages/sphinx/environment.py", line 1677, in check_consistency
2015-07-07 08:48:42.435 24115 ERROR nova self.warn(docname, 'document isn\'t included in any toctree')
2015-07-07 08:48:42.435 24115 ERROR nova File "/home/mriedem/git/nova/.tox/docs/local/lib/python2.7/site-packages/sphinx/environment.py", line 257, in warn
2015-07-07 08:48:42.435 24115 ERROR nova self._warnfunc(msg, (docname, lineno))
2015-07-07 08:48:42.435 24115 ERROR nova File "/home/mriedem/git/nova/.tox/docs/local/lib/python2.7/site-packages/sphinx/application.py", line 268, in warn
2015-07-07 08:48:42.435 24115 ERROR nova raise SphinxWarning(warntext)
2015-07-07 08:48:42.435 24115 ERROR nova SphinxWarning: /home/mriedem/git/nova/doc/source/api/autoindex.rst:: WARNING: document isn't included in any toctree
2015-07-07 08:48:42.43...

[Matt Riedemann (mriedem)]

The solution for the sphinx stuff is to configure sphinx using the conf.py in the doc source dir, like in nova that means updating this file:

https://github.com/openstack/nova/blob/master/doc/source/conf.py#L85

[Matt Riedemann (mriedem)]

Re-opening, comment 3 was for the pbr bug.

[Matt Riedemann (mriedem)]

LaTeX is running during the docs build because of the .. math:: directives in the hyperv code here:

https://github.com/openstack/nova/blob/master/nova/virt/hyperv/vhdutilsv2.py#L129

That's built through the sphinx.ext.pngmath extension here:

https://github.com/openstack/nova/blob/master/doc/source/conf.py#L33

But to build that stuff, you need latex and some other package for the utf8x.def file:

debs:

texlive (for latex)
texlive-latex-extra (for utf8x.def)

rpms:

texlive (for latex)
texlive-latex (for utf8x.def)

Alternatively we could just not support math in the docstrings and update the hyper-v module. I doubt that latex is on the nodes that build the nova-python-docs job, so that would be a dependent update to the job config in project-config if they aren't there.

[Matt Riedemann (mriedem)]

I got latex installed but not it also apparently requires dvipng:

SphinxWarning: WARNING: dvipng command 'dvipng' cannot be run (needed for math display), check the pngmath_dvipng setting

[Matt Riedemann (mriedem)]

Here is the change: https://review.openstack.org/#/c/199297/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/199297
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=8b24bf766d13ddf556f9df81a9a7b541fa3b49f4
Submitter: Jenkins
Branch: master

commit 8b24bf766d13ddf556f9df81a9a7b541fa3b49f4
Author: Matt Riedemann <email address hidden>
Date: Mon Jul 6 15:18:59 2015 -0700

    Set autodoc_index_modules=True so tox -e docs builds module docs again

    Commit bd7e62f796fe951fd42c2edad56e252a0b7393c8 disabled the
    autodoc_index_modules flag for building docs but it wasn't really
    necessary, that change was just to get the module index out of the main
    docs page.

    We want to autodoc the modules so we can view the actual module index in
    the tox -d docs build results, which also tells us if we have correct
    ReST format in doc strings.

    Notes
    -----

    1. Several doc string blocks have to be fixed as part of this to get
       the docs tox job to pass.
    2. A docstring in vhdutilsv2 is updated to remove the math directive
       since that requires the sphinx.ext.pngmath extension which requires
       latex and dvipng packages from the distro - which is overkill for
       what the docstring was actually doing with the math directive.
    3. We exclude autodoc for tests since we don't really care about
       docstrings on unit tests.
    4. We exclude the nova.wsgi.nova-* modules since those won't build with
       autodoc since they can't be imported (there is no
       nova/wsgi/__init__.py module). We could arguably add the __init__.py
       but it's not really necessary for what those scripts are used for.
    5. The sphinx.ext.ifconfig extension is removed since there are no docs
       that use the ifconfig directive.
    6. Update the developer docs to explicitly point out that graphviz must
       be installed prior to running tox -e docs.
    7. Hide doc/source/api/autoindex.rst from the toctree so that we don't
       regress the point of commit bd7e62f796fe951fd42c2edad56e252a0b7393c8.
    8. unused_docs and exclude_trees options are removed from conf.py since
       they are deprecated in Sphinx 1.2.3:

       https://github.com/sphinx-doc/sphinx/blob/1.2.3/sphinx/config.py#L54
    9. Fix imports for moved libvirt volume options.

    Closes-Bug: #1471934

    Change-Id: I946e2f89f2c9fc70e870faaf84e4a8b0fc703344