fuel-docs CI doesn't detect ERROR when building PDF

[Dmitry Borodaenko]

This review got a +1 from Fuel CI:
https://review.openstack.org/144373

Even though a number of ERROR lines are present in the output:
https://fuel-jenkins.mirantis.com/job/stackforge-verify-fuel-docs/2260/console

When ERROR is reported by sphinx, the CI job should give the patch -1 even if make's exit code is 0.

[Alexander Charykov (acharykov)]

There is a problem with detecting error, because sphinx returns non-zero code even when there is warnings only. And there is no option to change this behavior.

[Aleksandra Fedorova (bookwar)]

I am not sure how should we deal with "document isn't included in any toctree" warning which present in current master.

Should we filter it out or fix it?

[Dmitry Borodaenko (angdraug)]

At a minimum, I need ERROR messages to trigger a -1 (regardless of whether WARNING messages are handled).

WARNING messages about "pdf/pdf_*.rst" files should all be ignored.

One more optional thing that we used to do was to count the number of WARNING messages before and after the commit is applied, and fail the build of the number of such messages has increased (similar to what OpenStack projects do with pylint errors). I think this is nice to have, but if it takes too much effort to implement I'll be satisfied just having -1 from ERRORs.

[Mateusz Matuszkowiak (mmatuszkowiak)]

Hi,

Dmitry, by ignoring Warning mesgs regarding "pdf/pdf_*.rst" files you mean also to NOT count them?

Please have a look at these two test below job builds, which I ran recently after my first modds:

1) https://pkgs-ci.fuel-infra.org/job/verify-fuel-docs/21/consoleText
This job failed since there was at least ONE error present. Job has ignored (not even printed) these warnings, that refer to the "pdf/pdf_*.rst" files, that's why there is only one WARN present.

2) https://pkgs-ci.fuel-infra.org/job/verify-fuel-docs/20/consoleText
Job succeded since there are no errors. Also only one warning discovered (and the rest are ignored).

regards,
mateusz

[Mateusz Matuszkowiak (mmatuszkowiak)]

Also, Dmitry could you provide a link to an example regarding the "optional thing" related to counting warnings?

[Mateusz Matuszkowiak (mmatuszkowiak)]

First part of required FIX (if ERROR = return -1) has been merged.
Second, related to counting Warns from previous commit and comparing it against current, has been proposed on gerrit review.

[Aleksandra Fedorova (bookwar)]

It works now:

https://fuel-jenkins.mirantis.com/job/verify-fuel-docs/39/console

[Dmitry Borodaenko (angdraug)]

Based on build results for several recent commits (most importantly https://review.openstack.org/147712) it works exactly as requested, nice work!