Various LaTeX symbols missing in Yade documentation

[Robert Caulk]

See [1]. Looks like the builder is missing some important LaTeX libraries.

[1] https://yade-dev.gitlab.io/trunk/formulation.html#variables

[Robert Caulk (rcaulk)]

In fact, it looks like the compiler is ignoring \defs. For example:

\Dtcr [1] is defined by:

\def\Dtcr{\Dt_{\rm cr}}

Meanwhile other typical latex symbols are printing fine.

https://yade-dev.gitlab.io/trunk/formulation.html#estimation-of-by-wave-propagation-speed

[Robert Caulk (rcaulk)]

Ok, after downloading the artifacts from the build, the PDF is built perfectly fine.

This must be an issue on the HTML decoding webside, or something like that.

[Bruno Chareyre (bruno-chareyre)]

The problem appeared in [1].
Maybe it fixed the new sphinx while breaking the older one?
B

[11] https://gitlab.com/yade-dev/trunk/commit/34fb5ffef

[Robert Caulk (rcaulk)]

Ah, it is probably worth moving away from pngmath. I guess we just need to rewrite the \defs to accommodate mathjax?

[Bruno Chareyre (bruno-chareyre)]

Before 34fb5ffef all equations in the html were retrieved from png files.
With "mathjax instead of pngmath" they are replaced by inline code, which actually works for some of them, but not all of them.
You probably found why (#1).

[Bruno Chareyre (bruno-chareyre)]

Maybe http://docs.mathjax.org/en/latest/misc/faq.html#does-mathjax-support-tex-macros

[Robert Caulk (rcaulk)]

Yes, Mathjax seems cleaner. Does it result in better resolution of math on the web. The old equations used to look grainy with pngmath iirc.

Is there a way to test changes and their effect on the website without merging to develop branch?

[Anton Gladky (gladky-anton)]

Hi guys,

I committed those changes. pngmath is not part of sphinx any more,
so the change is really necessary....

Anton

Am Fr., 1. Feb. 2019 um 18:25 Uhr schrieb Robert Caulk
<email address hidden>:
>
> Yes, Mathjax seems cleaner. Does it result in better resolution of math
> on the web. The old equations used to look grainy with pngmath iirc.
>
> Is there a way to test changes and their effect on the website without
> merging to develop branch?
>
> --
> You received this bug notification because you are subscribed to Yade.
> https://bugs.launchpad.net/bugs/1814286
>
> Title:
> Various LaTeX symbols missing in Yade documentation
>
> Status in Yade:
> New
>
> Bug description:
> See [1]. Looks like the builder is missing some important LaTeX
> libraries.
>
>
> [1] https://yade-dev.gitlab.io/trunk/formulation.html#variables
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/yade/+bug/1814286/+subscriptions

[Janek Kozicki (cosurgi)]

> Is there a way to test changes and their effect on the website without
> merging to develop branch?

Yes, a merge request performs a build, then you can click
"artifacts" in the finished build, and then click "browse".

So you can examine output before approving the MR.

Good find, thanks.

if I understand this correctly we have to replace \def with
\newcommand, or accommodate them into formulas.

--
Janek Kozicki

[Chareyre (bruno-chareyre-9)]

This is not website-related. You can simply «make doc» locally and check
html output.
 It is a good opportunity to play with gitlab pipeline, still, but it will
also imply ~8min of full recompile time...
B

Le ven. 1 févr. 2019 18:20, Robert Caulk <email address hidden> a
écrit :

> Yes, Mathjax seems cleaner. Does it result in better resolution of math
> on the web. The old equations used to look grainy with pngmath iirc.
>
> Is there a way to test changes and their effect on the website without
> merging to develop branch?
>
> --
> You received this bug notification because you are a member of Yade
> developers, which is subscribed to Yade.
> https://bugs.launchpad.net/bugs/1814286
>
> Title:
> Various LaTeX symbols missing in Yade documentation
>
> Status in Yade:
> New
>
> Bug description:
> See [1]. Looks like the builder is missing some important LaTeX
> libraries.
>
>
> [1] https://yade-dev.gitlab.io/trunk/formulation.html#variables
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/yade/+bug/1814286/+subscriptions
>
> _______________________________________________
> Mailing list: https://launchpad.net/~yade-dev
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~yade-dev
> More help : https://help.launchpad.net/ListHelp
>
>

[Janek Kozicki (cosurgi)]

Right of course. I am doing this right now every 15 minutes, and I
forgot to tell Robert about that ;)

There is also `make doc/fast` but I didn't notice any difference.

Chareyre said: (by the date of Fri, 01 Feb 2019 19:25:57 -0000)

> This is not website-related. You can simply «make doc» locally and check
> html output.
> It is a good opportunity to play with gitlab pipeline, still, but it will
> also imply ~8min of full recompile time...
> B
>
> Le ven. 1 févr. 2019 18:20, Robert Caulk <email address hidden> a
> écrit :
>
> > Yes, Mathjax seems cleaner. Does it result in better resolution of math
> > on the web. The old equations used to look grainy with pngmath iirc.
> >
> > Is there a way to test changes and their effect on the website without
> > merging to develop branch?

--
Janek Kozicki http://janek.kozicki.pl/ |

[Robert Caulk (rcaulk)]

According to [1], \def and \let should both work fine with mathjax

[1]http://docs.mathjax.org/en/latest/tex.html#tex-commands

I think this has something more to do with the the delimiters [2]. Testing now, will report back.

[2] http://docs.mathjax.org/en/latest/tex.html#defining-tex-macros

[Robert Caulk (rcaulk)]

I am unable to identify the source of the problem. I tried reverting [1] and [2], but the problem persists. I tried changing delimiters, I tried replacing \def with \newcommand (despite the fact that we should not have to, see #12), but the problem persists.

It is worth reiterating that all PDF outputs look perfect.

Ok so I guess I will try these changes by WIP merge request on gitlab instead of make doc on my own computer.

It would help me if someone could help me further understand exactly how we "switched" to a new sphinx here. I am on ubuntu 18.04 and my python-sphinx is the latest version, 1.6.7. Is the new gitlab buildbot using a newer version compiled from sources? Was the 2018 buildbot using a newer version too?

[1]https://gitlab.com/yade-dev/trunk/commit/34fb5ffef
[2]https://gitlab.com/yade-dev/trunk/commit/fda92731e3854d09302cf44fa97316875c16d859

[Janek Kozicki (cosurgi)]

it was this commit actually:

https://gitlab.com/yade-dev/trunk/commit/34fb5ffef7b235274b838583cf8da1b149123e3e

[Janek Kozicki (cosurgi)]

Anton did this when preparing for debian relelase, and there python-sphinx is 1.7.9. Maybe that is related?

[Janek Kozicki (cosurgi)]

Hm, maybe this means that switching to mathjax breaks build on all older ubuntus. I wonder if it is possible to make this switch dependable on python-sphinx version?

[Robert Caulk (rcaulk)]

Strange, I tried reverting that commit [1] and the problem persisted.

[1] https://gitlab.com/yade-dev/trunk/commit/34fb5ffef

[Bruno Chareyre (bruno-chareyre)]

There must be a problem in your compile+buildDoc workflow Robert, since on gitlab side the problem clearly appeared with 34fb5ffef.
Let's discuss it directly.
B

[Janek Kozicki (cosurgi)]

Anton Gladky said: (by the date of Fri, 01 Feb 2019 17:46:29 -0000)

> I committed those changes. pngmath is not part of sphinx any more,
> so the change is really necessary....

Hmm, this search in Buster indicates that pngmath is still there,
am I missing something?

https://packages.debian.org/search?mode=filename&suite=buster&section=all&arch=any&searchon=contents&keywords=pngmath

Also I see that reverting your commit (git revert 34fb5ffef) fixes
the problem, for me at least.

I tried putting imgmath instead of pngmath, this did not help.

Only change (for me at least) was that with imgmath the rendering
errors were more informative: e.g. command \selectfont not working on
\bigO, and other errors.

--
Janek Kozicki

[Janek Kozicki (cosurgi)]

Yes, I had the same errors locally with imgmath: https://yade-dev.gitlab.io/-/trunk/-/jobs/156037536/artifacts/public/formulation.html

[Robert Caulk (rcaulk)]

>>Hmm, this search in Buster indicates that pngmath is still there,
am I missing something?

I guess it exists without support based on this warning thrown by make doc:

"WARNING: sphinx.ext.pngmath has been deprecated. Please use sphinx.ext.imgmath instead."

Have you tried replacing \def with \newcommand locally Janek?

[Anton Gladky (gladky-anton)]

Sorry for causing those problems. Yes, I did not check this
change on older distributions.

Anyway, the pngmath is deprecated in sphinx_1.8, so this
change is necessary for the newer distributions, where 1.8
is used.

If imgmath is not working properly, we need to investigate
the problem deeper.

Regards

Anton

Am Mo., 4. Feb. 2019 um 18:20 Uhr schrieb Robert Caulk
<email address hidden>:
>
> >>Hmm, this search in Buster indicates that pngmath is still there,
> am I missing something?
>
> I guess it exists without support based on this warning thrown by make
> doc:
>
> "WARNING: sphinx.ext.pngmath has been deprecated. Please use
> sphinx.ext.imgmath instead."
>
> Have you tried replacing \def with \newcommand locally Janek?
>
> --
> You received this bug notification because you are subscribed to Yade.
> https://bugs.launchpad.net/bugs/1814286
>
> Title:
> Various LaTeX symbols missing in Yade documentation
>
> Status in Yade:
> New
>
> Bug description:
> See [1]. Looks like the builder is missing some important LaTeX
> libraries.
>
>
> [1] https://yade-dev.gitlab.io/trunk/formulation.html#variables
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/yade/+bug/1814286/+subscriptions

[Janek Kozicki (cosurgi)]

Robert Caulk said: (by the date of Mon, 04 Feb 2019 17:07:16 -0000)

> Have you tried replacing \def with \newcommand locally Janek?

yes, it did not help.

--
Janek Kozicki

[Janek Kozicki (cosurgi)]

I checked on debian Buster (inside chroot). The problem is also present there.

[Bruno Chareyre (bruno-chareyre)]

It sounds like the pragmatic move is to replace the macros by what they mean. Search & replace should work, and I think there are very few pages where those macros are used (if not just one page).
Bruno

[Janek Kozicki (cosurgi)]

Bruno Chareyre said: (by the date of Mon, 04 Feb 2019 18:39:36 -0000)

> It sounds like the pragmatic move is to replace the macros by what they mean. Search & replace should work, and I think there are very few pages where those macros are used (if not just one page).

I've seen an error not related to \def (or \newcommand). \bigO letter
is not found by command \fontstyle.

I suppose there are other incompatibility (between pngmath and
mathjax) errors. Spotting all of them might be difficult.

--
Janek Kozicki

[Janek Kozicki (cosurgi)]

Just have a look at errors produced by imgmath

https://yade-dev.gitlab.io/-/trunk/-/jobs/156037536/artifacts/public/formulation.html

from what I saw they occur in exactly the same places as errors in mathjax (and where pngmath worked). The only difference is that they are more informative, and only some of them relate to \def

[Robert Caulk (rcaulk)]

Alright, I think I figured out the problem. See [1] and the solution [2]. He adds the latex macros (\def, \let, \newcommand, etc) to his layout.html. I will try this today and report back :-)

[1]https://github.com/sphinx-doc/sphinx/issues/726
[2]https://github.com/certik/theoretical-physics/pull/62/commits/36c583ddffd9c9375b8ed0971720237a0ef30e36

[Bruno Chareyre (bruno-chareyre)]

Ok, let us know Robert.

@Anton, do you mean that on some recent version at least the equations compile correctly?

[Janek Kozicki (cosurgi)]

aah, so it's entire preamble. No wonder that my simple \newcommand
replacement did not work. Congrats on finding this! :)

--
Janek Kozicki

[Janek Kozicki (cosurgi)]

It looks better and better! :)

https://yade-dev.gitlab.io/-/trunk/-/jobs/156522519/artifacts/public/formulation.html

now problem is with \ensuremath \sortlines \vec and {\rm ..}.

If I recall correctly \rm is deprecated in latex and used to set roman font.

https://tex.stackexchange.com/questions/151897/always-textrm-never-rm-a-counterexample

[Bruno Chareyre (bruno-chareyre)]

[1] replaces \vec by \boldsymbol using Robert's trick in layout's preamble.
To me it sounds like a dirty trick to define latex conventions in the layout of web pages but there seem to be no other option.

We could not see a problem with \rm, if there is one we can replace it with \text or something, or simply remove.

\ensuremath was removed.

For \sortlines I suggested to upload the images from previous builds to the doc and use them as-is.

Problems solved, it seems(?). Thanks guys.

Bruno

[1] https://gitlab.com/yade-dev/trunk/commit/bdbb374c

[Bruno Chareyre (bruno-chareyre)]

I did not follow all commits.
Were there changes in the latex_preamble used for building the pdf (ther should not, I would think)?
B

[Janek Kozicki (cosurgi)]

in formula (3) and (4) I guess that the problem is with \rm, not something else...

https://yade-dev.gitlab.io/-/trunk/-/jobs/156658986/artifacts/public/formulation.html

in further formulas there is also \curraccel, \nnextvel and \leq

By examining https://gitlab.com/yade-dev/trunk/merge_requests/60/diffs I see that there are no changes in latex_preamble, only the html copy of it.

There is also one Robert's extra commit in Flow stuff. But I guess it's a good commit anyway ;)

[Bruno Chareyre (bruno-chareyre)]

I think we can merge the branch now (without deleting it) since it fixes a number of problem online already. Right?

We can keep working on the same branch for further improvement. That's where the rebase question surfaces again maybe? ;)

[Janek Kozicki (cosurgi)]

agreed. Let's merge it. It will be useful in my branch where I add more text to documentation ;)

[Janek Kozicki (cosurgi)]

I think rebasing will work now :) Unless Robert will happen to have a conflict in lib/triangulation/FlowBoundingSphereLinSolv.ipp which is in the first commit in this branch. Let us know!

[Robert Caulk (rcaulk)]

It’s ok to merge that full commit including flowboundingsphere.

I will make a new merge request with the periodic insertion images.

Le mar. 5 févr. 2019 à 19:15, Janek Kozicki <email address hidden> a
écrit :

> I think rebasing will work now :) Unless Robert will happen to have a
> conflict in lib/triangulation/FlowBoundingSphereLinSolv.ipp which is in
> the first commit in this branch. Let us know!
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1814286
>
> Title:
> Various LaTeX symbols missing in Yade documentation
>
> Status in Yade:
> New
>
> Bug description:
> See [1]. Looks like the builder is missing some important LaTeX
> libraries.
>
>
> [1] https://yade-dev.gitlab.io/trunk/formulation.html#variables
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/yade/+bug/1814286/+subscriptions
>

[Janek Kozicki (cosurgi)]

Moved from https://bugs.launchpad.net/yade/+bug/1814286 to https://gitlab.com/yade-dev/trunk/issues/51