Comment 7 for bug 1850835

Revision history for this message
Eli Schwartz (eschwartz) wrote : Re: [Bug 1850835] Re: WebEngineContext used before QtWebEngine::initialize() or OpenGL context creation failed.

> @Eli Schwartz
>> Note that the Debian package is rather bad quality either way.
>
> I hear this from you at several places, without any particular explanation.
> Is your aim simple bashing, or are you interested in a cooperation?

I mentioned a couple of points here, I think:
https://bugs.launchpad.net/calibre/+bug/1714107
At least in this bug report, there wasn't a context where it would be
useful to do a technical analysis of things I think could be improved --
a user wanted to successfully run calibre, and probably didn't care at
all about packaging.

Anyway: the build instructions for calibre are described at the bottom
of this page: https://calibre-ebook.com/download_linux

And it links to
https://github.com/kovidgoyal/calibre/blob/master/bypy/sources.json

Some of these are only needed for frozen builds which generate the fully
independent binaries, but for our purposes, let's consider the PyPI
items in this json file. This is, BTW, an important file to track -- if
a dependency is added here, it should most likely be added to downstream
distribution packages as well. Failure to do so results in calibre's
binary .so extensions compiling, and the setup.py install succeeding,
but calibre failing to run once installed due to missing module imports.

The Debian package for calibre has IIRC been rather plagued by missing
dependencies, e.g. html2text, html5lib, pyqtwebengine.

You still have some dreadfully ancient dependency on cherrypy. One of
the major features of calibre 3.0, which was a full major release cycle
ago, and 52 minor feature releases and counting, was the complete
removal of cherrypy due to the content server being reimplemented from
scratch.

You have habitually patched out calibre's vendored dependencies. This is
wrong. I have habitually submitted PRs to calibre to use the system
version of these dependencies -- this benefits all linux distributions
other than Debian. Some of these modules -- feedparser and markdown --
were public API for the recipe subsystem, so calibre includes an import
hook allowing 'import calibre.ebooks.markdown' or
'import calibre.web.feeds.feedparser' to still work (and use the system
version). But debian applies a tree-wide sed to replace the markdown
module, which no longer applies correctly since its targets no longer
exist. It does, however, rewrite the import hook -- now, it hooks into
any caller that does 'import markdown' and causes it to try to import
the '' (empty string) module. Please do not use sed in your build
recipes, patches are more reliable.

Note that while debian shipped python-markdown 3.x and it was not yet
devendored from calibre, this was broken in a different way -- calibre
was not source-compatible with markdown 3.x, my PR to devendor it
included updating calibre to work with 3.x

Debian has an ancient version of python-mechanize available, which
calibre does not work with (upstream development essentially died, then
Kovid received permission to take over development, and added numerous
improvements including python3 support, without which calibre could not
be ported to python3).

Debian patches calibre to use the system mathjax. I submitted a PR to
add a build option to calibre, making it use the system mathjax. This PR
ended up causing several changes to the way calibre distributes/uses
mathjax -- it is no longer correct to symlink it yourself, as calibre
includes a manifest of the files, but the build option will handle this
all correctly. P.S. You'll need to run 'python2 setup.py rapydscript'
after re-vendoring mathjax as symlinks to the system mathjax, since the
content-server's transpiled javascript (in src/pyj/) needs to embed the
HTTP ETag from the manifest.

You'll probably want to package the
https://github.com/kovidgoyal/rapydscript-ng/ project, as it greatly
speeds up the process of recompiling the rapydscript (*.pyj) to
javascript. There is a slow embedded compiler which used to use dukpy
and now uses qtwebengine, though.

...

The problems with the missing modules, at least, should *never* get into
production packages. It's trivial to detect this, simply try importing
everything that calibre requires. In fact, there are explicit unittests
that try to import these needful modules. I also wrote a unittest
specifically to try importing every single module in the calibre
codebase, which proved invaluable during the python3 porting process.

Those unittests would also fail hard on the markdown breakage. I do
acknowledge there is no unittest that would catch the mathjax bundle
being broken.

Please run the unittests, it is *very* important.

I hope this helps. I would be amenable to providing additional advice
should anything come up.
Feel free suggest a better place to take this discussion.
You might find my build script for calibre 4.3.0 (as of today) on
archlinux to provide useful insights:
https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/calibre&id=1dbac5a10567d2c584786156b4d55eda2344d800