Bug #1850835 “WebEngineContext used before QtWebEngine::initiali...” : Bugs : calibre

Revision history for this message

Petr Vorel (petr-vorel) wrote on 2019-10-31:

#1

Actually the error is AttributeError: 'QWebEngineProfile' object has no attribute 'setUrlRequestInterceptor'

Revision history for this message

Eli Schwartz (eschwartz) wrote on 2019-10-31: Re: [Bug 1850835] Re: WebEngineContext used before QtWebEngine::initialize() or OpenGL context creation failed.

#2

Calibre does not officially support Linux distribution packages. Try
uninstalling the Debian package and using the official binaries. If that
works, then you have a Linux distribution issue and must report a bug to
the Debian bugtracker. If it is still broken using the binary install, then
feel free to reopen this bug.

Note that the Debian package is rather bad quality either way.

status invalid

Changed in calibre:
status:	New → Invalid

Revision history for this message

Petr Vorel (petr-vorel) wrote on 2019-10-31:

#3

OK, works with your tarball. I'll report it to Debian bug tracker. I wonder what package is missing.

Revision history for this message

Kovid Goyal (kovid) wrote on 2019-11-01: Re: calibre bug 1850835

#4

That indicates the version of PyQt is too old.

Revision history for this message

Norbert Preining (preining) wrote on 2019-11-22:

#5

Hi all,
yes indeed, Qt (not only PyQt) is at 5.12 in Debian, and setUrlRequestInterceptor was introduced in 5.13. The only place where this is used is in html_writer.py.

I see that the interceptor filters out/blocks all kind of non-local stuff (anything other than file://, and was introduced in e64b8b11fbe for 4.0.0.

Sad that for one single call 5.13 needs to be used.

@Eli Schwartz
> Note that the Debian package is rather bad quality either way.

I hear this from you at several places, without any particular explanation.
Is your aim simple bashing, or are you interested in a cooperation?
I can remove calibre from Debian, if this is your preference, it will hurt a lot of people.
Instead of permanently spreading FUD, please provide a list of things you consider bugs, and
I will look at them.

I am just sick of hearing the same old song from you over and over again.

Thanks

Revision history for this message

Kovid Goyal (kovid) wrote on 2019-11-22:

#6

yes indeed, it is needed to ensure that converting arbitrary HTML/JS to PDF does not allow access to system files. Sadly this is pretty much a hard requirement, no way around it. I dont know if that is the only place in calibre that relies on Qt 5.13 though. You might want to stick with calibre 3.x until you are ready to use Qt 5.13+ in debian.

Revision history for this message

Eli Schwartz (eschwartz) wrote on 2019-11-22: Re: [Bug 1850835] Re: WebEngineContext used before QtWebEngine::initialize() or OpenGL context creation failed.

#7

Download full text (4.9 KiB)

> @Eli Schwartz
>> Note that the Debian package is rather bad quality either way.
>
> I hear this from you at several places, without any particular explanation.
> Is your aim simple bashing, or are you interested in a cooperation?

I mentioned a couple of points here, I think:
https://bugs.launchpad.net/calibre/+bug/1714107
At least in this bug report, there wasn't a context where it would be
useful to do a technical analysis of things I think could be improved --
a user wanted to successfully run calibre, and probably didn't care at
all about packaging.

Anyway: the build instructions for calibre are described at the bottom
of this page: https://calibre-ebook.com/download_linux

And it links to
https://github.com/kovidgoyal/calibre/blob/master/bypy/sources.json

Some of these are only needed for frozen builds which generate the fully
independent binaries, but for our purposes, let's consider the PyPI
items in this json file. This is, BTW, an important file to track -- if
a dependency is added here, it should most likely be added to downstream
distribution packages as well. Failure to do so results in calibre's
binary .so extensions compiling, and the setup.py install succeeding,
but calibre failing to run once installed due to missing module imports.

The Debian package for calibre has IIRC been rather plagued by missing
dependencies, e.g. html2text, html5lib, pyqtwebengine.

You still have some dreadfully ancient dependency on cherrypy. One of
the major features of calibre 3.0, which was a full major release cycle
ago, and 52 minor feature releases and counting, was the complete
removal of cherrypy due to the content server being reimplemented from
scratch.

You have habitually patched out calibre's vendored dependencies. This is
wrong. I have habitually submitted PRs to calibre to use the system
version of these dependencies -- this benefits all linux distributions
other than Debian. Some of these modules -- feedparser and markdown --
were public API for the recipe subsystem, so calibre includes an import
hook allowing 'import calibre.ebooks.markdown' or
'import calibre.web.feeds.feedparser' to still work (and use the system
version). But debian applies a tree-wide sed to replace the markdown
module, which no longer applies correctly since its targets no longer
exist. It does, however, rewrite the import hook -- now, it hooks into
any caller that does 'import markdown' and causes it to try to import
the '' (empty string) module. Please do not use sed in your build
recipes, patches are more reliable.

Note that while debian shipped python-markdown 3.x and it was not yet
devendored from calibre, this was broken in a different way -- calibre
was not source-compatible with markdown 3.x, my PR to devendor it
included updating calibre to work with 3.x

Debian has an ancient version of python-mechanize available, which
calibre does not work with (upstream development essentially died, then
Kovid received permission to take over development, and added numerous
improvements including python3 support, without which calibre could not
be ported to python3).

Debian patches calibre to use the system mathjax. I submitted a PR to
add a build option to...

> @Eli Schwartz
>> Note that the Debian package is rather bad quality either way.
> 
> I hear this from you at several places, without any particular explanation. 
> Is your aim simple bashing, or are you interested in a cooperation?

I mentioned a couple of points here, I think:
https://bugs.launchpad.net/calibre/+bug/1714107
At least in this bug report, there wasn't a context where it would be
useful to do a technical analysis of things I think could be improved --
a user wanted to successfully run calibre, and probably didn't care at
all about packaging.

Anyway: the build instructions for calibre are described at the bottom
of this page: https://calibre-ebook.com/download_linux

And it links to
https://github.com/kovidgoyal/calibre/blob/master/bypy/sources.json

Some of these are only needed for frozen builds which generate the fully
independent binaries, but for our purposes, let's consider the PyPI
items in this json file. This is, BTW, an important file to track -- if
a dependency is added here, it should most likely be added to downstream
distribution packages as well. Failure to do so results in calibre's
binary .so extensions compiling, and the setup.py install succeeding,
but calibre failing to run once installed due to missing module imports.

The Debian package for calibre has IIRC been rather plagued by missing
dependencies, e.g. html2text, html5lib, pyqtwebengine.

You still have some dreadfully ancient dependency on cherrypy. One of
the major features of calibre 3.0, which was a full major release cycle
ago, and 52 minor feature releases and counting, was the complete
removal of cherrypy due to the content server being reimplemented from
scratch.

You have habitually patched out calibre's vendored dependencies. This is
wrong. I have habitually submitted PRs to calibre to use the system
version of these dependencies -- this benefits all linux distributions
other than Debian. Some of these modules -- feedparser and markdown --
were public API for the recipe subsystem, so calibre includes an import
hook allowing 'import calibre.ebooks.markdown' or
'import calibre.web.feeds.feedparser' to still work (and use the system
version). But debian applies a tree-wide sed to replace the markdown
module, which no longer applies correctly since its targets no longer
exist. It does, however, rewrite the import hook -- now, it hooks into
any caller that does 'import markdown' and causes it to try to import
the '' (empty string) module. Please do not use sed in your build
recipes, patches are more reliable.

Note that while debian shipped python-markdown 3.x and it was not yet
devendored from calibre, this was broken in a different way -- calibre
was not source-compatible with markdown 3.x, my PR to devendor it
included updating calibre to work with 3.x

Debian has an ancient version of python-mechanize available, which
calibre does not work with (upstream development essentially died, then
Kovid received permission to take over development, and added numerous
improvements including python3 support, without which calibre could not
be ported to python3).

Debian patches calibre to use the system mathjax. I submitted a PR to
add a build option to calibre, making it use the system mathjax. This PR
ended up causing several changes to the way calibre distributes/uses
mathjax -- it is no longer correct to symlink it yourself, as calibre
includes a manifest of the files, but the build option will handle this
all correctly. P.S. You'll need to run 'python2 setup.py rapydscript'
after re-vendoring mathjax as symlinks to the system mathjax, since the
content-server's transpiled javascript (in src/pyj/) needs to embed the
HTTP ETag from the manifest.

You'll probably want to package the
https://github.com/kovidgoyal/rapydscript-ng/ project, as it greatly
speeds up the process of recompiling the rapydscript (*.pyj) to
javascript. There is a slow embedded compiler which used to use dukpy
and now uses qtwebengine, though.

...

The problems with the missing modules, at least, should *never* get into
production packages. It's trivial to detect this, simply try importing
everything that calibre requires. In fact, there are explicit unittests
that try to import these needful modules. I also wrote a unittest
specifically to try importing every single module in the calibre
codebase, which proved invaluable during the python3 porting process.

Those unittests would also fail hard on the markdown breakage. I do
acknowledge there is no unittest that would catch the mathjax bundle
being broken.

Please run the unittests, it is *very* important.

I hope this helps. I would be amenable to providing additional advice
should anything come up.
Feel free suggest a better place to take this discussion.
You might find my build script for calibre 4.3.0 (as of today) on
archlinux to provide useful insights:
https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/calibre&id=1dbac5a10567d2c584786156b4d55eda2344d800

Revision history for this message

Petr Vorel (petr-vorel) wrote on 2019-11-22:

#8

Hi all,

thank you for working to sort this error.

@Eli Schwartz Thank you for stating what is wrong in Debian, instead of just complaining about bad quality :).
I actually do care about being this fixed for Debian. It's nice you provide compiled packages, but still many people want to use software through distro packages.
I learnt from my role as developer for software, which is being used in some Linux distros and distro packager for other software both sides can gain a lot if they cooperate.

calibre

WebEngineContext used before QtWebEngine::initialize() or OpenGL context creation failed.

Bug Description

Other bug subscribers

Remote bug watches