Bug #1771833 “Süddeutsche” : Bugs : calibre

Revision history for this message

Kovid Goyal (kovid) wrote on 2018-05-17: Re: calibre bug 1771833

#1

Changing the component for this bug.

assignee darko-miletic
status triaged

Changed in calibre:
assignee:	nobody → darkom (darko-miletic)
status:	New → Triaged

Revision history for this message

darkom (darko-miletic) wrote on 2018-06-03:

#2

To test this I would need valid access credentials. If you want this fixed contact me directly on my personal email <email address hidden>

Revision history for this message

Axel (nospamforaxel-1) wrote on 2019-10-28:

#3

Download full text (6.8 KiB)

Apparently this is not limited to paid content. On my setup here, Ubuntu 19.10 with Calibre 4.2, I cannot download news. Not from Süddeutsche nor from Heise.

Is there a solution for that?

Here is the log of Calibre

---
calibre, version 4.2.0 (linux2, embedded-python: True)
Konvertierungsfehler: Fehlgeschlagen: Nachrichten abrufen von Heise Open

Nachrichten abrufen von Heise Open
Conversion options changed from defaults:
output_profile: u'generic_eink'
verbose: 2
Resolved conversion options
calibre version: 4.2.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': u'original',
'chapter': None,
'chapter_mark': u'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'dont_split_on_page_breaks': True,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'epub_flatten': False,
'epub_inline_toc': False,
'epub_toc_at_end': False,
'epub_version': u'2',
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'flow_size': 260,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x7fdf6c4911d0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_default_epub_cover': False,
'no_inline_navbars': False,
'no_svg_cover': False,
'output_profile': <calibre.customize.profiles.GenericEink object at 0x7fdf6c4915d0>,
'page_breaks_before': None,
'prefer_metadata_cover': False,
'preserve_cover_aspect_ratio': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': u'',
'search_replace': None,
'series': None,
'series_index': None,
'smarten_punctuation': False,
'sr1_replace': u'',
'sr1_search': u'',
'sr2_replace': u'',
'sr2_search': u'',
'sr3_replace': u'',
'sr3_search': u'',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:hei...

Apparently this is not limited to paid content. On my setup here, Ubuntu 19.10 with Calibre 4.2, I cannot download news. Not from Süddeutsche nor from Heise.

Is there a solution for that?

Here is the log of Calibre

---
calibre, version 4.2.0 (linux2, embedded-python: True)
Konvertierungsfehler: Fehlgeschlagen: Nachrichten abrufen von Heise Open

Nachrichten abrufen von Heise Open
Conversion options changed from defaults:
  output_profile: u'generic_eink'
  verbose: 2
Resolved conversion options
calibre version: 4.2.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': u'original',
 'chapter': None,
 'chapter_mark': u'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': u'2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x7fdf6c4911d0>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.GenericEink object at 0x7fdf6c4915d0>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': u'',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': u'',
 'sr1_search': u'',
 'sr2_replace': u'',
 'sr2_search': u'',
 'sr3_replace': u'',
 'sr3_search': u'',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:heise_open
Trying to get latest version of recipe: heise_open
Skipping article Samsung: Linux für DeX wird eingestellt (Mo, 21 Okt, 2019 10:38) from feed Heise Open as it is too old.
Skipping article OpenBSD 6.6 mit automatischen System-Updates (Fr, 18 Okt, 2019 14:36) from feed Heise Open as it is too old.
Skipping article Ubuntu 19.10 ist da: Unvollkommener ZFS-Support im Installer (Fr, 18 Okt, 2019 06:30) from feed Heise Open as it is too old.
Skipping article Stream Processing: Cedalo gibt Streamsheets als Open Source frei (Do, 17 Okt, 2019 18:26) from feed Heise Open as it is too old.
Skipping article IBM verzeichnet Gewinneinbruch und Umsatzrückgang (Do, 17 Okt, 2019 08:17) from feed Heise Open as it is too old.
Skipping article Linux-Desktop KDE Plasma 5.17 punktet mit schnellem Start (Mi, 16 Okt, 2019 17:35) from feed Heise Open as it is too old.
Skipping article Linux: Bug in sudo verleiht Root-Rechte (Mi, 16 Okt, 2019 11:07) from feed Heise Open as it is too old.
Skipping article Smartphone mit Linux (Mi, 16 Okt, 2019 06:00) from feed Heise Open as it is too old.
Skipping article "Humanistische Antwort auf das Silicon Valley“ (Di, 15 Okt, 2019 07:30) from feed Heise Open as it is too old.
Skipping article Verbesserter Grafiktreiber für Raspberry Pi 4 in Arbeit (So, 13 Okt, 2019 13:57) from feed Heise Open as it is too old.
Skipping article OpenSSH 8.1: Schutz vor RAM-Seitenkanalangriffen und Sicherheits-Bugfix (Fr, 11 Okt, 2019 13:29) from feed Heise Open as it is too old.
Skipping article heise+ | Blender-Workshop, Teil 1: Vom einfachen 3D-Objekt zum realistischen Rendering (Fr, 11 Okt, 2019 08:00) from feed Heise Open as it is too old.
Skipping article heise+ | VoIP-Installation mit Kamailio aufbauen (Mi, 09 Okt, 2019 11:17) from feed Heise Open as it is too old.
Skipping article Missing Link: Was ist eigentlich aus alten Smartphone-Betriebssystemen geworden? (So, 06 Okt, 2019 08:05) from feed Heise Open as it is too old.
Skipping article PostgreSQL optimiert in Version 12 Indizes und Partitionen (Fr, 04 Okt, 2019 09:50) from feed Heise Open as it is too old.
Skipping article China plant PCs und Notebooks mit chinesischem ARM-Chip FT-2000/4 (Mi, 02 Okt, 2019 15:16) from feed Heise Open as it is too old.
Skipping article heise+ | Skriptgesteuerte Backups in Bareos (Mi, 02 Okt, 2019 10:33) from feed Heise Open as it is too old.
Skipping article heise+ | Machine Learning: Frameworks und Services im Vergleich (Di, 01 Okt, 2019 08:49) from feed Heise Open as it is too old.
Skipping article Highlights von Linux 5.4: exFAT, I/O-Controller und Treiber für neue AMD-GPUs (Di, 01 Okt, 2019 06:47) from feed Heise Open as it is too old.
Skipping article Fernwartungschip Aspeed AST2600 bringt deutlich mehr Rechenleistung (Mo, 30 Sep, 2019 17:29) from feed Heise Open as it is too old.
Skipping article Mesa 19.2 veröffentlicht: Linux-3D-Treiber für neue AMD-Grafikkarten (Fr, 27 Sep, 2019 17:55) from feed Heise Open as it is too old.
Skipping article heise+ | Linux-Container: Docker ersetzen mit Podman (Fr, 27 Sep, 2019 08:00) from feed Heise Open as it is too old.
Skipping article Gnome hat Patentklage wegen Fotomanager Shotwell am Hals (Do, 26 Sep, 2019 19:05) from feed Heise Open as it is too old.
Failed to download cover
Traceback (most recent call last):
  File "site-packages/calibre/web/feeds/news.py", line 1331, in download_cover
  File "site-packages/calibre/web/feeds/news.py", line 1304, in _download_cover
  File "site-packages/mechanize/_mechanize.py", line 253, in open
  File "site-packages/mechanize/_mechanize.py", line 309, in _mech_open
httperror_seek_wrapper: HTTP Error 404: Not Found

Synthesizing mastheadImage

--------------------------------------------------------------------------------------------

Revision history for this message

Axel (nospamforaxel-1) wrote on 2019-11-11:

#4

Hi all,

Is there any clue how to solve this? I recognized that this bug does not exists on my Desktop-Computer runnung Ubuntu 18.04. However it happens on my Laptop runnung 19.10.

Revision history for this message

Kovid Goyal (kovid) wrote on 2019-11-11:

#5

Heise has nothing to do with this, the log you posted for it is incomplete. As for the original recipe the darko asked for login credentials for it, but no one responded.

Revision history for this message

Harald Riedel (lisux) wrote on 2019-11-11: Re: [Bug 1771833] Re: Süddeutsche

#6

Download full text (5.4 KiB)

What about heise? It is about "Süddeutsche Zeitung«
I hat given my credential at the time I posted the bug.

Am 11.11.2019 um 10:45 schrieb Kovid Goyal <email address hidden>:

Heise has nothing to do with this, the log you posted for it is
incomplete. As for the original recipe the darko asked for login
credentials for it, but no one responded.

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1771833

Title:
Süddeutsche

Status in calibre:
Triaged

Bug description:
# Calibre Version:
Calibre 3.23.0 - OS X Version.
(Testet with 3.10 too, same result).

# Hint:

Maybe "Süddeutsche Zeitung" changed something

---8<--- Snip Snip Snip ---8<---

# Call and Log:

Süddeutsche Zeitung
Erstellt von Darko Miletic
News from Germany. Access to paid content.

---8<--- Snip Snip Snip ---8<---

After pressing the Button "Jetzt downloaden" - Waiting -pressing the Button "Speichern" -> nothing happend.
The log says:

---8<--- Snip Snip Snip ---8<---

calibre, version 3.23.0 (darwin, embedded-python: True)
Konvertierungsfehler: Fehlgeschlagen: Nachrichten abrufen von Süddeutsche Zeitung

Nachrichten abrufen von Süddeutsche Zeitung
Conversion options changed from defaults:
   output_profile: u'kindle_oasis'
   verbose: 2
Resolved conversion options
calibre version: 3.23.0
{'asciiize': False,
  'author_sort': None,
  'authors': None,
  'base_font_size': 0,
  'book_producer': None,
  'change_justification': 'original',
  'chapter': None,
  'chapter_mark': 'pagebreak',
  'comments': None,
  'cover': None,
  'debug_pipeline': None,
  'dehyphenate': True,
  'delete_blank_paragraphs': True,
  'disable_font_rescaling': False,
  'dont_compress': False,
  'dont_download_recipe': False,
  'duplicate_links_in_toc': False,
  'embed_all_fonts': False,
  'embed_font_family': None,
  'enable_heuristics': False,
  'expand_css': False,
  'extra_css': None,
  'extract_to': None,
  'filter_css': None,
  'fix_indents': True,
  'font_size_mapping': None,
  'format_scene_breaks': True,
  'html_unwrap_factor': 0.4,
  'input_encoding': None,
  'input_profile': <calibre.customize.profiles.InputProfile object at 0x11874d390>,
  'insert_blank_line': False,
  'insert_blank_line_size': 0.5,
  'insert_metadata': False,
  'isbn': None,
  'italicize_common_cases': True,
  'keep_ligatures': False,
  'language': None,
  'level1_toc': None,
  'level2_toc': None,
  'level3_toc': None,
  'line_height': 0,
  'linearize_tables': False,
  'lrf': False,
  'margin_bottom': 5.0,
  'margin_left': 5.0,
  'margin_right': 5.0,
  'margin_top': 5.0,
  'markup_chapter_headings': True,
  'max_toc_links': 50,
  'minimum_line_height': 120.0,
  'mobi_file_type': 'old',
  'mobi_ignore_margins': False,
  'mobi_keep_original_images': False,
  'mobi_toc_at_start': False,
  'no_chapters_in_toc': False,
  'no_inline_navbars': True,
  'no_inline_toc': False,
  'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x11874db10>,
  'page_breaks_before': None,
  'personal_doc': '[PDOC]',
  'prefer_author_sort': False,
  'prefer_metadata_cover': False,
  'pretty_print': False,
  'pubdate': None,
  'publisher...

What about heise? It is about "Süddeutsche Zeitung«
I hat given my credential at the time I posted the bug.

Am 11.11.2019 um 10:45 schrieb Kovid Goyal <1771833@bugs.launchpad.net>:

Heise has nothing to do with this, the log you posted for it is
incomplete. As for the original recipe the darko asked for login
credentials for it, but no one responded.

-- 
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1771833

Title:
 Süddeutsche

Status in calibre:
 Triaged

Bug description:
 # Calibre Version:
 Calibre 3.23.0 - OS X Version.
 (Testet with 3.10 too, same result).

# Hint:

Maybe "Süddeutsche Zeitung" changed something

---8<--- Snip Snip Snip ---8<---

# Call and Log:

Süddeutsche Zeitung
 Erstellt von Darko Miletic
 News from Germany. Access to paid content.

---8<--- Snip Snip Snip ---8<---

After pressing the Button "Jetzt downloaden" - Waiting -pressing the Button "Speichern" -> nothing happend. 
 The log says:

---8<--- Snip Snip Snip ---8<---

calibre, version 3.23.0 (darwin, embedded-python: True)
 Konvertierungsfehler: Fehlgeschlagen: Nachrichten abrufen von Süddeutsche Zeitung

Nachrichten abrufen von Süddeutsche Zeitung
 Conversion options changed from defaults:
   output_profile: u'kindle_oasis'
   verbose: 2
 Resolved conversion options
 calibre version: 3.23.0
 {'asciiize': False,
  'author_sort': None,
  'authors': None,
  'base_font_size': 0,
  'book_producer': None,
  'change_justification': 'original',
  'chapter': None,
  'chapter_mark': 'pagebreak',
  'comments': None,
  'cover': None,
  'debug_pipeline': None,
  'dehyphenate': True,
  'delete_blank_paragraphs': True,
  'disable_font_rescaling': False,
  'dont_compress': False,
  'dont_download_recipe': False,
  'duplicate_links_in_toc': False,
  'embed_all_fonts': False,
  'embed_font_family': None,
  'enable_heuristics': False,
  'expand_css': False,
  'extra_css': None,
  'extract_to': None,
  'filter_css': None,
  'fix_indents': True,
  'font_size_mapping': None,
  'format_scene_breaks': True,
  'html_unwrap_factor': 0.4,
  'input_encoding': None,
  'input_profile': <calibre.customize.profiles.InputProfile object at 0x11874d390>,
  'insert_blank_line': False,
  'insert_blank_line_size': 0.5,
  'insert_metadata': False,
  'isbn': None,
  'italicize_common_cases': True,
  'keep_ligatures': False,
  'language': None,
  'level1_toc': None,
  'level2_toc': None,
  'level3_toc': None,
  'line_height': 0,
  'linearize_tables': False,
  'lrf': False,
  'margin_bottom': 5.0,
  'margin_left': 5.0,
  'margin_right': 5.0,
  'margin_top': 5.0,
  'markup_chapter_headings': True,
  'max_toc_links': 50,
  'minimum_line_height': 120.0,
  'mobi_file_type': 'old',
  'mobi_ignore_margins': False,
  'mobi_keep_original_images': False,
  'mobi_toc_at_start': False,
  'no_chapters_in_toc': False,
  'no_inline_navbars': True,
  'no_inline_toc': False,
  'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x11874db10>,
  'page_breaks_before': None,
  'personal_doc': '[PDOC]',
  'prefer_author_sort': False,
  'prefer_metadata_cover': False,
  'pretty_print': False,
  'pubdate': None,
  'publisher': None,
  'rating': None,
  'read_metadata_from_opf': None,
  'remove_fake_margins': True,
  'remove_first_image': False,
  'remove_paragraph_spacing': False,
  'remove_paragraph_spacing_indent_size': 1.5,
  'renumber_headings': True,
  'replace_scene_breaks': '',
  'search_replace': None,
  'series': None,
  'series_index': None,
  'share_not_sync': False,
  'smarten_punctuation': False,
  'sr1_replace': '',
  'sr1_search': '',
  'sr2_replace': '',
  'sr2_search': '',
  'sr3_replace': '',
  'sr3_search': '',
  'start_reading_at': None,
  'subset_embedded_fonts': False,
  'tags': None,
  'test': False,
  'timestamp': None,
  'title': None,
  'title_sort': None,
  'toc_filter': None,
  'toc_threshold': 6,
  'toc_title': None,
  'transform_css_rules': None,
  'unsmarten_punctuation': False,
  'unwrap_lines': True,
  'use_auto_toc': False,
  'verbose': 2}
 Python function terminated unexpectedly: HTTP Error 404: Not Found
 InputFormatPlugin: Recipe Input running
 Using custom recipe
 Traceback (most recent call last):
   File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 154, in main
     return run_entry_point()
   File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 114, in run_entry_point
     return getattr(pmod, func)()
   File "site-packages/calibre/utils/ipc/worker.py", line 195, in main
   File "site-packages/calibre/gui2/convert/gui_conversion.py", line 26, in gui_convert
   File "site-packages/calibre/ebooks/conversion/plumber.py", line 1088, in run
   File "site-packages/calibre/customize/conversion.py", line 245, in __call__
   File "site-packages/calibre/ebooks/conversion/plugins/recipe_input.py", line 119, in convert
   File "site-packages/calibre/web/feeds/news.py", line 1018, in download
   File "site-packages/calibre/web/feeds/news.py", line 1185, in build_index
   File "<string>", line 97, in parse_index
   File "site-packages/calibre/web/feeds/news.py", line 675, in index_to_soup
   File "site-packages/mechanize/_mechanize.py", line 239, in open_novisit
   File "site-packages/mechanize/_mechanize.py", line 310, in _mech_open
 mechanize._response.httperror_seek_wrapper: HTTP Error 404: Not Found

To manage notifications about this bug go to:
https://bugs.launchpad.net/calibre/+bug/1771833/+subscriptions

Revision history for this message

Kovid Goyal (kovid) wrote on 2019-11-11:

#7

I just tried heise it works fine.

Revision history for this message

Harald Riedel (lisux) wrote on 2019-11-11:

#8

Download full text (5.2 KiB)

With what credential do you test?

Am 11.11.2019 um 13:01 schrieb Kovid Goyal <email address hidden>:

I just tried heise it works fine.

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1771833

Title:
Süddeutsche

Status in calibre:
Triaged

Bug description:
# Calibre Version:
Calibre 3.23.0 - OS X Version.
(Testet with 3.10 too, same result).

# Hint:

Maybe "Süddeutsche Zeitung" changed something

---8<--- Snip Snip Snip ---8<---

# Call and Log:

Süddeutsche Zeitung
Erstellt von Darko Miletic
News from Germany. Access to paid content.

---8<--- Snip Snip Snip ---8<---

After pressing the Button "Jetzt downloaden" - Waiting -pressing the Button "Speichern" -> nothing happend.
The log says:

---8<--- Snip Snip Snip ---8<---

calibre, version 3.23.0 (darwin, embedded-python: True)
Konvertierungsfehler: Fehlgeschlagen: Nachrichten abrufen von Süddeutsche Zeitung

Nachrichten abrufen von Süddeutsche Zeitung
Conversion options changed from defaults:
   output_profile: u'kindle_oasis'
   verbose: 2
Resolved conversion options
calibre version: 3.23.0
{'asciiize': False,
  'author_sort': None,
  'authors': None,
  'base_font_size': 0,
  'book_producer': None,
  'change_justification': 'original',
  'chapter': None,
  'chapter_mark': 'pagebreak',
  'comments': None,
  'cover': None,
  'debug_pipeline': None,
  'dehyphenate': True,
  'delete_blank_paragraphs': True,
  'disable_font_rescaling': False,
  'dont_compress': False,
  'dont_download_recipe': False,
  'duplicate_links_in_toc': False,
  'embed_all_fonts': False,
  'embed_font_family': None,
  'enable_heuristics': False,
  'expand_css': False,
  'extra_css': None,
  'extract_to': None,
  'filter_css': None,
  'fix_indents': True,
  'font_size_mapping': None,
  'format_scene_breaks': True,
  'html_unwrap_factor': 0.4,
  'input_encoding': None,
  'input_profile': <calibre.customize.profiles.InputProfile object at 0x11874d390>,
  'insert_blank_line': False,
  'insert_blank_line_size': 0.5,
  'insert_metadata': False,
  'isbn': None,
  'italicize_common_cases': True,
  'keep_ligatures': False,
  'language': None,
  'level1_toc': None,
  'level2_toc': None,
  'level3_toc': None,
  'line_height': 0,
  'linearize_tables': False,
  'lrf': False,
  'margin_bottom': 5.0,
  'margin_left': 5.0,
  'margin_right': 5.0,
  'margin_top': 5.0,
  'markup_chapter_headings': True,
  'max_toc_links': 50,
  'minimum_line_height': 120.0,
  'mobi_file_type': 'old',
  'mobi_ignore_margins': False,
  'mobi_keep_original_images': False,
  'mobi_toc_at_start': False,
  'no_chapters_in_toc': False,
  'no_inline_navbars': True,
  'no_inline_toc': False,
  'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x11874db10>,
  'page_breaks_before': None,
  'personal_doc': '[PDOC]',
  'prefer_author_sort': False,
  'prefer_metadata_cover': False,
  'pretty_print': False,
  'pubdate': None,
  'publisher': None,
  'rating': None,
  'read_metadata_from_opf': None,
  'remove_fake_margins': True,
  'remove_first_image': False,
  'remove_paragraph_spacing': False,
  'remove_paragraph_spacing_indent_size': 1.5,
  're...

With what credential do you test?

Am 11.11.2019 um 13:01 schrieb Kovid Goyal <1771833@bugs.launchpad.net>:

I just tried heise it works fine.

-- 
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1771833

Title:
 Süddeutsche

Status in calibre:
 Triaged

Bug description:
 # Calibre Version:
 Calibre 3.23.0 - OS X Version.
 (Testet with 3.10 too, same result).

# Hint:

Maybe "Süddeutsche Zeitung" changed something

---8<--- Snip Snip Snip ---8<---

# Call and Log:

Süddeutsche Zeitung
 Erstellt von Darko Miletic
 News from Germany. Access to paid content.

---8<--- Snip Snip Snip ---8<---

After pressing the Button "Jetzt downloaden" - Waiting -pressing the Button "Speichern" -> nothing happend. 
 The log says:

---8<--- Snip Snip Snip ---8<---

calibre, version 3.23.0 (darwin, embedded-python: True)
 Konvertierungsfehler: Fehlgeschlagen: Nachrichten abrufen von Süddeutsche Zeitung

Nachrichten abrufen von Süddeutsche Zeitung
 Conversion options changed from defaults:
   output_profile: u'kindle_oasis'
   verbose: 2
 Resolved conversion options
 calibre version: 3.23.0
 {'asciiize': False,
  'author_sort': None,
  'authors': None,
  'base_font_size': 0,
  'book_producer': None,
  'change_justification': 'original',
  'chapter': None,
  'chapter_mark': 'pagebreak',
  'comments': None,
  'cover': None,
  'debug_pipeline': None,
  'dehyphenate': True,
  'delete_blank_paragraphs': True,
  'disable_font_rescaling': False,
  'dont_compress': False,
  'dont_download_recipe': False,
  'duplicate_links_in_toc': False,
  'embed_all_fonts': False,
  'embed_font_family': None,
  'enable_heuristics': False,
  'expand_css': False,
  'extra_css': None,
  'extract_to': None,
  'filter_css': None,
  'fix_indents': True,
  'font_size_mapping': None,
  'format_scene_breaks': True,
  'html_unwrap_factor': 0.4,
  'input_encoding': None,
  'input_profile': <calibre.customize.profiles.InputProfile object at 0x11874d390>,
  'insert_blank_line': False,
  'insert_blank_line_size': 0.5,
  'insert_metadata': False,
  'isbn': None,
  'italicize_common_cases': True,
  'keep_ligatures': False,
  'language': None,
  'level1_toc': None,
  'level2_toc': None,
  'level3_toc': None,
  'line_height': 0,
  'linearize_tables': False,
  'lrf': False,
  'margin_bottom': 5.0,
  'margin_left': 5.0,
  'margin_right': 5.0,
  'margin_top': 5.0,
  'markup_chapter_headings': True,
  'max_toc_links': 50,
  'minimum_line_height': 120.0,
  'mobi_file_type': 'old',
  'mobi_ignore_margins': False,
  'mobi_keep_original_images': False,
  'mobi_toc_at_start': False,
  'no_chapters_in_toc': False,
  'no_inline_navbars': True,
  'no_inline_toc': False,
  'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x11874db10>,
  'page_breaks_before': None,
  'personal_doc': '[PDOC]',
  'prefer_author_sort': False,
  'prefer_metadata_cover': False,
  'pretty_print': False,
  'pubdate': None,
  'publisher': None,
  'rating': None,
  'read_metadata_from_opf': None,
  'remove_fake_margins': True,
  'remove_first_image': False,
  'remove_paragraph_spacing': False,
  'remove_paragraph_spacing_indent_size': 1.5,
  'renumber_headings': True,
  'replace_scene_breaks': '',
  'search_replace': None,
  'series': None,
  'series_index': None,
  'share_not_sync': False,
  'smarten_punctuation': False,
  'sr1_replace': '',
  'sr1_search': '',
  'sr2_replace': '',
  'sr2_search': '',
  'sr3_replace': '',
  'sr3_search': '',
  'start_reading_at': None,
  'subset_embedded_fonts': False,
  'tags': None,
  'test': False,
  'timestamp': None,
  'title': None,
  'title_sort': None,
  'toc_filter': None,
  'toc_threshold': 6,
  'toc_title': None,
  'transform_css_rules': None,
  'unsmarten_punctuation': False,
  'unwrap_lines': True,
  'use_auto_toc': False,
  'verbose': 2}
 Python function terminated unexpectedly: HTTP Error 404: Not Found
 InputFormatPlugin: Recipe Input running
 Using custom recipe
 Traceback (most recent call last):
   File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 154, in main
     return run_entry_point()
   File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 114, in run_entry_point
     return getattr(pmod, func)()
   File "site-packages/calibre/utils/ipc/worker.py", line 195, in main
   File "site-packages/calibre/gui2/convert/gui_conversion.py", line 26, in gui_convert
   File "site-packages/calibre/ebooks/conversion/plumber.py", line 1088, in run
   File "site-packages/calibre/customize/conversion.py", line 245, in __call__
   File "site-packages/calibre/ebooks/conversion/plugins/recipe_input.py", line 119, in convert
   File "site-packages/calibre/web/feeds/news.py", line 1018, in download
   File "site-packages/calibre/web/feeds/news.py", line 1185, in build_index
   File "<string>", line 97, in parse_index
   File "site-packages/calibre/web/feeds/news.py", line 675, in index_to_soup
   File "site-packages/mechanize/_mechanize.py", line 239, in open_novisit
   File "site-packages/mechanize/_mechanize.py", line 310, in _mech_open
 mechanize._response.httperror_seek_wrapper: HTTP Error 404: Not Found

To manage notifications about this bug go to:
https://bugs.launchpad.net/calibre/+bug/1771833/+subscriptions

Revision history for this message

Kovid Goyal (kovid) wrote on 2019-11-11: Re: calibre bug 1771833

#9

heise does not need credentials

Revision history for this message

Harald Riedel (lisux) wrote on 2019-11-11: Re: [Bug 1771833] calibre bug 1771833

#10

Download full text (5.2 KiB)

With what credential do you test "Süddeutsche Zeitung«?

Am 11.11.2019 um 13:57 schrieb Kovid Goyal <email address hidden>:

heise does not need credentials

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1771833

Title:
Süddeutsche

Status in calibre:
Triaged

Bug description:
# Calibre Version:
Calibre 3.23.0 - OS X Version.
(Testet with 3.10 too, same result).

# Hint:

Maybe "Süddeutsche Zeitung" changed something

---8<--- Snip Snip Snip ---8<---

# Call and Log:

Süddeutsche Zeitung
Erstellt von Darko Miletic
News from Germany. Access to paid content.

---8<--- Snip Snip Snip ---8<---

After pressing the Button "Jetzt downloaden" - Waiting -pressing the Button "Speichern" -> nothing happend.
The log says:

---8<--- Snip Snip Snip ---8<---

calibre, version 3.23.0 (darwin, embedded-python: True)
Konvertierungsfehler: Fehlgeschlagen: Nachrichten abrufen von Süddeutsche Zeitung

Nachrichten abrufen von Süddeutsche Zeitung
Conversion options changed from defaults:
   output_profile: u'kindle_oasis'
   verbose: 2
Resolved conversion options
calibre version: 3.23.0
{'asciiize': False,
  'author_sort': None,
  'authors': None,
  'base_font_size': 0,
  'book_producer': None,
  'change_justification': 'original',
  'chapter': None,
  'chapter_mark': 'pagebreak',
  'comments': None,
  'cover': None,
  'debug_pipeline': None,
  'dehyphenate': True,
  'delete_blank_paragraphs': True,
  'disable_font_rescaling': False,
  'dont_compress': False,
  'dont_download_recipe': False,
  'duplicate_links_in_toc': False,
  'embed_all_fonts': False,
  'embed_font_family': None,
  'enable_heuristics': False,
  'expand_css': False,
  'extra_css': None,
  'extract_to': None,
  'filter_css': None,
  'fix_indents': True,
  'font_size_mapping': None,
  'format_scene_breaks': True,
  'html_unwrap_factor': 0.4,
  'input_encoding': None,
  'input_profile': <calibre.customize.profiles.InputProfile object at 0x11874d390>,
  'insert_blank_line': False,
  'insert_blank_line_size': 0.5,
  'insert_metadata': False,
  'isbn': None,
  'italicize_common_cases': True,
  'keep_ligatures': False,
  'language': None,
  'level1_toc': None,
  'level2_toc': None,
  'level3_toc': None,
  'line_height': 0,
  'linearize_tables': False,
  'lrf': False,
  'margin_bottom': 5.0,
  'margin_left': 5.0,
  'margin_right': 5.0,
  'margin_top': 5.0,
  'markup_chapter_headings': True,
  'max_toc_links': 50,
  'minimum_line_height': 120.0,
  'mobi_file_type': 'old',
  'mobi_ignore_margins': False,
  'mobi_keep_original_images': False,
  'mobi_toc_at_start': False,
  'no_chapters_in_toc': False,
  'no_inline_navbars': True,
  'no_inline_toc': False,
  'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x11874db10>,
  'page_breaks_before': None,
  'personal_doc': '[PDOC]',
  'prefer_author_sort': False,
  'prefer_metadata_cover': False,
  'pretty_print': False,
  'pubdate': None,
  'publisher': None,
  'rating': None,
  'read_metadata_from_opf': None,
  'remove_fake_margins': True,
  'remove_first_image': False,
  'remove_paragraph_spacing': False,
  'remove_paragraph_spacing_ind...

With what credential do you test "Süddeutsche Zeitung«?

Am 11.11.2019 um 13:57 schrieb Kovid Goyal <1771833@bugs.launchpad.net>:

heise does not need credentials

-- 
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1771833

Title:
 Süddeutsche

Status in calibre:
 Triaged

Bug description:
 # Calibre Version:
 Calibre 3.23.0 - OS X Version.
 (Testet with 3.10 too, same result).

# Hint:

Maybe "Süddeutsche Zeitung" changed something

---8<--- Snip Snip Snip ---8<---

# Call and Log:

Süddeutsche Zeitung
 Erstellt von Darko Miletic
 News from Germany. Access to paid content.

---8<--- Snip Snip Snip ---8<---

After pressing the Button "Jetzt downloaden" - Waiting -pressing the Button "Speichern" -> nothing happend. 
 The log says:

---8<--- Snip Snip Snip ---8<---

calibre, version 3.23.0 (darwin, embedded-python: True)
 Konvertierungsfehler: Fehlgeschlagen: Nachrichten abrufen von Süddeutsche Zeitung

Nachrichten abrufen von Süddeutsche Zeitung
 Conversion options changed from defaults:
   output_profile: u'kindle_oasis'
   verbose: 2
 Resolved conversion options
 calibre version: 3.23.0
 {'asciiize': False,
  'author_sort': None,
  'authors': None,
  'base_font_size': 0,
  'book_producer': None,
  'change_justification': 'original',
  'chapter': None,
  'chapter_mark': 'pagebreak',
  'comments': None,
  'cover': None,
  'debug_pipeline': None,
  'dehyphenate': True,
  'delete_blank_paragraphs': True,
  'disable_font_rescaling': False,
  'dont_compress': False,
  'dont_download_recipe': False,
  'duplicate_links_in_toc': False,
  'embed_all_fonts': False,
  'embed_font_family': None,
  'enable_heuristics': False,
  'expand_css': False,
  'extra_css': None,
  'extract_to': None,
  'filter_css': None,
  'fix_indents': True,
  'font_size_mapping': None,
  'format_scene_breaks': True,
  'html_unwrap_factor': 0.4,
  'input_encoding': None,
  'input_profile': <calibre.customize.profiles.InputProfile object at 0x11874d390>,
  'insert_blank_line': False,
  'insert_blank_line_size': 0.5,
  'insert_metadata': False,
  'isbn': None,
  'italicize_common_cases': True,
  'keep_ligatures': False,
  'language': None,
  'level1_toc': None,
  'level2_toc': None,
  'level3_toc': None,
  'line_height': 0,
  'linearize_tables': False,
  'lrf': False,
  'margin_bottom': 5.0,
  'margin_left': 5.0,
  'margin_right': 5.0,
  'margin_top': 5.0,
  'markup_chapter_headings': True,
  'max_toc_links': 50,
  'minimum_line_height': 120.0,
  'mobi_file_type': 'old',
  'mobi_ignore_margins': False,
  'mobi_keep_original_images': False,
  'mobi_toc_at_start': False,
  'no_chapters_in_toc': False,
  'no_inline_navbars': True,
  'no_inline_toc': False,
  'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x11874db10>,
  'page_breaks_before': None,
  'personal_doc': '[PDOC]',
  'prefer_author_sort': False,
  'prefer_metadata_cover': False,
  'pretty_print': False,
  'pubdate': None,
  'publisher': None,
  'rating': None,
  'read_metadata_from_opf': None,
  'remove_fake_margins': True,
  'remove_first_image': False,
  'remove_paragraph_spacing': False,
  'remove_paragraph_spacing_indent_size': 1.5,
  'renumber_headings': True,
  'replace_scene_breaks': '',
  'search_replace': None,
  'series': None,
  'series_index': None,
  'share_not_sync': False,
  'smarten_punctuation': False,
  'sr1_replace': '',
  'sr1_search': '',
  'sr2_replace': '',
  'sr2_search': '',
  'sr3_replace': '',
  'sr3_search': '',
  'start_reading_at': None,
  'subset_embedded_fonts': False,
  'tags': None,
  'test': False,
  'timestamp': None,
  'title': None,
  'title_sort': None,
  'toc_filter': None,
  'toc_threshold': 6,
  'toc_title': None,
  'transform_css_rules': None,
  'unsmarten_punctuation': False,
  'unwrap_lines': True,
  'use_auto_toc': False,
  'verbose': 2}
 Python function terminated unexpectedly: HTTP Error 404: Not Found
 InputFormatPlugin: Recipe Input running
 Using custom recipe
 Traceback (most recent call last):
   File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 154, in main
     return run_entry_point()
   File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 114, in run_entry_point
     return getattr(pmod, func)()
   File "site-packages/calibre/utils/ipc/worker.py", line 195, in main
   File "site-packages/calibre/gui2/convert/gui_conversion.py", line 26, in gui_convert
   File "site-packages/calibre/ebooks/conversion/plumber.py", line 1088, in run
   File "site-packages/calibre/customize/conversion.py", line 245, in __call__
   File "site-packages/calibre/ebooks/conversion/plugins/recipe_input.py", line 119, in convert
   File "site-packages/calibre/web/feeds/news.py", line 1018, in download
   File "site-packages/calibre/web/feeds/news.py", line 1185, in build_index
   File "<string>", line 97, in parse_index
   File "site-packages/calibre/web/feeds/news.py", line 675, in index_to_soup
   File "site-packages/mechanize/_mechanize.py", line 239, in open_novisit
   File "site-packages/mechanize/_mechanize.py", line 310, in _mech_open
 mechanize._response.httperror_seek_wrapper: HTTP Error 404: Not Found

To manage notifications about this bug go to:
https://bugs.launchpad.net/calibre/+bug/1771833/+subscriptions

Revision history for this message

Axel (nospamforaxel-1) wrote on 2019-11-11:

#11

I think, at least in my case, this bug is not limited to the "Süddeutsche Zeitung", therefore I mentioned "heise online" as an example to demonstrate that it also does not depend on a login. I tried also "Zeit online", same thing.

Correct me if I'm wrong, but this would be the most suspicious part in the logs:

---
Traceback (most recent call last):
  File "site-packages/calibre/web/feeds/news.py", line 1331, in download_cover
  File "site-packages/calibre/web/feeds/news.py", line 1304, in _download_cover
  File "site-packages/mechanize/_mechanize.py", line 253, in open
  File "site-packages/mechanize/_mechanize.py", line 309, in _mech_open
httperror_seek_wrapper: HTTP Error 404: Not Found
---

And yes, no problem at all on my desktop computer with Ubuntu 18.04. My desktop (works) and my laptop (does not work) share even the same calibre library that is synchronized over Nextcloud. I don't know whether this affects the bug, but probably not.

Revision history for this message

Kovid Goyal (kovid) wrote on 2019-11-12: Re: calibre bug 1771833

#12

that's the cover download failing it wont affect overall download.

Revision history for this message

Axel (nospamforaxel-1) wrote on 2019-11-13:

#13

Perhaps I should mention that in my laptop, where this bug is showing up, calibre is installed as flatpak. On my desktop, where everything runs smoothly, it is a "normal" installation with the command from the calibre website.

Is there anything I can contribute to the debugging?

Revision history for this message

Kovid Goyal (kovid) wrote on 2019-11-13:

#14

Dont use flatpaks, they are totally unsupported.

Revision history for this message

Axel (nospamforaxel-1) wrote on 2019-11-14:

#15

I confirm that in my case when using the installation from the calibre-website the issue is not there. Downloading of news-sources works as expected.

This suggests that the issue is related to flatpak.

I thought flatpak is okay to use, since it is offered in flathub.

calibre

Süddeutsche

Bug Description

Other bug subscribers

Remote bug watches