Beautiful soup + html5lib - AttributeError: 'NoneType' object has no attribute 'next_element'

[Guy King]

https://github.com/html5lib/html5lib-python/issues/406#issuecomment-434431366

run in conda environment -

channels:
  - defaults
  - conda-forge
dependencies:
  - beautifulsoup4=4.6.0
  - flake8=3.5.0
  - flake8-import-order=0.17
  - geopy=1.17.0
  - html5lib=1.0.1
  - ipython=7.0.1
  - requests=2.19.1
  - pymongo=3.7.1
  - pyquery=1.3.0
  - pytest=3.7.4
  - python-dateutil=2.7.3
  - pytz=2018.4
  - pip=9.0.1
  - pip:
      - flake8-commas
      - requests_mock==1.5.2
      - scrapy-crawlera==1.4.0
      - fuzzywuzzy==0.17.0
      - python-Levenshtein==0.12.0
  - python=3.6.2
  - Scrapy=1.5.0

[Isaac Muse (facelessuser)]

Digging through the example on the GitHub link, it appears the problematic source had the `<!---->` in the source. This is related to issue #1806598 which I just looked into. I believe I understand this problem and have a suitable fix for this. I plan on creating a merge request to handle this.