Markdown article importer messes up article tree
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu Developer Portal |
Fix Released
|
Critical
|
Daniel Holbach |
Bug Description
In a postgres environment articles end up in the wrong place in the article tree.
I created a debug branch which should make things a bit easier to debug and retry quickly.
Instructions to reproduce
Prerequisites
$ bzr branch lp:~developer-ubuntu-com-dev/developer-ubuntu-com/debug-md-importer
$ cd debug-md-importer
$ virtualenv ./env
$ ./env/bin/pip install -r requirements.txt
$ ./env/bin/pip install django psycopg2
This branch mostly just adds some test data, debug messages and some convenience scripts.
To change from sqlite (default) to postgres apply a diff like http://
$ sudo apt install python-pip python-dev libpq-dev postgresql postgresql-contrib
$ sudo -u postgres psql -c "CREATE USER developerportal
The issue itself
Run
$ ./postgres-retry.sh
in a postgres setting and
$ ./sqlite-retry.sh
in the sqlite setting.
The script will create a new db from scratch, run the importer with debug messages and run runserver for you, so you can easily check in the pages list how the structure changed.
Current state of things:
sqlite does fine and postgres has the article tree messed up.
Related branches
- David Callé (community): Approve
- Michael Hall: Pending requested
- Ubuntu App Developer site developers: Pending requested
-
Diff: 41 lines (+6/-2)2 files modifieddeveloper_portal/management/commands/import-external-docs-branches.py (+5/-1)
requirements.txt (+1/-1)
- Ubuntu App Developer site developers: Pending requested
-
Diff: 1625 lines (+953/-399)28 files modifiedTODO (+13/-0)
developer_portal/admin.py (+1/-18)
developer_portal/migrations/0001_initial.py (+0/-9)
developer_portal/models.py (+0/-21)
developer_portal/settings.py (+2/-0)
locale/de.po (+2/-2)
locale/en_GB.po (+2/-2)
locale/es.po (+2/-2)
locale/ug.po (+2/-2)
locale/zh_CN.po (+2/-2)
md_importer/admin.py (+36/-0)
md_importer/management/commands/import-external-docs-branches.py (+35/-335)
md_importer/management/importer/article.py (+118/-0)
md_importer/management/importer/publish.py (+64/-0)
md_importer/management/importer/repo.py (+168/-0)
md_importer/management/importer/source.py (+60/-0)
md_importer/migrations/0001_initial.py (+46/-0)
md_importer/models.py (+54/-0)
md_importer/tests/__init__.py (+8/-0)
md_importer/tests/data/link-test/file1.md (+5/-0)
md_importer/tests/data/link-test/file2.md (+3/-0)
md_importer/tests/test_branch_fetch.py (+42/-0)
md_importer/tests/test_branch_import.py (+67/-0)
md_importer/tests/test_link_rewrite.py (+44/-0)
md_importer/tests/test_snappy_import.py (+70/-0)
md_importer/tests/test_utils.py (+33/-0)
md_importer/tests/utils.py (+67/-0)
requirements.txt (+7/-6)
- Ubuntu App Developer site developers: Pending requested
-
Diff: 886 lines (+636/-75) (has conflicts)12 files modifiedTODO (+7/-0)
developer_portal/admin.py (+22/-4)
developer_portal/management/commands/import-external-docs-branches.py (+42/-56)
developer_portal/management/importer/article.py (+126/-0)
developer_portal/management/importer/local_branch.py (+132/-0)
developer_portal/management/importer/publish.py (+60/-0)
developer_portal/management/importer/source.py (+51/-0)
developer_portal/migrations/0002_rework_importer.py (+59/-0)
developer_portal/migrations/0003_track_imported_articles.py (+24/-0)
developer_portal/models.py (+46/-15)
developer_portal/settings.py (+20/-0)
requirements.txt (+47/-0)
tags: | added: site-dev |
Changed in developer-ubuntu-com: | |
importance: | Undecided → Critical |
description: | updated |
description: | updated |
description: | updated |
Changed in developer-ubuntu-com: | |
assignee: | nobody → Daniel Holbach (dholbach) |
status: | New → In Progress |
Changed in developer-ubuntu-com: | |
status: | In Progress → Fix Committed |
Changed in developer-ubuntu-com: | |
status: | In Progress → Fix Committed |
Changed in developer-ubuntu-com: | |
status: | Fix Committed → Fix Released |
It works in a local postgres environment, but not on staging...