cache corruption on api.launchpad.net,1.0,-application,vnd.sun.wadl+xml,fc06437932a618a4b30d0d0417f9234c

[Vincent Ladeuil]

While running tests on quantal for the udd package importer I encountered a series of tracebacks:

Traceback (most recent call last):
  File "/pkgimport/scripts/bin/import-package", line 5, in <module>
    sys.exit(main(sys.argv))
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 1172, in main
    only_before=options.only_before)
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 1072, in _import_package
    possible_transports=possible_transports)
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 952, in handle_collisions
    if clean_collision(importp, suite, db, temp_dir, download_dir, name, updates_branch):
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 886, in clean_collision
    return check_same(importp, db, revid, name, updates_branch, temp_dir, download_dir)
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 853, in check_same
    importp, db, revid, name, updates_branch, temp_dir, download_dir)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 132, in run
    self.cleanups, self.func, self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 166, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 873, in _check_same
    pull_debian=False)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/import_dsc.py", line 1243, in import_package
    pull_debian=pull_debian)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/import_dsc.py", line 1154, in _import_normal_package
    timestamp=timestamp, file_ids_from=file_ids_from)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/import_dsc.py", line 1003, in import_debian
    get_commit_info_from_changelog(changelog, self.branch)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/util.py", line 532, in get_commit_info_from_changelog
    bugs = find_bugs_fixed(changes, branch, _lplib=_lplib)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/util.py", line 458, in find_bugs_fixed
    deb_bugs = _lplib.debian_bugs_for_ubuntu_bug(match.group(0))
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/launchpad.py", line 84, in debian_bugs_for_ubuntu_bug
    lp = get_launchpad()
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/launchpad.py", line 43, in get_launchpad
    service_root=LPNET_SERVICE_ROOT)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/launchpad.py", line 373, in login_anonymously
    version=version)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/launchpad.py", line 198, in __init__
    credentials, service_root, cache, timeout, proxy_info, version)
  File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/resource.py", line 460, in __init__
    self._wadl = self._browser.get_wadl_application(self._root_uri)
  File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/_browser.py", line 300, in get_wadl_application
    return Application(str(url), content)
  File "/usr/lib/python2.7/dist-packages/wadllib/application.py", line 1071, in __init__
    self.doc = self._from_string(markup)
  File "/usr/lib/python2.7/dist-packages/wadllib/application.py", line 1108, in _from_string
    return self._from_stream(BytesIO(markup))
  File "/usr/lib/python2.7/dist-packages/wadllib/application.py", line 1095, in _from_stream
    for event, elem in ET.iterparse(stream, events):
  File "<string>", line 107, in next
cElementTree.ParseError: unclosed token: line 13343, column 6

Deleting .launchpadlib/api.launchpad.net/cache/api.launchpad.net,1.0,-application,vnd.sun.wadl+xml,fc06437932a618a4b30d0d0417f9234c addressed my issue.

I had a brief look at the file before deleting it and it appeared truncated.

This occurred at least three times so I expect it will happen again, if the
file itself is needed for analysis, I'll keep a copy next time.

I don't remember seeing that kind of failure on jubany (running lucid) nor
on precise.

[Vincent Ladeuil (vila)]

My understanding here is that two processes are doing the same query and stepping on each other toes. Updating the file itself should be atomic to avoid this (create a tmp file, rename to the existing one) or something.

[Chris Johnston (cjohnston)]

I'm seeing the same issue. It starts after receiving a timeout from LP, and since LP timeouts are increasing in frequency, I suspect this will start happening more often.