apt-clone crashed with SystemError in open(): E:Malformed line 5 in source list /target/etc/apt/sources.list.d/ubuntu-ja.list (dist parse)

[Yuta Ogasawara]

Binary package hint: apt-clone

none

ProblemType: Crash
DistroRelease: Ubuntu 11.04
Package: apt-clone 0.1.6
ProcVersionSignature: Ubuntu 2.6.38-8.42-generic 2.6.38.2
Uname: Linux 2.6.38-8-generic i686
Architecture: i386
Date: Mon May 2 02:44:30 2011
ExecutablePath: /usr/bin/apt-clone
InterpreterPath: /usr/bin/python2.7
LiveMediaBuild: Ubuntu 11.04 "Natty Narwhal" - Release i386 (20110427.1)
PackageArchitecture: all
ProcCmdline: /usr/bin/python /usr/bin/apt-clone restore-new-distro /target/ubiquity-apt-clone/apt-clone-state-hostname.tar.gz natty --destination /target
ProcEnviron:
 LANGUAGE=ja_JP.UTF-8
 PATH=(custom, no user)
 LANG=ja_JP.UTF-8
 SHELL=/bin/bash
 LC_COLLATE=C
PythonArgs: ['/usr/bin/apt-clone', 'restore-new-distro', '/target/ubiquity-apt-clone/apt-clone-state-hostname.tar.gz', 'natty', '--destination', '/target']
SourcePackage: apt-clone
Title: apt-clone crashed with SystemError in open(): E:Malformed line 5 in source list /target/etc/apt/sources.list.d/ubuntu-ja.list (dist parse)
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

[Nobuto Murata (nobuto)]

Note:
/etc/apt/sources.list.d/ubuntu-ja.list
means third party repository, Ubuntu Japanese LoCo's.
https://www.ubuntulinux.jp/sources.list.d/natty.list

It currently contains the lines below.
-----
## Ubuntu Japanese LoCo Team's 3rd party repository for Ubuntu 11.04
## Please report any bug on https://bugs.launchpad.net/ubuntu-jp-improvement
deb http://archive.ubuntulinux.jp/ubuntu natty main
deb-src http://archive.ubuntulinux.jp/ubuntu natty main
deb http://archive.ubuntulinux.jp/ubuntu-ja natty-non-free/
deb-src http://archive.ubuntulinux.jp/ubuntu-ja natty-non-free/
-----

[Fumihito YOSHIDA (hito)]

This bug has two mechanism,

1) apt_clone.py::rewrite_sources_list() supports only pockets of dist-{updates, backports, security}.
----------------------------------------------
Quote from apt_clone.py::rewrite_sources_list()::

       for entry in sources.list[:]:
           if entry.invalid or entry.disabled:
               continue
           replacement = ''
           for pocket in ('updates', 'security', 'backports'):
               if entry.dist.endswith('-%s' % pocket):
                   replacement = '%s-%s' % (new_distro, pocket)
                   break
           if replacement:
               entry.dist = replacement
           else:
               entry.dist = new_distro
----------------------------------------------
That replaces:
   a) "dist-{updates, backports, security}" => new_distro-{updates, backports, security}
   b) Any "non-a)" pattern => new_distro

So, apt-clone breaks the third-party apt-line (that has different naming , e.g.: pocket = "natty-non-free").

2) apt-clone runs apt-get, but apt-line has broken by 1), that cause crash with "E:Malformed line N" messages.

[Fumihito YOSHIDA (hito)]

So, this bug affects some derivatives of Ubuntu, if derivatives includes apt-line of DIST-SUBNAME(such as "natty-non-free") , the apt-clone breaks their sources.list files.

But this bug caused in rare conditions.

Hearing investigation(in Japanese forum; https://forums.ubuntulinux.jp/viewtopic.php?id=11165) with Yuta Ogasawara, our analysis are:

 * We can reproduce only these situations:
   - Install to pre-formatted partition (that has already ext4-formated with gparted).
      - "manually partitioning" selected at Ubiquity.
   - In these situation, Ubiquity invokes apt-clone.
   - apt-clone is crashed. the crash is reported problem.

 * We could not reproduce in pure clean-install process.

[Apport retracing service (apport)]

Thank you for taking the time to report this crash and helping to make Ubuntu better. This particular crash has already been reported and is a duplicate of bug #779682, so is being marked as such. Please look at the other bug report to see if there is any missing information that you can provide, or to see if there is a workaround for the bug. Additionally, any further discussion regarding the bug should occur in the other report. Please continue to report any other bugs you may find.