add-apt-repository will write duplicate entries to /etc/apt/sources.list

Bug #854841 reported by moma
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
software-properties (Ubuntu)
Fix Released
Low
Brian Murray

Bug Description

Hello,
Add-apt-repository command allows one to add duplicate entries to sources list. Would it be possible to improve this utility it so it adds only unique entries?

Eg. This command, when run twice, adds duplicate entries.
$ sudo add-apt-repository "deb http://archive.canonical.com/ubuntu oneiric partner

$ grep -Ri partner /etc/apt/sources.list

ProblemType: BugDistroRelease: Ubuntu 11.10
Package: python-software-properties 0.81.9
ProcVersionSignature: Ubuntu 3.0.0-11.18-generic 3.0.4
Uname: Linux 3.0.0-11-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 1.23-0ubuntu1
Architecture: amd64
Date: Tue Sep 20 18:38:17 2011
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Beta amd64 (20110919)
PackageArchitecture: all
ProcEnviron:
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bashSourcePackage: software-properties
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
moma (osmoma) wrote :
summary: - add-apt-repository writes duplicate entries to /etc/apt/sources.list
+ add-apt-repository will write duplicate entries to /etc/apt/sources.list
Changed in software-properties (Ubuntu):
status: New → Triaged
importance: Undecided → Low
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote :

I was able to confirm this using Oneiric. Thanks for providing detailed steps to recreate this bug.

tags: added: testcase
Changed in software-properties (Ubuntu):
assignee: nobody → Michael Vogt (mvo)
Revision history for this message
Nick Russo (narusso-u) wrote :

I found that add-apt-repository calls

 SoftwareProperties().add_source_from_line(line)

In /usr/share/pyshared/softwareproperties/SoftwareProperties.py I found

 self.sourceslist.list.append(new_deb_entry)
 self.sourceslist.list.append(new_debsrc_entry)

I think it would be better to call self.sourceslist.add(), since it avoids
duplicates. The add() function requires more arguments, but the only ones
I had to hard-code were elsewhere in SoftwareProperties.py anyway.

I've included a patch (against 0.80.9 through 0.82) which seems to make thing
work nicely. I hard-coded deb, deb-src, and main.

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "patch to SoftwareProperties.py to make add_source_from_line idempotent" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

tags: added: patch
Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Thanks for the patch Nick!

While your use of self.sourceslist.add() seems good, I don't like how it is using similar logic that is in ppa.py in SoftwareProperties.py's add_source_from_line(). Wouldn't it be better to extract out the parts of new_deb_entry and new_debsrc_entry and give those parts to self.sourceslist.add()?

NAK pending comments from submitter. Please resubscribe ubuntu-sponsors once you have commented and/or upddated your patch. Thanks again!

Revision history for this message
Nick Russo (narusso-u) wrote :

Thanks Jamie,
Does this new patch address your concerns? Instead of hard-coding the type, distro, components, etc. it extracts them back out of the SourceEntry object.
Nick

Revision history for this message
Valdur Kana (valdur55) wrote :

Does it check ^#deb lines?

Changed in software-properties (Ubuntu):
assignee: Michael Vogt (mvo) → Brian Murray (brian-murray)
status: Triaged → In Progress
Revision history for this message
Brian Murray (brian-murray) wrote :

Nick Russo - thanks for the patch. I've merged it with a couple of minor changes.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package software-properties - 0.82.3

---------------
software-properties (0.82.3) precise; urgency=low

  [ Brian Murray ]
  * when adding new repositories use sourceslist.add instead of append thereby
    preventing duplicate entires. Thanks to Nick Russo for the patch.
    LP: #854841

  [ Robert Roth ]
  * Updated expand properties to properly expand the bottom component to avoid
    putting empty space between components. LP: #912557
  * Added symbolic link for add-apt-repository manpage under the
    apt-add-repository name, LP: #620098
  * Changed Revert button mnemonic to avoid collision with Remove, LP: #652523
  * Handle URLError from ppa pages, instruct the user to check the internet
    connection (LP: #502698)

  [ Marc Deslauriers ]
  * SECURITY UPDATE: incorrect ssl certificate validation (LP: #915210)
    - softwareproperties/ppa.py: use pycurl to download the signing key
      fingerprint.
    - tests/test_lp.py: add test.
    - debian/control: add python-pycurl dependency.
    - CVE-2011-4407
  * Wait for PPA GPG key to get imported before ending thread (LP: #888417)
 -- Marc Deslauriers <email address hidden> Fri, 03 Feb 2012 08:16:22 -0500

Changed in software-properties (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
John Doe (johndoe32102002) wrote :

This bug is NOT fixed yet!

I am getting:

W: Duplicate sources.list entry http://ppa.launchpad.net/xxxxx oneiric/main amd64 Packages (/var/lib/apt/lists/ppa.launchpad.net_xxxxx_dists_oneiric_main_binary-amd64_Packages)
W: Duplicate sources.list entry http://ppa.launchpad.net/xxxxx/ubuntu/ oneiric/main i386 Packages (/var/lib/apt/lists/ppa.launchpad.net_xxxxx_dists_oneiric_main_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems

After running sudo apt-get update, this problem is not resolved.

lsb_release -a
LSB Version: core-2.0-amd64:core-2.0-noarch:core-3.0-amd64:core-3.0-noarch:core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:core-4.0-amd64:core-4.0-noarch
Distributor ID: Ubuntu
Description: Ubuntu 11.10
Release: 11.10
Codename: oneiric

grep -Ri launchpad* /etc/apt/sources.list
deb http://ppa.launchpad.net/steubens/transmission-proxy-support/ubuntu natty main
deb-src http://ppa.launchpad.net/steubens/transmission-proxy-support/ubuntu natty main
deb http://ppa.launchpad.net/gnome3-team/gnome3/ubuntu oneiric main
deb-src http://ppa.launchpad.net/gnome3-team/gnome3/ubuntu oneiric main
(seems like some are not listed here)

Corrective measure needed:
Implement a check to see if any PPA was already added prior to adding it to the system.

Revision history for this message
Brian Murray (brian-murray) wrote :

@John Doe - you'll notice in comment 9 the following:

This bug was fixed in the package software-properties - 0.82.3

---------------
software-properties (0.82.3) precise; urgency=low

This bug was fixed, just for the precise (12.04), release of Ubuntu - not for 11.10 which you are running.

Revision history for this message
Kyle Stevens (kstevens715) wrote :

@Brian Murray - This still seems to be a problem in trusty (14.04).

Revision history for this message
Eric (erichahn) wrote :

Bump. This still exists.

Revision history for this message
Brian Murray (brian-murray) wrote :

Could you add a test case for this? I only see a duplicated commented out deb-src entry which doesn't seem terribly important.

Revision history for this message
Brian Murray (brian-murray) wrote :

Disregarding my typo of partner this was my test:

(trusty-amd64)root@impulse:/home/bdmurray# add-apt-repository "deb http://archive.canonical.com/ubuntu trusty partnet"
(trusty-amd64)root@impulse:/home/bdmurray# cat /etc/apt/sources.list
deb http://ubuntu.osuosl.org/ubuntu trusty main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu trusty main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu trusty-updates main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu trusty-updates main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu trusty-security main restricted universe multiverse
deb http://archive.canonical.com/ubuntu trusty partnet
# deb-src http://archive.canonical.com/ubuntu trusty partnet
deb-src http://ubuntu.osuosl.org/ubuntu trusty-security main restricted universe multiverse
(trusty-amd64)root@impulse:/home/bdmurray# add-apt-repository "deb http://archive.canonical.com/ubuntu trusty partnet"
(trusty-amd64)root@impulse:/home/bdmurray# cat /etc/apt/sources.list
deb http://ubuntu.osuosl.org/ubuntu trusty main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu trusty main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu trusty-updates main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu trusty-updates main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu trusty-security main restricted universe multiverse
deb http://archive.canonical.com/ubuntu trusty partnet
# deb-src http://archive.canonical.com/ubuntu trusty partnet
# deb-src http://archive.canonical.com/ubuntu trusty partnet
deb-src http://ubuntu.osuosl.org/ubuntu trusty-security main restricted universe multiverse

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.