Support Private PPAs

Bug #645404 reported by Vincent Stehlé on 2010-09-22
70
This bug affects 12 people
Affects Status Importance Assigned to Milestone
software-properties (Ubuntu)
Medium
Dan Streetman
Bionic
Low
Dan Streetman
Cosmic
Low
Dan Streetman
Disco
Low
Dan Streetman
Eoan
Medium
Dan Streetman

Bug Description

Software properties add-apt-repository currently does not support adding private PPAs.

software-properties should connect to the API and observe that it gets permission denied trying to read the ppa. Then it can reconnect to the API asking for authentication, which will open a browser window where you can do the openid ritual. Then using that token it ought to be possible for it to get the password etc.

ProblemType: BugDistroRelease: Ubuntu 12.04
Package: python-software-properties 0.82.4

Related branches

Vincent Stehlé (vstehle) wrote :
Vincent Stehlé (vstehle) wrote :

Also, invoking with private-ppa gives an error:

root@vincent-desktop:~# add-apt-repository private-ppa:tiomap-dev/private-release
Error: 'private-ppa:tiomap-dev/private-release' invalid

Martin Pool (mbp) wrote :

To correct this (I think), software-properties would have to connect to the API and observe that it gets permission denied trying to read the ppa. Then it can reconnect to the API asking for authentication, which will open a browser window where you can do the openid ritual. Then using that token it ought to be possible for it to get the password etc.

Steve Langasek (vorlon) on 2011-02-16
tags: removed: armel
Robert Roth (evfool) wrote :

Currently the apt-add-repository does not support adding private PPAs, and it already mentions this when you try to add a private PPA, so this bug as decribed in the title does not exist anymore. Based on the latest comments, I am updating this to a Wishlist bug to support Private PPA's with the method described in comment 3.
---
Ubuntu Bug Squad volunteer triager
http://wiki.ubuntu.com/BugSquad

summary: - add-apt-repository incorrect sources with private-ppa
+ Support Private PPAs
Changed in software-properties (Ubuntu):
importance: Undecided → Wishlist
status: New → Confirmed
description: updated
tags: removed: apport-bug maverick ubuntu-une
James Westby (james-w) wrote :

Calling lp.people[lp.me.name].getArchiveSubscriptionURL(archive) once the archive has been found
will get the url to be used for private PPAs.

getArchiveSubscriptionURL

Get a text line that is suitable to be used for a sources.list entry.

It will create a new IArchiveAuthToken if one doesn't already exist.
Parameters
Parameter Value Description

ws.op

(required)

Fixed: getArchiveSubscriptionURL

archive

(required)
Link to a archive.

Kip Warner (kip) wrote :

This problem is almost 6 years old. Is anyone working on it?

Stuart Bishop (stub) wrote :

Private PPAs are accessed with basic auth using https: URLs to private-ppa.launchpad.net.

https://stub:<email address hidden>/teamname/ppaname/ubuntu

It would be great if add-apt-repository accepted this or a similar URL, use it to retrieve the signing key security and add it as a source.

Julian Andres Klode (juliank) wrote :

OK, so:

* add-apt-repository should support logging in to launchpad and adding private ppas

* private PPAs should be added with auth.conf.d for the auth data

* to avoid having to login to launchpad on a server, something like:

  - add-apt-repository should gain a --stdin switch that contains lines of repositories with
    options
  - add-apt-repository should gain a --remote/-r switch to specify a remote host, and do the auth
    locally and then run the --stdin one on the remote host.

Changed in software-properties (Ubuntu):
importance: Wishlist → Low
Changed in software-properties (Ubuntu):
status: Confirmed → Triaged
Carl Karsten (carlfk) wrote :

please consider the CI case: headless process that runs periodically for months without user interaction.

something like: deploy a private key (or similarly sensitive secret) file and a publicly readable script (apt-add-repository...) and it just runs, no browser, no user, no typing.

Dan Streetman (ddstreet) on 2019-05-10
Changed in software-properties (Ubuntu Eoan):
assignee: nobody → Dan Streetman (ddstreet)
Changed in software-properties (Ubuntu Disco):
assignee: nobody → Dan Streetman (ddstreet)
Changed in software-properties (Ubuntu Cosmic):
assignee: nobody → Dan Streetman (ddstreet)
Changed in software-properties (Ubuntu Bionic):
assignee: nobody → Dan Streetman (ddstreet)
Changed in software-properties (Ubuntu Eoan):
status: Triaged → In Progress
Changed in software-properties (Ubuntu Disco):
status: New → In Progress
Changed in software-properties (Ubuntu Cosmic):
status: New → In Progress
Changed in software-properties (Ubuntu Bionic):
status: New → In Progress
Changed in software-properties (Ubuntu Disco):
importance: Undecided → Low
Changed in software-properties (Ubuntu Cosmic):
importance: Undecided → Low
Changed in software-properties (Ubuntu Bionic):
importance: Undecided → Low
Dan Streetman (ddstreet) wrote :

targeted to b/c/d/e, although all my initial work is on e only. It includes a large amount of refactoring that I don't think is appropriate to sru back, so b/c/d would get a much smaller update - if at all.

> * add-apt-repository should support logging in to launchpad and adding private ppas

added this functionality.

> * private PPAs should be added with auth.conf.d for the auth data

added this functionality.

one note on this - when a private ppa is removed, the auth file is not currently removed, i don't have the code removing it, but probably it should be; i'll look at adding that.

> * to avoid having to login to launchpad on a server, something like:
>
> - add-apt-repository should gain a --stdin switch that contains lines of repositories with
> options

have not implemented this yet; however add-apt-repository does support providing a full sources.list line, so it could be called one-line-at-a-time. however, the way just drops the line directly into the main sources.list file, and doesn't parse out the login:password to put into an auth.conf.d/ file.

> - add-apt-repository should gain a --remote/-r switch to specify a remote host, and do the auth
> locally and then run the --stdin one on the remote host.

have not implemented this yet

Dan Streetman (ddstreet) wrote :

one note to mention on changing software-properties over to using launchpadlib for lp logins; launchpadlib will create cache files in ~ and when running as sudo ~ isn't changed (without -i or -H). This results in bug 1828208

launchpadlib is not the only thing to suffer from this feature of sudo; I'm going to start an email list discussion about ubuntu's handling of $HOME under sudo.

for now, if testing with my ppa build, add-apt-repository should be run with 'sudo -H' (or sudo -i), and it has a temporary check to prevent running under plain 'sudo'.

Dan Streetman (ddstreet) on 2019-05-17
Changed in software-properties (Ubuntu Eoan):
importance: Low → Medium
Dan Streetman (ddstreet) wrote :

@carlfk, can you elaborate on:

> deploy a private key (or similarly sensitive secret) file and a publicly
> readable script (apt-add-repository...) and it just runs, no browser,
> no user, no typing

for a private ppa, the script will only generate 3 files (e.g. for ppa:ddstreet/private on disco):
1) /etc/apt/sources.list.d/ddstreet-ubuntu-private-disco.list
2) /etc/apt/auth.conf.d/ddstreet_ubuntu_private.conf
3) /etc/apt/trusted.gpg.d/ddstreet_ubuntu_private.gpg

you could just copy those directly to whatever system you want, and not have to run anything at all on the system (besides 'sudo apt update' and then upgrade/install). Are you asking for something different?

Dan Streetman (ddstreet) wrote :

> * add-apt-repository should support logging in to launchpad and adding private ppas

added this functionality.

> * private PPAs should be added with auth.conf.d for the auth data

added this functionality.

still does not remove this auth data when the corresponding sources.list entry is removed.

> * to avoid having to login to launchpad on a server, something like:
>
> - add-apt-repository should gain a --stdin switch that contains lines of repositories with
> options

implemented standard '-' to read a line from stdin

> - add-apt-repository should gain a --remote/-r switch to specify a remote host, and do the auth
> locally and then run the --stdin one on the remote host.

implemented this as well, although I am not sure this is the best design. It will fail if the remote add-apt-repository isn't at a newer version (i.e. that supports the '-' param). Additionally, the gpg has to be manually added, since (for private ppas) that can't be queried without logging into lp, which is what the remote session is trying to avoid.

I think simply creating the resulting files - sourceparts, netrcparts, and trustedparts - on the remote system is going to be more reliable and simpler than using the remote system's installed add-apt-repository to try to process things into place. Calling apt-config on the remote system to gather the right directories to place the files should be simple; I think the hardest part will be figuring out how to do it without multiple subprocess runs to call ssh to 1) gather apt-config on the remote system and then 2) scp/sftp or ssh create-from-stdin the 3 files, and then 3) make sure they are the correct mode.

Julian Andres Klode (juliank) wrote :

So multiple ssh subprocesses should not be a huge issue, you could after all use ControlMaster and stuff to reuse the connection.

That said, it seems reasonable to create a shell script, or a Python script and execute that on the remote from stdin.

python3 and python3-apt should be available on all systems I think to do config lookups, and the content of files can be embedded into a generated script in base64.

That said, it would be an entirely different code path which I think might be worse than it maybe not working if the remote is at an older version.

Dan Streetman (ddstreet) wrote :

> So multiple ssh subprocesses should not be a huge issue, you could after all
> use ControlMaster and stuff to reuse the connection.

nice, I hadn't used ControlMaster before; that's useful.

> That said, it seems reasonable to create a shell script, or a Python script and
> execute that on the remote from stdin.

the problem with that, or even with directly calling add-apt-repository on the remote system, is it needs to be called under sudo, and there is no guarantee the sudo call is passwordless on the remote system, so the request for password (stdin/stdout) needs to be provided to the user. I don't see any clean way of doing that, but also piping commands or a script into stdin for execution on the remote side, without significant complications, or pulling in an external lib like paramiko.

> That said, it would be an entirely different code path which I think might be worse
> than it maybe not working if the remote is at an older version.

Not sure how it would be worse. Apt is quite clear on what needs to be in place, and where, to add a new repo; place the apt source server conf in Dir::Etc::sourceparts/d, place the apt archive's gpg key in Dir::Etc::trustedparts/d, and place the apt archive login details (if any) in Dir::Etc::netrcparts/d. Is any of that incorrect, and/or will any of that ever change? If not, simply creating the proper files in-place on the remote host should be all that's needed; all the logic of figuring out the naming and content of the files takes place on the local host.

Dan Streetman (ddstreet) wrote :

I changed the MP over to 'Needs Review' as I think the code is in a state ready for review.

Note that I removed the --remote support from it; while that is definitely something that should be added, I believe it's separate and complex enough it should be done separately, later.

tags: added: rls-ee-notfixing
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers