Sources published in a private ppa are downloadable by any subscriber

[Dustin Kirkland ]

We have a commercial Launchpad license, with which we are using private PPAs to build binary packages.

However, part of our code is proprietary (sorry, but that's how it goes).

We love using Launchpad for building our binary packages, but unfortunately we're not able to do so for our proprietary bits because Launchpad publishes the sources along with the binaries.

What we really, really need is an option in each Private PPA configuration that would allow the PPA owner to "disable" publishing the resulting source packages.

Thanks!
:-Dustin

[Julian Edwards (julian-edwards)]

This is how I would implement this:

* Add a publish_sources BOOL on the Archive table (this needs a release to production before more code can be written)
* Add that to model/interface code
* Add the new field to the +edit form in the browser code but make sure it can only be changed by private PPAs using field validators.

These are small changes and at this point you will be able to edit and save the flag. Optionall you can add an API method for it but I'd not bother for now.

* Split Distribution.getPendingPublicationPPAs() into two sub-methods that return the sources and binaries separately.
* Fix test breakage that calls this method.
* The method is called from lib/lp/soyuz/scripts/publishdistro.py, it can be changed to pull in pending binaries and then optionally UNION with the pending sources depending on the archive's publish_sources flag.
* Add tests for this scenario.

This way you'll have PENDING sources forever if the flag is not set, but it reflects reality and I prefer that. You can also get the sources published if the flag is re-set.

My only question is around domination. Perhaps William can comment on this approach.

[William Grant (wgrant)]

Note that private PPA builds rely on the sources being published in the archive, since they're not available from the unrestricted librarian. We could reimplement it based on TLTs to the public restricted librarian.

There's also the issue that subscribers have access to the Archive's pages now -- a change which I strongly resisted at the time for reasons very similar to this.

[Julian Edwards (julian-edwards)]

Ah crap, I forgot about the published source thing. And I was the one who implemented that.

(What's a TLT?)

Ok, we need a different approach. I think it needs a way of adding a new .htaccess file for sources. I don't know enough about Apache offhand to suggest anything useful at the moment.

[Robert Collins (lifeless)]

TLT is a time limited token. Easy to use - just an API call to get a token and pass that as the url across to the builder.

[Robert Collins (lifeless)]

This would certainly be a good feature, its not currently in the LP roadmap, nor is it more important than the other soyuz things we have queued up for maintenance AFAICT.

[Julian Edwards (julian-edwards)]

Rob, it's not us that wants to code this, so we should help Dustin.

[Dustin Kirkland  (kirkland)]

You know, thinking more about this, it could even be trivially solved
by a couple of .htaccess rules on published archive itself. If a
private PPA user has configured that their PPA should *not* serve
source packages, then install an .htaccess rule that denies serving
files that match the sources pattern.

Ultimately, that's my end goal -- to prevent download of
non-open-source code that was built by Launchpad in a Private PPA.

What do you think?

[Dustin Kirkland  (kirkland)]

I'm sure you fellas already know all of this, but I'll document it here just case...

 - You'd need Apache's rewrite module:
 sudo a2enmod rewrite

 - You'd need to allow overrides in the apache configuration for the private PPA's virtual site.

 - When the PPA owner decides they want to deny access to sources, put this into this repository's ppa/ubuntu/pool/.htaccess file:
RewriteEngine On
RewriteRule .*\.gz$ - [F,L]

 - This is going to happen so infrequently that I'd be fine with contacting a LOSA when and if this needs to happen for my PPAs.

That will throw up a 403 Permission Denied error on trying to access any files ending in .gz (ie, sources and diff tarballs). Only do this in the pool, as package lists in dists/ can be gzipped.

I'm guessing this is going to be too simple of a solution to be accepted. I'm sure we'll need to find something far more complex.

Could we try this out at least? Maybe with a LOSA tossing this .htacess into and enabling mod_rewrite on a single PPA?

[Robert Collins (lifeless)]

We covered this on IRC but...

other buildds could then trivially access the source.

If it is proprietary as stated, this is probably an unacceptable hole.

So, all the things julian said +
 - change PPA's to get their sources from the public restricted
librarian using a 'TimeLimitedToken' (grep in the codebase for this)

should do it.

Fiddling with apache simply cannot deal with the buildds-are-untrusted hole.

[William Grant (wgrant)]

App permissions would also need to be tweaked, as subscribers gained launchpad.View 18monthish ago. Archive dependencies also need to be sorted out, as does log scrubbing.

[Dustin Kirkland  (kirkland)]

This all sounds complicated and a long shot for any sort of near term
acceptance into Launchpad.

Instead, we'll continue building in a private ppa and then we'll host
our own archive mirror (using ubumirror or something) of the debs only
on our own site.

If that works for us, I'll publish a rune for others who might find it useful.

Do you know of any gotchas with that approach?