Disable --update option

[Pierre Rudloff]

[Impact]
The --update option downloads content from a third party site (a URL hardcoded in the script) and then copies it into /usr/bin/youtube-dl. This is unsafe.

Unfortunately, the --update option is extraordinarily handy since youtube frequently changes its web interface, which breaks the script. Indeed the version of youtube-dl currently in Precise is broken as of Nov 4, 2012 and can't retrieve videos. Running youtube-dl --update fixes this by installing a new version of the script. However this new script is a binary executable.

[Test Case]
$ ls -l /usr/bin/youtube-dl; md5sum /usr/bin/youtube-dl; file /usr/bin/youtube-dl
-rwxr-xr-x 1 root root 159848 Feb 27 2012 /usr/bin/youtube-dl*
bd2f1db2f3edafcbf207fab805d36e23 /usr/bin/youtube-dl
/usr/bin/youtube-dl: a python script, UTF-8 Unicode text executable

$ sudo youtube-dl --update
[sudo] password for XXX:
Updating to latest version...
Updated youtube-dl. Restart youtube-dl to use the new version.

$ ls -l /usr/bin/youtube-dl; md5sum /usr/bin/youtube-dl; file /usr/bin/youtube-dl
-rwxr-xr-x 1 root root 43730 Nov 4 20:50 /usr/bin/youtube-dl*
02c2a961099f067a8595ac771baed12a /usr/bin/youtube-dl
/usr/bin/youtube-dl: data

[Regression Risk]
The patch removes code (and functionality), so doesn't risk new bugs. The loss of functionality is intentional to fix this issue.

[Original Report]
Shouldn't automatic updates be disabled, as with other packages (e.g. Firefox) ?

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: youtube-dl 2012.02.27-1 [modified: usr/bin/youtube-dl]
ProcVersionSignature: Ubuntu 3.2.0-31.50-generic-pae 3.2.28
Uname: Linux 3.2.0-31-generic-pae i686
ApportVersion: 2.0.1-0ubuntu13
Architecture: i386
Date: Mon Oct 8 02:04:53 2012
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=fr_FR:en
 TERM=xterm
 PATH=(custom, no user)
 LANG=fr_FR.UTF-8
 SHELL=/bin/bash
SourcePackage: youtube-dl
UpgradeStatus: Upgraded to precise on 2012-04-27 (164 days ago)

[Bryce Harrington (bryce)]

Agreed. I think this option might pose a security risk.

[Bryce Harrington (bryce)]

This patch simply deletes the option. I left the function itself in place to hopefully keep the patch applicable for future releases.

[Bryce Harrington (bryce)]

It looks like quantal and raring will need a different fix; the youtube-dl included in those versions is a binary file and looks like it's created by zipping several .py files (presumably to make the download easier). The package should probably be changed to use a more standard python build/install system.

[Pierre Rudloff (rudloff)]

The binary is build from the source by the make command but you can still apply a patch.

This patch has been successfully tested on Quantal.

[TomasHnyk (sup)]

I do not understand this. First, it should be fix released for non-specific version of ubuntu as I cannot update in Saucy.

Second, there is nothing automatic about this. The user must explicitly download a new version (and even provide the password). If he does not trust it, he should not do that.

Would not an explicit warning be better then just removing the option?

i.e.:
1) sudo youtube-dl -U
2) there is no way to know that youtube-dl is not evil, do you know what you are doing? yes/no

This just means that youtube-dl gets useless early (or even before) the distribution is released as I do not think it qualifies for a SRU. Then the users must download the software from upstream defeating the purpose of a distribution.

[TomasHnyk (sup)]

For 2), it could also include some of the following informtion:: To check the signature, type:
sudo wget https://yt-dl.org/downloads/2013.11.07/youtube-dl.sig -O youtube-dl.sig
gpg --verify youtube-dl.sig /usr/local/bin/youtube-dl
rm youtube-dl.sig

The following GPG keys will be used to sign the binaries and the git tags:
4096R/A4826A18 Philipp Hagemeister Key fingerprint = 7D33 D762 FD6C 3513 0481 347F DB4B 54CB A482 6A18
4096R/BCF05F6B Filippo Valsorda Key fingerprint = 428D F5D6 3EF0 7494 BB45 5AC0 EBF0 1804 BCF0 5F6B

Older releases are also signed with one of:
1024D/FAFB085C Philipp Hagemeister Key fingerprint = 0600 E1DB 6FB5 3A5D 95D8 FC0D F5EA B582 FAFB 085C (until 2013-06-01)

Taken from here:

http://rg3.github.io/youtube-dl/download.html

[Rogério Theodoro de Brito (rbrito)]

Hi.

On Thu, Nov 7, 2013 at 7:34 PM, TomasHnyk <email address hidden> wrote:
> I do not understand this. First, it should be fix released for non-
> specific version of ubuntu as I cannot update in Saucy.

You can usually grab the newest versions of youtube-dl from Debian
unstable, as that is where I'm doing my job. I think that I can set up
something (say a reminder) to ask Ubuntu to sync from Debian every
time that I upload a new version.

I plan on uploading one this weekend by the way.

> Second, there is nothing automatic about this. The user must explicitly
> download a new version (and even provide the password). If he does not
> trust it, he should not do that.
>
> Would not an explicit warning be better then just removing the option?
>
> i.e.:
> 1) sudo youtube-dl -U
> 2) there is no way to know that youtube-dl is not evil, do you know what you are doing? yes/no

In my Debian packages, I don't consider the option of removing the
update, as it is, exactly as you say, a strictly voluntary option of
the user.

I consider this the equivalent of removing the option of the user to
download python packages via pip, perl packages via cpan, node.js
packages via npm, eclipse extensions etc.

So, as long as I maintain youtube-dl in Debian, the Debian package
won't be patched and will be as similar to upstream as possible (and,
in fact, I have avoided patching youtube-dl in Debian and submitted my
changes upstream directly *before* I incorporated them in Debian).

> This just means that youtube-dl gets useless early (or even before) the
> distribution is released as I do not think it qualifies for a SRU. Then
> the users must download the software from upstream defeating the purpose
> of a distribution.

Indeed. This kind of software is a moving target. See my long comments
on Debian's NEWS file regarding the changes in Youtube's provision of
videos with audio and video in separate. The next upload that I do
will have fixes for vimeo, as they have changed things.

In other words, web scrapers consist of a class of packages that don't
really qualify for an long term release. The same thing happens with
browsers, but a big browser is simply a very large program, while the
program that I package is a small one that people can even opt to not
ship in a release.

What do all the big browsers and the web scrapers have in common? They
manipulate changing code provided by 3rd parties.

Regards,

--
Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[TomasHnyk (sup)]

Hi Rogério, thanks for this, I have not read anything as common-sense on a bug tracker in a long time.

[Adolfo Jayme Barrientos (fitojb)]

Agree with Rogério as well...

[Rolf Leggewie (r0lf)]

it looks like upstream fixed this. From a current trusty installation.

$ sudo youtube-dl -U
It looks like you installed youtube-dl with a package manager, pip, setup.py or a tarball. Please use that to update.

[Steve Langasek (vorlon)]

The Precise Pangolin has reached end of life, so this bug will not be fixed for that release