rubygems bin in PATH potentially breaks other applications and violates all sense of decency in packaging.

[Scott Kitterman]

The fix to Bug #145267 is a really bad idea and in uploading it seems to have ignored all the feedback on ubuntu-devel.

Gems install their own versions of stuff in a way that's opaque to the system administrator. Now whatever random libraries Gems bring onto a system get used in place of the correct libraries.

[Michael Casadevall (mcasadevall)]

As an added note, this package installs things into /usr/local/bin, which is explicatively disallowed Debian FHS. No package that we install should touch /usr/local.

This entire patch is a horrible idea and poorly executed.

Moving bug to critical due to policy violation, and the possibility that a gems package could easily hose your system if it overrides a shared library in place.

[Michael Casadevall (mcasadevall)]

Here's the relevant section of the Debian Policy Guide

http://www.debian.org/doc/debian-policy/ch-opersys.html

9.1.2 Site-specific programs

As mandated by the FHS, packages must not place any files in /usr/local, either by putting them in the file system archive to be unpacked by dpkg or by manipulating them in their maintainer scripts.

However, the package may create empty directories below /usr/local so that the system administrator knows where to place site-specific files. These are not directories in /usr/local, but are children of directories in /usr/local. These directories (/usr/local/*/dir/) should be removed on package removal if they are empty.

Note, that this applies only to directories below /usr/local, not in /usr/local. Packages must not create sub-directories in the directory /usr/local itself, except those listed in FHS, section 4.5. However, you may create directories below them as you wish. You must not remove any of the directories listed in 4.5, even if you created them.

Since /usr/local can be mounted read-only from a remote server, these directories must be created and removed by the postinst and prerm maintainer scripts and not be included in the .deb archive. These scripts must not fail if either of these operations fail.

For example, the emacsen-common package could contain something like

     if [ ! -e /usr/local/share/emacs ]
     then
       if mkdir /usr/local/share/emacs 2>/dev/null
       then
         chown root:staff /usr/local/share/emacs
         chmod 2775 /usr/local/share/emacs
       fi
     fi

in its postinst script, and

     rmdir /usr/local/share/emacs/site-lisp 2>/dev/null || true
     rmdir /usr/local/share/emacs 2>/dev/null || true

in the prerm script. (Note that this form is used to ensure that if the script is interrupted, the directory /usr/local/share/emacs will still be removed.)

If you do create a directory in /usr/local for local additions to a package, you should ensure that settings in /usr/local take precedence over the equivalents in /usr.

However, because /usr/local and its contents are for exclusive use of the local administrator, a package must not rely on the presence or absence of files or directories in /usr/local for normal operation.

The /usr/local directory itself and all the subdirectories created by the package should (by default) have permissions 2775 (group-writable and set-group-id) and be owned by root.staff.

[Michael Casadevall (mcasadevall)]

After posting that, I realize that its not clear on the specific violation, in general, the problem is this:

Steps to Reproduce:
Install rubygems
Install any gem:

mcasadevall@blacksteel:~$ sudo gem install rubygems-update
Using '/var/lib/gems/1.8/bin/update_rubygems' to provide 'update_rubygems'.
Successfully installed rubygems-update-1.2.0
1 gem installed
mcasadevall@blacksteel:~$ ls /usr/local/bin/update_rubygems
/usr/local/bin/update_rubygems
mcasadevall@blacksteel:~$

Having packages dump files (or symlinks) into /usr/local breaks the FHS; if its a package installed by the system, it shouldn't install itself or addons into /usr/local. In addition, if gems is updated, the update gets dumped into /usr/local, and is not tracked by APT, making removing the package difficult.

No package in Debian that offers its own package system supports using it. Perl offers CPAN, and PHP offers PEAR, and neither one is recommended for use. The Debian Perl group repackages CPAN modules, and I assume PHP does too. The right and PROPER solution is to depericate the use of the gems package manager in Ubuntu, and leave it without symlink, and then package gems so APT is aware of them. Write a script to automate the packaging like the CPAN guys do.

[Steve Langasek (vorlon)]

Other concerns notwithstanding, this package in fact does not violate the FHS. libgems-ruby does not itself install any files under /usr/local, nor AIUI is it intended that this be used to create other packages which ship files under /usr/local (or that modify /usr/local at install time).

The FHS states that OS packages must use the /usr and /var heirarchies for files that they install; it does not say that OSes may not provide other services, in package form, that allow the user to manage /usr/local if she chooses. Here is a short list of other pieces of software we ship that facilitate management of /usr/local, without being FHS violations:
- autoconf
- ExtUtils::MakeMaker (in perl-modules)
- distutis (in python2.x-dev)
- stow

As long as "gem install" is a command run by the user (or by third-party software the user chooses to run), and not something that will be run as a consequence of invoking the package manager, I don't see that this is a problem.

Downgrading back to 'high', since an FHS violation was given as the justification for 'critical'.

[Michael Casadevall (mcasadevall)]

I accept your opinion on this being not being a FHS, although I still believe having things end up in /usr/local is a horrible idea.

[Neil Wilson (neil-aldur)]

I hear a lot of complaints, but I hear no solutions. If a user installs a gem they expect it to work. If it doesn't then they will remove the package and install gems from source. That is what is happening.

Users don't care about Ubuntu developers prejudices. They just want their software installed now.

The rubygems package exists in the archive. If proponents of this bug are not suggesting removing rubygems from the archive (which will incidentally break ruby1.9 since rubygems is fundamentally integrated into that version of ruby) then I'm struggling to understand what is different between the new system and the one that already exists in Ubuntu Hardy - which will completely override whatever ruby libraries it clashes with if you install a gem.

Package gem does not touch /usr/local unless the system administrator executes 'gem install' with root privileges. The 1.3.0 version of gem allows a non-privilege user to install gems in their own directory and that is vital to allow Rails 2.1 to function properly - which incidentally incorporates gem into the very core of the framework configuration.

Rubygems does things your average Ubuntu developer will not like. However the solution is not to fold your arms and try and deny its existence. That just means the user installs gems from source, and believe me that is worse because gems will then scribble in /usr/bin, /usr/lib and generally make a huge nuisance of itself.

By making the package useful and sorting out the path issues, we get people to use the package rather than the source installation. Then perhaps we can get the apt packages to talk nicely to gems so that people will use the apt package rather than have gem automatically override it. Then perhaps we make a version of gem that looks in the apt package database first. Then perhaps we automate the packaging of gems as best we can.

Then we win.

Each journey begins with a single step. This is that first step - away from chaos and towards control.

[Scott Kitterman (kitterman)]

So find a sane solution.

I had a long discussion with mathiaz on IRC yesterday and proposed
alternatives. He didn't seem interested.

[Michael Casadevall (mcasadevall)]

As motu-release has decided to revert the upon, I'm marking this fix committed. That being said, we still need to find a sane alternative, so I don't believe we should close this bug until one exists.

Looking at the other interpreters in Debian and Ubuntu, we don't support the usage of CPAN or PEAR since it installs things without the package systems knowledge, nor do they get properly autoremoved if you remove the interpreter itself. Neither CPAN or PEAR has been modified to interact with APT, which would be a very complicated change, and the wrong one to make.

In this case, I believe the correct course of action is to depreciate the gems package manager (perhaps modify it to display a giant warning), but leave it available so a user can use it if they choose to override our choices, and then package gems directly into APT. I'm assuming gem installation from source packages are relatively striaghtforward (something like perl Makefile.PL; make; make install, or python setup.py), so it is likely possible to create a template set of files for debian/, allowing for rapiding packaging of every gem.

Now looking at Rubyforge, there are roughly 7,000 gems, so packaging each one individually may be a little close to impossible. Still, the most common, and most popular ones should be packaged and archived; Rails should be in the archive for instance.

No matter what we will do, people will still have the opportunity to shoot themselves in the foot by installing the source packages or using gems over APT. However, by providing a sane alternative, we can avoid that possibility.

[Neil Wilson (neil-aldur)]

Michael,

I appreciate your viewpoint, but unfortunately it comes from one of inexperience. The only one here with experience of managing rubygems on an Ubuntu server is me. I have knocking on for 200 of them at Brightbox and 125 paying customers who disagree with your notion that they don't need Rubygems on Ubuntu. We have installed gems from source because the Ubuntu package is non-functional. So if there is anybody who understands Gems on Ubuntu it is me, and you haven't even bothered to speak to me about it.

gem installation from source is completely arbitrary, which you'd know if you worked with them. It compiles things on the fly. Might run rakefiles. Might run setup. Might pull in libraries. There is no standard layout.

Gems exists, and is real. You can't use Rails without it. And quite frankly what gems does with my server is outside your policy remit as an Ubuntu developer.

And that is the nub of the issue that needs addressing. Where does Ubuntu policy end and the System Administrators responsibility take over.

Gem installs programs from a foreign repository.

So does apt if you tell it to, and alien will create you an apt package from any RPM you fancy.

Similarly the 'install' program will put files whereever I tell it. The 'ln' program will quite happily link /sbin/halt to /bin/ls if I tell it, and I can always chmod 4755 something if I want to have some real fun.

And that's before we get onto the compilers which can even create kernel modules.

Ubuntu policy ends at the point the program is installed. The package should ensure that the program runs correctly according to what the original developer intended and in a way that doesn't stand on the files of anything else.

If I choose to install gems from the archive, then the ubuntu developers responsibility for system stabililty is merely to make sure that the program runs normally without standing on the files of anything else (as you would with a C compiler). What that program creates when you run it (as with a C compiler) is outside the Ubuntu developers remit. If the function of the program is to put things in the system path (as with the 'install' program) then it should install things in the system path.

If this isn't clear in policy then it should be so. The programs and files created by installed programs are not the concern of the Ubuntu developers.

[Michael Casadevall (mcasadevall)]

Comparing the behavior of using install to replace /sbin/halt to /bin/ls is apples to oranges. This is about the sane behavior of packages and the way they are configured.

http://pkg-ruby-extras.alioth.debian.org/rubygems.html - Debian's stance, and it is valid.

Ubuntu's responsibility is to give packages sane defaults that are not going to flood a system with garbage. Both CPAN and PEAR respect the FHS, and install into /usr/local even though their binaries go into /usr. Ruby on the other hand rejects the FHS, and if installs to /usr, will install its modules to /usr, possibly overriding system files and having complete disregard for the package maintainer.

A (competent) system administrator installing gems from source would install to /usr/local which avoids the problem since there is no chance of Ruby overriding critical system files.

The only sane alternative then to make gems available and not violate the FHS w.r.t. to /usr/local is to install out of the /usr or /usr/local hierarchies which is currently how gems work. While possibly adding gems to the path is a valid option (I'm not going to weigh one way or another), RubyGems's developers have stated they do not wish to fix the (in my opinion, horrid) design issues with the way gems are installed, thus making it dangerous to offer gems configured for any path other then its own where it can't possibly hose someones system because they install a bad gem.

[Joseph Method (tristil)]

I can't freaking believe we're going to go another cycle with a broken Rubygems. The sad thing is that Ruby has a very creative and productive developer community that is staying on or moving to MacOS because it's so easy to get into Linux-y Ruby development there. Rubygems comes with Leopard! Apple is actively targeting Ruby and Python open source developers.

One misunderstanding about Rubygems in these reports is that Rubygems is *not* used for system scripts (I mean, as a rule). It's used to keep track of extremely quick-moving distributed development that's pulled together for web applications (again, as a rule). Nobody would ever dream of using a gem as a requirement in a Deb package. They would do the right thing and package up the library for Apt. That's not the issue here. Rubygems is a tool for keeping pace with the world of Ruby development, which proceeds much faster than a distro packaging process.

One horrible irony in all this is that at least until recently the most recent source package for Rubygems is actually broken on Ubuntu systems. You have to manually edit a file to include a require statement presumably because of a difference in the Ruby Debian package from the standard upstream distribution. This is what a Deb package is supposed to do, work out all the little details and even take up slack for the upstream developers. Instead we get this. I'm pretty sure it loses users.

[Charles Lowell (cowboyd-thefrontside)]

Let me weigh in as a developer who makes his living writing ruby programs, and is also, incidentally, a big fan of APT (though I don't have any visibility into the subtleties of the various package management system.)

I'm not a huge fan of rubygems. I'd even go so far as to say that I don't really like them. They suck.

Given my preference for APT over rubygems, I have tried intermittently tried to use debian packages to manage dependencies on external ruby libraries, but this has *never* worked in the long run.

In every single non-trivial project in the last 5 years I have grudgingly had to revert back to rubygems again, and again. The reasons are the same as Joseph already mentioned: 1) rubygems move more quickly than the OS packages, and are often I cannot afford to be 1 or more versions behind 2) Often, there isn't a debian package available *at all*, and so rubygems is the only option available other than installing from source.

And that's the rub. Unless you can guarantee 100% equivalence, then people will always fall back on rubygems. 95% doesn't cut it, because then for all the edge-cases (and there are always edge-cases) you have to use a second package management system, and so you might as well be using it the whole time.

I might add that every single one of the 10,000 rails tutorials on the internet begins with:

sudo gem install rails
rails /path/to/rails/app

That worked on 8.4, but it is broken on 8.10beta(along with every other rubygem) and that's a *very* serious regression.

Let me be perfectly clear: I don't like rubygems. If I can avoid it, I will. If there were a reasonable solution using APT I would prefer it, but I do not believe that will ever happen. As such, rubygems should work as expected on Ubuntu.

[Scott Kitterman (kitterman)]

There was no intent to cause a regression when we reverted the upload that
this bug is about. AFAIK, we put it back like it was.

If there's a regression from Hardy we should fix it. I'm open to
suggestions.

[Eric Mill (konklone)]

Rubygems is certainly broken again on Jaunty. Behold:

$ gem list | grep rails
rails (2.3.2)

$ rails
The program 'rails' is currently not installed. You can install it by typing:
sudo apt-get install rails
bash: rails: command not found

This is incredibly frustrating, as a dedicated Ruby/Rails developer and Ubuntu user. There are many other binaries that come along with certain gems that I want to use, and I don't want to have to symlink this stuff myself.

Can we please find the right solution here that doesn't cause potentially dealbreaking headaches for Rubyists trying out Ubuntu?

[Eric Mill (konklone)]

I can't read or write.

[Scott Kitterman (kitterman)]

This particular bug is fixed and should stay that way. Reopening old bugs is not the appropriate way to report new problems.