Add support for two argument (upper and lower bound constraints) on :VERSION dependencies

[Robert P. Goldman]

Faré writes:

A future maintainer of ASDF may extend the
`(:version ,dependency ,minimum-version)
pattern for specifying dependencies with a
`(:version ,dependency ,minimum-version ,too-large-version)
pattern that allows to specify an exclusive upper bound on compatible versions.
This would allow finer specification of compatible version ranges than just
a match on the major version. But I didn't implement it.
I leave it as an exercise for whoever volunteers to replace me,
since I'm definitely leaving active maintenance of ASDF
and unwilling to implement new features,
though I will still fix bugs until a new maintainer rises.

[Faré (fahree)]

Following the discussion in asdf-devel today (2013-11-21), I move that this bug be resolved as "Invalid".

Clients should be able to specify an open-ended range with a minimal version, and only that.

Providers should be able to specify also a range with a minimum API compatibility version, and only that — with the current version always being the maximum supported version.

If the client system specifies a minimum version, it means it, and any older version is an error that better occur early than late.
If he specified a maximum version — he shouldn't, and ASDF *must not* provide a means to do that.

Backward compatibility if specified must be specified by the provider system, not the client system, (and then defsystem should have a :backward-compatible-to keyword argument). And if the provider system declares that it doesn't support
compatibility with the old API — it means it, too.

If somehow things are actually compatible, then either system has to be modified indeed, and/or you should be pulling newer more compatible versions, anyway. If what you really want is a fork of an old system, then use a fork of an old system, with its own name, e.g. "hunchentoot-0.13". If you're using extreme ways of being compatible with a wider range of versions of a system than expressible through ASDF version constraints, then don't use ASDF version constraints and stick to your extreme ways.

I also vote against "semantic versioning" as a meaningful thing for Common Lisp code. It's a great notion for the binary release of C dynamic libraries that just doesn't mean anything for Lisp source (or even fasl) releases.

[Robert P. Goldman (rpgoldman)]

I absolutely don't agree with the above claim. I have, for example, modified the XMLS API to be incompatible at version 2.0. I think that clients that are aware of this, and that don't have time or inclination to update, should be able to say this by providing an upper bound on the version compatibility.

Lots of real systems in the real world end up relying on old versions of libraries, and I don't think it's our job to tell people they can't do this.

Your argument seems to violate your own postulate about having the person who knows specify. In this case, the provider can't know all the consumers. A consumer's programmer can know that the consumer is incompatible with the new API. Why make the poor *users* of the consumer guess what's gone wrong when they try to use the new, shiny, incompatible library? Allow the consumer's programmer to tell his users that they must use the old version.

[Faré (fahree)]

Clients can't know in advance that you'll make an incompatible change at one version rather than another. They can't honestly write (:version "foo" "2.0" "3.0").

You on the other hand, know that you broke this or that client and can write :breaks ((:version "this-client" "2.3")).

If the client wishes to continue to use your library foo, they MUST NOT say (:version "foo" "2.0" "3.0"), but MUST update your library and then use (:version "foo" "3.0").

If the client insists on not supporting the new versions of the library, they should fork the library, and depend on the fork, e.g. (:version "foo2" "2.0").

A database of constraints outside the .asd files could also make sense, e.g.
(:breaks (:version "foo" "3.0") (:version "this-client "2.3"))

In any case, you can't predict the future. Any code indicating breakage is the responsibility of the future guy who breaks stuff, i.e. the guy who introduces incompatibilities, not of the present guy who makes things work.

[Robert P. Goldman (rpgoldman)]

I think you are getting hung up on this "see the future" thing inappropriately. And I think that :breaks is incorrect, because it assumes some strange flow of information back from the client to the library provider.

My desire to have an upper bound doesn't require any clairvoyance. Here's the simple use case (that has happened to me): I fix my library, XMLS to support a new behavior that didn't exist before, and that required a (partly) backwards-incompatible modification to the API.

My colleague, who has system X, sees the modification. He doesn't have time to modify system X to use the new version of XMLS (and if I tell him "he MUST NOT say (:version "xmls" "1.0" "2.0"), but MUST update [his] library and then use (:version "xmls" "2.0")," he will quite rightly tell me that it's not my job to tell him what he must and must not do, and that he doesn't have the resources to change system X to use XMLS 2 today. Instead he changes the :depends-on to (:version "xmls" "1.0" "2.0"), which keeps system X users from being confused if they update XMLS to 2, and something weird happens.

No seeing the future, and no need for me (maintainer of XMLS) to know that system X even exists.

I think my intuition that "the library provider doesn't have to know who uses his system" is a valid form of abstraction, and should be supported.

Maybe in the best of all worlds, everyone is constantly maintaining their software to keep up to date with the latest library releases, but in the world of finite resources, there are times when it is reasonable for this not to happen.

At any rate, version upper bounding hurts no one, so I propose to add it.

[Faré (fahree)]

Debian packages have a Breaks: entry, and it was recently used in the SBCL package to warn users that SBCL broke ASDF 3.1.4 and earlier, so this created a conflict that users were warned of if they tried to upgrade. So there is a precedent in a :breaks entry. Software authors do not have to know all the clients they break, but when they know or learn of the incompatibilities, it's nice of them to declare them. And if the client is not updated in a timely fashion or at all (as oftens fails to happen), it's good to warn users that their software is going to be broken if they continue.

Granted, debian also have the ability to declare maximum versions that a package is compatible with. So yes, it's sometimes OK as a temporary non-fix to at least tell your users that they can't update some dependency yet. But either
1- you intend to update your code to use the latest version of the dependency some day, just not now, and it's just a temporary crock, or
2- you don't intend to update your code to use this new versions of this library anymore, and you're effectively forking the library from the old version you like, and never ever again will you be compatible; so the nice thing to do is to rename your library so it has a different name, because the authors and "owners" of the name have made choices you disapprove and won't support.

[Faré (fahree)]

So, sure, adding an upper limit in :version is a nice feature for declaring a temporary incompatibility. But it is a very bad thing that should be discouraged in the long run. Update your software or fork the dependency; don't gratuitously prevent your users from indefinitely using new versions of the software you depend on.

[Robert P. Goldman (rpgoldman)]

I'm definitely ok with a breaks declaration. But let's move that to a separate ticket so that it's more easily actionable. Thanks.

[Antonis Kanouras (akanouras)]

Just another data point:

Python's pip and Ruby's Bundler also support an upper version bound on dependencies, for example:

;; In requirements.txt:
Django>=1.7,<1.8

;; In gemfile:
gem "RedCloth", ">= 4.1.0", "< 4.2.0"