Vagrant <2.x can no longer fetch box metadata from Hashicorp

[Tom Hirst-Dunton]

[Impact]

As of February 2nd, the legacy Vagrant Cloud URLs were disabled by Hashicorp[0], meaning that the versions of vagrant in xenial and artful can no longer fetch Vagrant boxes from the global index (which is a fundamental part of Vagrant use).

[0] https://groups.google.com/forum/#!topic/vagrant-up/H8C68UTkosU

[Test Case]

Before fix:

$ vagrant box add 'ubuntu/trusty64'
[...]
URL: ["https://atlas.hashicorp.com/ubuntu/trusty64"]
Error: The requested URL returned error: 404 Not Found

After fix:

$ vagrant box add 'ubuntu/trusty64'
==> box: Loading metadata for box 'ubuntu/trusty64'
    box: URL: https://vagrantcloud.com/ubuntu/trusty64
[...]

[Regression Potential]

These changes were cherry-picked directly from upstream (where they have been for several upstream releases), and they only affect the parts of the code concerned with the fetching of boxes (which doesn't work at all currently). I believe the potential for regression outside this functionality is low.

[Original Report]

When trying to update my ubuntu/trusty64 vagrant box, I get the following failure:

    $ vagrant box update --box 'ubuntu/trusty64'
    Checking for updates to 'ubuntu/trusty64'
    Latest installed version: 20180124.0.0
    Version constraints: > 20180124.0.0
    Provider: virtualbox
    There was an error while downloading the metadata for this box.
    The error message is shown below:

    The requested URL returned error: 404 Not Found

I have the following Vagrant version:

    $ vagrant --version
    Vagrant 2.0.2

and am able to access the resources where I assume they are hosted:

    $ curl -O http://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-amd64-vagrant-disk1.box
      % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
      4 425M 4 17.1M 0 0 1062k 0 0:06:50 0:00:16 0:06:34 972k

[Patricia Gaughen (gaughen)]

Thanks for reporting. We're looking into the issue.

[Dan Watkins (daniel-thewatkins)]

It looks like Hashicorp have discontinued the atlas.hashicorp.com API endpoints used by the version of Vagrant in every Ubuntu release (in favour of vagrantcloud.com).

I've confirmed that this is fixed in the version of Vagrant in bionic-proposed:

# vagrant box add 'ubuntu/trusty64'
==> box: Loading metadata for box 'ubuntu/trusty64'
    box: URL: https://vagrantcloud.com/ubuntu/trusty64
==> box: Adding box 'ubuntu/trusty64' (v20180125.0.0) for provider: virtualbox
    box: Downloading: https://vagrantcloud.com/ubuntu/boxes/trusty64/versions/20180125.0.0/providers/virtualbox.box
...

I'm looking in to this further.

[Dan Watkins (daniel-thewatkins)]

So this looks like it may not be an issue in trusty, because I believe that predates Atlas/Vagrant Cloud in their entirety. I'm preparing patches for xenial/artful.

[Adam Conrad (adconrad)]

Hello Tom, or anyone else affected,

Accepted vagrant into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/vagrant/1.9.1+dfsg-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Łukasz Zemczak (sil2100)]

Hello Tom, or anyone else affected,

Accepted vagrant into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/vagrant/1.8.1+dfsg-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Dan Watkins (daniel-thewatkins)]

I performed the following test in each of a xenial and artful container:

* Launch the container
* Enable -proposed for universe
* apt install vagrant
* vagrant box add 'ubuntu/trusty64'

and confirmed that box download started successfully.

[Launchpad Janitor (janitor)]

This bug was fixed in the package vagrant - 1.8.1+dfsg-1ubuntu0.1

---------------
vagrant (1.8.1+dfsg-1ubuntu0.1) xenial; urgency=medium

  * debian/patches/0013-Switch-to-vagrantcloud.patch: Switch from Atlas
    to Vagrant Cloud. Closes LP: #1747426.

 -- Daniel Watkins <email address hidden> Mon, 05 Feb 2018 18:00:06 -0500

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for vagrant has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Brian Murray (brian-murray)]

There are some autopkgtests failing in artful with the new version of vagrant that look legitimate. Could someone have a look?

[Balint Reczey (rbalint)]

On Tue, Feb 20, 2018 at 2:30 AM, Brian Murray <email address hidden> wrote:
> There are some autopkgtests failing in artful with the new version of
> vagrant that look legitimate. Could someone have a look?

I see the s390x ones, but there are already hints for ignoring them
because they never passed, just were skipped.

[Balint Reczey (rbalint)]

On Tue, Feb 20, 2018 at 2:17 PM, Balint Reczey
<email address hidden> wrote:
> On Tue, Feb 20, 2018 at 2:30 AM, Brian Murray <email address hidden> wrote:
>> There are some autopkgtests failing in artful with the new version of
>> vagrant that look legitimate. Could someone have a look?
>
> I see the s390x ones, but there are already hints for ignoring them
> because they never passed, just were skipped.

The hints were set only for bionic, this MP adds them to artful:
https://code.launchpad.net/~rbalint/britney/vagrant-and-other-s390x-hints-artful/+merge/338261

[Launchpad Janitor (janitor)]

This bug was fixed in the package vagrant - 1.9.1+dfsg-1ubuntu0.1

---------------
vagrant (1.9.1+dfsg-1ubuntu0.1) artful; urgency=medium

  * debian/patches/0008-Switch-to-vagrantcloud.patch: Switch from Atlas
    to Vagrant Cloud. Closes LP: #1747426.

 -- Daniel Watkins <email address hidden> Mon, 05 Feb 2018 14:47:35 -0500