[DOC] MAAS documentation needs apt-mirror documentation.

[Rod Smith]

Running MAAS 1.7 from the MAAS Stable PPA (ppa:maas-maintainers/stable), using the web UI (http://localhost/MAAS/images/) fails to import boot images. I've tried selecting various combinations of release and architecture (usually 14.04 and 12.04 with i386 and amd64, or some subset of those). These images fail to download, with no useful feedback in the web UI and not much better in log files (attached). Switching to the standard archive.ubuntu.com archive site works around the problem. The issue really is one of logging, since the log files refer to a failure to retrieve grubnetx64.efi.signed from "the archive," but fail to specify WHICH archive.

$ dpkg -l '*maas*'|cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=====================================================-===================================================-============-===============================================================================
ii maas 1.7.0+bzr3299-0ubuntu1~trusty all MAAS server all-in-one metapackage
ii maas-cli 1.7.0+bzr3299-0ubuntu1~trusty all MAAS command line API tool
ii maas-cluster-controller 1.7.0+bzr3299-0ubuntu1~trusty all MAAS server cluster controller
ii maas-common 1.7.0+bzr3299-0ubuntu1~trusty all MAAS server common files
ii maas-dhcp 1.7.0+bzr3299-0ubuntu1~trusty all MAAS DHCP server
ii maas-dns 1.7.0+bzr3299-0ubuntu1~trusty all MAAS DNS server
ii maas-proxy 1.7.0+bzr3299-0ubuntu1~trusty all MAAS Caching Proxy
ii maas-region-controller 1.7.0+bzr3299-0ubuntu1~trusty all MAAS server complete region controller
ii maas-region-controller-min 1.7.0+bzr3299-0ubuntu1~trusty all MAAS Server minimum region controller
ii python-django-maas 1.7.0+bzr3299-0ubuntu1~trusty all MAAS server Django web framework
ii python-maas-client 1.7.0+bzr3299-0ubuntu1~trusty all MAAS python API client
ii python-maas-provisioningserver 1.7.0+bzr3299-0ubuntu1~trusty all MAAS server provisioning libraries

[Jeff Lane  (bladernr)]

This bug is a two parter. Part one involves an incomplete mirror of archive.ubuntu.com. Jason Hobbs has a writeup on properly mirroring the boot resources to fix that issue. The second part is about the error output itself. In this case, the OOPS and Tracebacks are pretty generic:

2014-11-30 08:54:31-0500 [-] Unhandled Error
        Traceback (most recent call last):
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 423, in errback
            self._startRunCallbacks(fail)
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 490, in _startRunCallbacks
            self._runCallbacks()
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 577, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 1155, in gotResult
            _inlineCallbacks(r, g, deferred)
        --- <exception caught here> ---
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 1097, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/usr/lib/python2.7/dist-packages/provisioningserver/pserv_services/image_download_service.py", line 135, in maybe_start_download
            yield self._start_download()
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/usr/lib/python2.7/dist-packages/provisioningserver/pserv_services/image_download_service.py", line 121, in _start_download
            get_proxy_url("https"))
2014-11-30 08:54:31-0500 [-] Unhandled Error
        Traceback (most recent call last):
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 423, in errback
            self._startRunCallbacks(fail)
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 490, in _startRunCallbacks
            self._runCallbacks()
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 577, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 1155, in gotResult
            _inlineCallbacks(r, g, deferred)
        --- <exception caught here> ---
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", lin
e 1097, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/usr/lib/python2.7/dist-packages/provis...

[Rod Smith (rodsmith)]

FWIW, the writeup to which Jeff refers is:

https://docs.google.com/a/canonical.com/document/d/1jrBA4KeKb2oTjM3eHQJx0bxHe9LnMGheYWOiJjxzWAI/edit#

The key is this:

> MAAS also requires some grub net booting files, which aren’t included in the content apt-mirror typically mirrors. These can be included by adding this line (as one line) to /var/spool/apt-mirror/var/postmirror.sh:
>
> wget -nH --cut-dirs 4 -r --no-parent -P /var/spool/apt-mirror/mirror/tw.archive.ubuntu.com/ubuntu/dists/trusty-updates/main/ http://tw.archive.ubuntu.com/ubuntu/dists/trusty/main/uefi/grub2-amd64/

The mirror issue isn't really a MAAS bug per se, but this needs to be documented whenever a mirror to be used for MAAS is documented -- or maybe apt-mirror should be updated. (Should a bug be filed against apt-mirror?) As Jeff says, better error logging would be extremely helpful.

[Mike Pontillo (mpontillo)]

Here's my workaround. (I used rsync rather than wget)

rsync -xav rsync://us.archive.ubuntu.com/ubuntu/dists/trusty/main/uefi /home/mirror/ubuntu/dists/trusty/main/uefi

[Mike Pontillo (mpontillo)]

Oops, missed a trailing /. It should be:

rsync -xav rsync://us.archive.ubuntu.com/ubuntu/dists/trusty/main/uefi/ /home/mirror/ubuntu/dists/trusty/main/uefi

[mahmoh (mahmoh)]

It would be nice for customers to have this noted in the UI comments or a pop-up when you populate that information: "Does your archive have the <link-to-post-mirror.sh> grub uefi </> binaries in it not normally included in mirrors?"

[Andres Rodriguez (andreserl)]

Hi!

We believe this is no longer an issue in the latest MAAS releases. As such, we are marking this bug report as invalid. If you believe this is still an issue, please re-open this bug report.

Thanks