Install progress should display curtin reporter output

[Mathieu Trudel-Lapierre]

When installing on system with slow IO, there's not enough information reported back to keep user confidence that things are still going OK. We should have a separate call out to the reporter to fetch the current status and publish that to the install progress screen.

With PR #124 merged, we're back to a two-stage curtin install and observing the output of curtin in the progress view. Some of the next bit of work is related to curtin itself w.r.t what level of messages come out. We may in the mean time apply a filter to the stream and classify what's happening:

Stage 1: Curtin Install

    Configuring Installer
    Preparing Storage
    Installing OS
    Configuring OS

Stage 2: Curtin Postinstall

    Customizing OS

[Michael Hudson-Doyle (mwhudson)]

Is it easy to filter these out of the curtin log? It would be pretty easy to do this from the subiquity side now.

[Ryan Harper (raharper)]

On Wed, Mar 15, 2017 at 8:05 PM, Michael Hudson-Doyle <
<email address hidden>> wrote:

> Is it easy to filter these out of the curtin log? It would be pretty
> easy to do this from the subiquity side now.
>

curtin reporting[1] will return events as they happen with timestamps;
 However, there are some
*long* poll events that do not have any interactive feedback at this time;
for example, the rsync
(which is done async); as well as any package upgrade/install; if a kernel
package needs updated
this is a bit of a blackhold (curtin will invoke apt or dpkg to do this
which we don't have a mechanism
to bubble an progress there).

w.r.t to filtering, we can re-introduce filtering inside the reporter,
however we may want to
configure subiquity to accept the events and filter itself.

1. http://curtin.readthedocs.io/en/latest/topics/reporting.html

Here's a pretty-print of an install which sent back events via webhook.

% ./tools/curtin-log-print
output/ZestyTestMdadmBcache/logs/webhooks-events.json
The total time elapsed since completing an event is printed after the "@"
character.
The time the event takes is printed after the "+" character.

[00.00000] curtin command install
|`->preparing for installation @00.05052s +00.01165s
| `->removing previous storage devices @00.57202s +00.15468s
| `->configuring disk: sda @01.72016s +01.14527s
| `->configuring partition: bios_boot_partition @03.12751s +01.40073s
| `->configuring partition: sda1 @04.73120s +01.59648s
| `->configuring partition: sda2 @06.53184s +01.79702s
| `->configuring partition: sda3 @08.50274s +01.96819s
| `->configuring partition: sda4 @10.64785s +02.13898s
| `->configuring partition: sda5 @12.91095s +02.25991s
| `->configuring partition: sda6 @15.35934s +02.44189s
| `->configuring disk: sdb @15.43740s +00.07361s
| `->configuring disk: sdc @16.57825s +01.13677s
| `->configuring partition: sdc1 @18.11159s +01.52621s
| `->configuring raid: mddevice @19.98164s +01.85602s
| `->configuring bcache: bcache1_raid @20.66754s +00.67885s
| `->configuring bcache: bcache_normal @22.02653s +01.35241s
| `->configuring bcache: bcachefoo @22.63083s +00.59545s
| `->configuring format: sda1_fs @24.02448s +01.38501s
| `->configuring format: bcache_raid_storage @24.29879s +00.26120s
| `->configuring format: bcache_normal_storage @24.87102s +00.56441s
| `->configuring format: bcachefoo_fulldiskascache_storage @25.33135s
+00.45056s
| `->configuring mount: sda1_mount @25.75830s +00.41929s
| `->configuring mount: bcache1_raid_mount @25.94002s +00.17267s
| `->configuring mount: bcache0_mount @26.72055s +00.77280s
| `->configuring mount: sda1_non_root_mount @27.02695s +00.29974s
| `->curtin command block-meta @27.03383s +00.30662s
| `->running 'curtin block-meta simple' @27.07044s +27.01338s
|`->configuring storage @27.07817s +27.02310s
| `->curtin command net-meta @27.46295s +00.01612s
| `->running 'curtin net-meta auto' @27.49044s +00.40116s
|`->configuring network @27.49830s +00.41487s
| `->acquiring and extracting image from
http://10.245.168.20:21066//vmtest.root-tgz @41.56700s +13.69421s
| `->curtin command extract @41.57522s +13.70242s
| `->running 'curtin extract' @4...

[Michael Hudson-Doyle (mwhudson)]

Ah that's nice. I think it makes sense to configure subiquity to receive
the webhooks and shouldn't be too hard.

[Michael Hudson-Doyle (mwhudson)]

https://github.com/CanonicalLtd/subiquity/pull/230

[Michael Hudson-Doyle (mwhudson)]

A version of this has landed now. We might need to iterate on how the reporter output is presented but it's a start.