Need better support for logfiles

[James Slagle]

The only way to save the log from a run of diskimage-builder is to redirect or pipe the output to a file. When using these methods, you lose the buffered output on the command line from any programs executed during the image create. The big culprit here is yum specifically with it's download and installation status buffered output. Without seeing this output, it's often hard to tell if you've ended up pointed at a very slow mirror (or one that throttles aggressively). Slow mirrors can result in very slow image builds with not a lot of feedback of what's going on.

Ideally, diskimage-builder needs a way to write to a log file and preserve buffered output on the terminal as well.

[Mark McLoughlin (markmc)]

See also https://review.openstack.org/50295

[Robert Collins (lifeless)]

The 'ideally' here seems like 'unbreak yum' ? I think this should be filed upstream, on yum - short of playing nasty tricks with tty's, there's nothing that diskimage-builder can do - and in fact, that sort of nasty trick shouldn't live in diskimage-builder, but in a reusable tool.

[James Slagle (james-slagle)]

I agree that yum handling of the slow mirror is not great. There are already most likely already upstream bugs (I haven't searched) about this issue. There in fact used to be a way to tell it to switch to a different mirror on demand during a download, but that got removed.

But, it's how you have to call diskimage-builder in order to get a log file saved that is the reason that we can't see the output from yum. If we didn't have to call it with "2>&1 | tee foo", then we could see the output from yum that's already telling you that you got stuck with a slow mirror.

Just even forgetting yum for a moment, I think diskimage-builder needs logging. Unless you remember to run it (or we tell people to do so 100% of the time) with redirection and tee, there's no log. That seems strange to me to rely on users forethought and good intuition to capture the output of every command you run from your shell. It instead seems likely that a program like diskimage-builder that has hundreds/thousands of lines of output to log something on it's own.

And in fact, I think the pattern is often:
- run image build, it fails
- realize you forgot to tee the output
- run image build again, this time with output saved