Mirror jobs should not copy new packages if installation fails.

[Clark Boylan]

http://logs.openstack.org/periodic/periodic-mirror-python27/140/console.html shows a mirror job run that attempted to install xattr 0.7.0. This failed because xattr depends on cffi which depends on the system libffi-dev package being present (it was not). Despite failing to properly install xattr 0.7.0 was copied to the mirror and broke testing for all projects that depend on xattr.

[Clark Boylan (cboylan)]

There has been some confusion on what the failure mode of the mirror jobs should be. We will have a summit session to talk openstack/requirements and the mirror to sort out what has worked, what hasn't and how we can make it better. This particular bug should be brought up there.

[Jeremy Stanley (fungi)]

This was only minimally touched on at the summit sessions on requirements. The takeaway though is that our mirror should, whenever possible, reflect any of the same breakage exposed upstream in PyPI and not try to shield our CI processes from it... proposals were floated to perform some sort of notification of breakage during mirror builds and then fall back on safe/working behavior instead, but the primary concern there is visibility. The compromise with the most support was if we had some process to automagically open bugs when this happens, and include the list of packages added when that happened.

There was also a related proposals for preemptive testing of upstream dependencies to provide feedback for us (and possibly them depending on openness to it and proximity of our communities) as an early warning system for these sorts of issues.

[James E. Blair (corvus)]

We now use bandersnatch so most of this bug is no longer relevant.