[iso] research docker image creator for Fuel

Bug #1336911 reported by Matthew Mosesohn
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Matthew Mosesohn
6.0.x
Invalid
Wishlist
Matthew Mosesohn

Bug Description

Currently we use ami-creator, which relies on centos-specific build information. We either need to create a CentOS chroot or find a native ubuntu tool that can do our CentOS base docker image.

Tags: devops
Changed in fuel:
milestone: none → 6.0
assignee: nobody → Matthew Mosesohn (raytrac3r)
Changed in fuel:
importance: Undecided → Medium
Revision history for this message
Dmitry Ilyin (idv1985) wrote :
Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

I believe it should this bug converted to blueprint

Dmitry Ilyin (idv1985)
summary: - research docker image creator for Fuel
+ [iso] research docker image creator for Fuel
Changed in fuel:
status: New → Confirmed
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

We can run ami-creator inside a centos Docker container, then return the built artifact to the build host (presumably Ubuntu) to do the final preparation and image tagging for import into Docker.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (master)

Fix proposed to branch: master
Review: https://review.openstack.org/113283

Changed in fuel:
status: Confirmed → In Progress
Changed in fuel:
assignee: Matthew Mosesohn (raytrac3r) → Stanislaw Bogatkin (sbogatkin)
Changed in fuel:
assignee: Stanislaw Bogatkin (sbogatkin) → Matthew Mosesohn (raytrac3r)
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :
Changed in fuel:
milestone: 6.0 → 6.1
status: In Progress → Triaged
Changed in fuel:
status: Triaged → In Progress
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

waiting on https://bugs.launchpad.net/fuel/+bug/1376733 to put ami-creator in stable mirror. Code is ready for merge when this is done.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-main (master)

Reviewed: https://review.openstack.org/113283
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=333ff0219d1ad8217977061dd2fdd3cfa8d10c14
Submitter: Jenkins
Branch: master

commit 333ff0219d1ad8217977061dd2fdd3cfa8d10c14
Author: Matthew Mosesohn <email address hidden>
Date: Mon Aug 11 18:53:01 2014 +0400

    Add fuel-centos-build container for fuel/centos

    fuel/centos will now be built from CentOS6
    container dynamically with current package set
    instead of being periodically rebuilt. It uses
    ami-creator, which only runs on CentOS. It is
    run inside Docker so it can be used on an
    Ubuntu build host. After the image is created,
    img2docker loads the content into Docker as a
    Docker image.

    Consolidates save/load to a single archive
    instead of rolling up separate container tar
    files.

    Change-Id: Ibff08d309fe091cc40d053781b7556fd36ebf173
    Closes-Bug: #1336911

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (master)

Fix proposed to branch: master
Review: https://review.openstack.org/155686

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/156204

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-main (master)

Reviewed: https://review.openstack.org/156204
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=7c001aa79d7d23b51eddcb12c4f5fc6eb2771055
Submitter: Jenkins
Branch: master

commit 7c001aa79d7d23b51eddcb12c4f5fc6eb2771055
Author: Matthew Mosesohn <email address hidden>
Date: Mon Feb 16 15:38:46 2015 +0300

    Add ami-creator package

    ami-creator package will be used for generating
    fuel/centos Docker image. This is committed separately
    because of the package requirement.

    Change-Id: I180bd7ef05450dc63384bf7239427c46c1781742
    Partial-Bug: #1336911

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/155686
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=381491f7e884361124292857b4cc1b7c0ea4aaf5
Submitter: Jenkins
Branch: master

commit 381491f7e884361124292857b4cc1b7c0ea4aaf5
Author: Matthew Mosesohn <email address hidden>
Date: Fri Feb 13 09:08:40 2015 +0000

    Add fuel-centos-build container for fuel/centos

    fuel/centos will now be built from CentOS6
    container dynamically with current package set
    instead of being periodically rebuilt. It uses
    ami-creator, which only runs on CentOS. It is
    run inside Docker so it can be used on an
    Ubuntu build host. After the image is created,
    img2docker loads the content into Docker as a
    Docker image.

    Consolidates save/load to a single archive
    instead of rolling up separate container tar
    files.

    Note that fuel-centos.done target depends on
    RPM package build to avoid race conditions
    where repo metadata is updated while base image
    is being built.

    Change-Id: I62abc07e79847b5c69eaa1a983575b214ebed649
    Closes-Bug: #1336911

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-main (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/168392

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-web (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/168399

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-main (master)

Reviewed: https://review.openstack.org/168392
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=d09dd7f5a3e0f4f9082b8df1466260d9b904de3a
Submitter: Jenkins
Branch: master

commit d09dd7f5a3e0f4f9082b8df1466260d9b904de3a
Author: Igor Kalnitsky <email address hidden>
Date: Fri Mar 27 17:52:35 2015 +0200

    upgrade: keep docker images in one tarball

    The patch I62abc07e79847b5c69eaa1a983575b214ebed649 introduced new
    content for fuel-images.tar.lrz artifact: now we don't have images
    per container. Basically it means that the fuel_upgrade script is
    unable to load them and we need to distribute the fuel-images.tar
    (exactly tarball, not unpacked content) in order to load all
    images at once.

    Change-Id: I15f5bf771cb95504791a55a02e412a40f330ecd9
    Related-Bug: #1336911

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/168399
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=900cd9108fa9d0f96e0ce40836210db80ad839d2
Submitter: Jenkins
Branch: master

commit 900cd9108fa9d0f96e0ce40836210db80ad839d2
Author: Igor Kalnitsky <email address hidden>
Date: Fri Mar 27 18:09:19 2015 +0200

    Upload Docker images from fuel-images.tar

    Since the patch I62abc07e79847b5c69eaa1a983575b214ebed649 we have a new
    format of 'fuel-images.tar.lrz' artifact, so now it doesn't contain
    images per container. Hence, we have to change the fuel_upgrade script
    in order to fit these changes.

    Basically it means that the script have to load just one file with all
    images - fuel-images.tar.

    Closes-Bug: #1430221
    Closes-Bug: #1435314
    Related-Bug: #1336911

    Depends on: I15f5bf771cb95504791a55a02e412a40f330ecd9
    Change-Id: I7f5bad9315edffe6f4e1feb29c471fb7a5bfd22a

Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

Moved bug to high. https://review.openstack.org/#/c/177868/ needs to be merged to improve user experience of dockerctl backup with large base images. Not merging will result in ~1 hour backup time for a full container backup.

Changed in fuel:
importance: Medium → High
Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

+1. Agree with Matt

Changed in fuel:
status: Fix Committed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/177868
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=daadbb91fd9eb839c26ef4bac3044b3b7fca2b91
Submitter: Jenkins
Branch: master

commit daadbb91fd9eb839c26ef4bac3044b3b7fca2b91
Author: Matthew Mosesohn <email address hidden>
Date: Mon Apr 27 19:46:31 2015 +0300

    Backup all container images together

    With common fat base image for docker containers,
    it is much faster to back up all of them together.
    Additionally, this is now possible since Docker
    upgrade beyond 1.2.

    Change-Id: Icbf3497d58b6f880eb945a952cb23dcdca98e9b1
    Related-Bug: #1336911

Changed in fuel:
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.