octavia image upload logic doesn't work properly

Bug #1843059 reported by Brent Eagles
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
High
Brent Eagles

Bug Description

Apologies for poor summary but the ansible for updating the amphora images has several issues. The original bug report is a RedHat bugzilla report https://bugzilla.redhat.com/show_bug.cgi?id=1745643

The output of the queries has flaws in how it handles output and error codes. Note that ideally this should be migrated to openstack ansible but for the time being the current logic should be fixed.

- the output of the query for detecting existing amphora images is not properly checked so empty output is used in subsequent calls which leads to invalid commands
- we do not check that the contents of stdout for command output to see if it is non-empty before using it in key places
- we do qcow2->raw conversion whether an image needs to be updated or not.

Brent Eagles (beagles)
Changed in tripleo:
importance: Undecided → High
assignee: nobody → Brent Eagles (beagles)
milestone: none → train-3
status: New → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/680756

Changed in tripleo:
status: Triaged → In Progress
Changed in tripleo:
milestone: train-3 → ussuri-1
Changed in tripleo:
milestone: ussuri-1 → ussuri-2
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-2 → ussuri-3
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-3 → ussuri-rc3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/721715

wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-rc3 → victoria-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-ansible (master)

Reviewed: https://review.opendev.org/721715
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=5fccdd3c2ce9a6259b1b38e950e638ad0b2c46fc
Submitter: Zuul
Branch: master

commit 5fccdd3c2ce9a6259b1b38e950e638ad0b2c46fc
Author: Brent Eagles <email address hidden>
Date: Tue Apr 21 16:56:59 2020 -0230

    Octavia: handle empty amphora image search result

    In some situations, openstack image list queries on name return a 0
    exit code even though there is no result.

    Change-Id: I20f63ea45e52181810654f7afab11499fef9baa2
    Related-Bug: #1843059

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/735567

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/735568

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-ansible (stable/ussuri)

Reviewed: https://review.opendev.org/735567
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=2e9959649be1b3b9a3a72bae26aae12e2e18403e
Submitter: Zuul
Branch: stable/ussuri

commit 2e9959649be1b3b9a3a72bae26aae12e2e18403e
Author: Brent Eagles <email address hidden>
Date: Tue Apr 21 16:56:59 2020 -0230

    Octavia: handle empty amphora image search result

    In some situations, openstack image list queries on name return a 0
    exit code even though there is no result.

    Change-Id: I20f63ea45e52181810654f7afab11499fef9baa2
    Related-Bug: #1843059
    (cherry picked from commit 5fccdd3c2ce9a6259b1b38e950e638ad0b2c46fc)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-ansible (stable/train)

Reviewed: https://review.opendev.org/735568
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=2d1a48b505f8adcfb3e73c5c0c0f3e3fc7c068ca
Submitter: Zuul
Branch: stable/train

commit 2d1a48b505f8adcfb3e73c5c0c0f3e3fc7c068ca
Author: Brent Eagles <email address hidden>
Date: Tue Apr 21 16:56:59 2020 -0230

    Octavia: handle empty amphora image search result

    In some situations, openstack image list queries on name return a 0
    exit code even though there is no result.

    Note: this differs slightly from the original commit because the linters
    are catching a new rule "comparison against empty strings" since the
    original commit merged.

    Change-Id: I20f63ea45e52181810654f7afab11499fef9baa2
    Related-Bug: #1843059
    (cherry picked from commit 5fccdd3c2ce9a6259b1b38e950e638ad0b2c46fc)

tags: added: in-stable-train
Changed in tripleo:
milestone: victoria-1 → victoria-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-ansible (master)

Change abandoned by wes hayutin (<email address hidden>) on branch: master
Review: https://review.opendev.org/680756
Reason: https://specs.openstack.org/openstack/tripleo-specs/specs/policy/patch-abandonment.html

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/788507

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/788508

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/788509

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/788507
Committed: https://opendev.org/openstack/tripleo-ansible/commit/4fcd5cac1e0e80552c0bc2acd70b37915593be17
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 4fcd5cac1e0e80552c0bc2acd70b37915593be17
Author: Brent Eagles <email address hidden>
Date: Fri Sep 6 15:45:52 2019 -0230

    Move away from md5 digests in managing octavia amphora images

    This patch changes the image management code from using deprecated md5
    checksum methods to sha digests.

    This patch also fixes amphora update code so it uses the checksum of the
    image after conversion to raw if it raw conversion was enabled.

    Closes-Bug: #1843059
    Change-Id: I1817f11bcce90ab5ac29ea3bbf30b3dbf488de5f
    (cherry picked from commit 7d212d68c5a147da39b119bde5f2c9e40af4b617)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/788508
Committed: https://opendev.org/openstack/tripleo-ansible/commit/5ded5fd7ccc348af38413d1e1d2c7206d65c8e9e
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 5ded5fd7ccc348af38413d1e1d2c7206d65c8e9e
Author: Brent Eagles <email address hidden>
Date: Fri Sep 6 15:45:52 2019 -0230

    Move away from md5 digests in managing octavia amphora images

    This patch changes the image management code from using deprecated md5
    checksum methods to sha digests.

    This patch also fixes amphora update code so it uses the checksum of the
    image after conversion to raw if it raw conversion was enabled.

    Closes-Bug: #1843059
    Change-Id: I1817f11bcce90ab5ac29ea3bbf30b3dbf488de5f
    (cherry picked from commit 7d212d68c5a147da39b119bde5f2c9e40af4b617)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (stable/train)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/788509
Committed: https://opendev.org/openstack/tripleo-ansible/commit/3630862668e23d51946c8bcac566e7b5ebfbaa50
Submitter: "Zuul (22348)"
Branch: stable/train

commit 3630862668e23d51946c8bcac566e7b5ebfbaa50
Author: Brent Eagles <email address hidden>
Date: Fri Sep 6 15:45:52 2019 -0230

    Move away from md5 digests in managing octavia amphora images

    This patch changes the image management code from using deprecated md5
    checksum methods to sha digests.

    This patch also fixes amphora update code so it uses the checksum of the
    image after conversion to raw if it raw conversion was enabled.

    Closes-Bug: #1843059
    Change-Id: I1817f11bcce90ab5ac29ea3bbf30b3dbf488de5f
    (cherry picked from commit 7d212d68c5a147da39b119bde5f2c9e40af4b617)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/789429

Revision history for this message
Brent Eagles (beagles) wrote :

Fixes have merged for

- the output of the query for detecting existing amphora images is not properly checked so empty output is used in subsequent calls which leads to invalid commands
- we do not check that the contents of stdout for command output to see if it is non-empty before using it in key places

The last one in #c15 should close the bug.

Serves me right for filing an omnibus bug even if they were small issues.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-ansible 3.1.0

This issue was fixed in the openstack/tripleo-ansible 3.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-ansible (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/789429
Committed: https://opendev.org/openstack/tripleo-ansible/commit/820f84ef42a9c36c677481760989ad36f13de4c0
Submitter: "Zuul (22348)"
Branch: master

commit 820f84ef42a9c36c677481760989ad36f13de4c0
Author: Brent Eagles <email address hidden>
Date: Mon May 3 22:25:28 2021 -0230

    Do not convert amphora images to raw if they are already raw

    Adds a check on the provided image to see if it is raw before conversion
    (if requested).

    Change-Id: I79b03709deffe6cb788aba9e24c63ca33bd297ee
    Related-Bug: #1843059

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/792453

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/792455

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (stable/victoria)

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/792456

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/792457

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-ansible 2.3.0

This issue was fixed in the openstack/tripleo-ansible 2.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-ansible 0.8.0

This issue was fixed in the openstack/tripleo-ansible 0.8.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-ansible (stable/train)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/792457
Committed: https://opendev.org/openstack/tripleo-ansible/commit/0d1ebe943a85476d8f8c6be9e881eeb519a4f5fe
Submitter: "Zuul (22348)"
Branch: stable/train

commit 0d1ebe943a85476d8f8c6be9e881eeb519a4f5fe
Author: Brent Eagles <email address hidden>
Date: Mon May 3 22:25:28 2021 -0230

    Do not convert amphora images to raw if they are already raw

    Adds a check on the provided image to see if it is raw before conversion
    (if requested).

    Change-Id: I79b03709deffe6cb788aba9e24c63ca33bd297ee
    Related-Bug: #1843059
    (cherry picked from commit 820f84ef42a9c36c677481760989ad36f13de4c0)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-ansible (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/792455
Committed: https://opendev.org/openstack/tripleo-ansible/commit/1a171e2db1c85c94a464c5530f652926805b276e
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 1a171e2db1c85c94a464c5530f652926805b276e
Author: Brent Eagles <email address hidden>
Date: Mon May 3 22:25:28 2021 -0230

    Do not convert amphora images to raw if they are already raw

    Adds a check on the provided image to see if it is raw before conversion
    (if requested).

    Change-Id: I79b03709deffe6cb788aba9e24c63ca33bd297ee
    Related-Bug: #1843059
    (cherry picked from commit 820f84ef42a9c36c677481760989ad36f13de4c0)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-ansible (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/792453
Committed: https://opendev.org/openstack/tripleo-ansible/commit/4b784bd8c6b94c35ddf33ae7873bbc32d669bf17
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 4b784bd8c6b94c35ddf33ae7873bbc32d669bf17
Author: Brent Eagles <email address hidden>
Date: Mon May 3 22:25:28 2021 -0230

    Do not convert amphora images to raw if they are already raw

    Adds a check on the provided image to see if it is raw before conversion
    (if requested).

    Change-Id: I79b03709deffe6cb788aba9e24c63ca33bd297ee
    Related-Bug: #1843059
    (cherry picked from commit 820f84ef42a9c36c677481760989ad36f13de4c0)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-ansible (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/792456
Committed: https://opendev.org/openstack/tripleo-ansible/commit/9a34d5623111acb527d77f72e7b2312b28ca7efd
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 9a34d5623111acb527d77f72e7b2312b28ca7efd
Author: Brent Eagles <email address hidden>
Date: Mon May 3 22:25:28 2021 -0230

    Do not convert amphora images to raw if they are already raw

    Adds a check on the provided image to see if it is raw before conversion
    (if requested).

    Change-Id: I79b03709deffe6cb788aba9e24c63ca33bd297ee
    Related-Bug: #1843059
    (cherry picked from commit 820f84ef42a9c36c677481760989ad36f13de4c0)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-ansible 1.5.4

This issue was fixed in the openstack/tripleo-ansible 1.5.4 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers