[RFE] Stop storing configdrive in the database

Bug #1596421 reported by David Edery on 2016-06-27
28
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Ironic
Fix Released
Wishlist
Julia Kreger

Bug Description

In the Ironic database the 'instance_info' field as stored as a TEXT field. With MySQL this has a limit of 64KiB. The configdrive is stored in the instance_info field and so can not be larger than 64KiB. We have reports from multiple users who would like to be able to use configdrives larger than 64KIB.

Proposal is to switch to using the MEDIUMTEXT field type for instance_info for MySQL databases, which would allow up to 16MiB for the instance_info.

Also propose to have a configuration setting for the maximum size for the configdrive to by default be set to 64KiB. Database entry would still be MEDIUMTTEXT. Any configdrive that exceeds the config setting would be rejected. This would maintain the current size limitation and operators would not be surprised by their database growing unexpectedly. Operators could then change the config setting to increase the maximum size of the configdrive.

Update from dtantsur: since bug #1575935 is fixed, we no longer need to store the configdrive in the database at all.

David Edery (david-edery) wrote :
Changed in ironic:
assignee: nobody → David Edery (david-edery)
David Edery (david-edery) wrote :

I'm going to follow Mistral's solution to the same text-is-too-short-for-me issue:
https://github.com/openstack/mistral/blob/master/mistral/db/sqlalchemy/types.py#L86

TBD: medium vs long text type

David Edery (david-edery) wrote :

TBD: other db flavors in which the same solution is needed

Fix proposed to branch: master
Review: https://review.openstack.org/334967

Changed in ironic:
status: New → In Progress
Changed in ironic:
importance: Undecided → Medium
summary: - nodes are not powered-on due to short nodes.instance_info
+ node deployment fails when user passes a long cloud-init script
summary: - node deployment fails when user passes a long cloud-init script
+ RFE: Increase size of data base entry for instance_info to allow
+ configdrives larger than 64KB
tags: added: rfe
description: updated
summary: RFE: Increase size of data base entry for instance_info to allow
- configdrives larger than 64KB
+ configdrives larger than 64KiB
description: updated

Fix proposed to branch: master
Review: https://review.openstack.org/369617

Changed in ironic:
assignee: David Edery (david-edery) → John L. Villalovos (happycamp)
Changed in ironic:
importance: Medium → Wishlist

We've discussed this, and the real solution is to do this: https://bugs.launchpad.net/ironic/+bug/1575935

Going to close this RFE.

Changed in ironic:
status: In Progress → Invalid
Ruby Loo (rloo) wrote :

The discussion was at Monday's ironic meeting. Starting at 17:31:51: http://eavesdrop.openstack.org/meetings/ironic/2016/ironic.2016-11-21-17.00.log.html

Change abandoned by John L. Villalovos (<email address hidden>) on branch: master
Review: https://review.openstack.org/369617

Dmitry Tantsur (divius) on 2018-02-13
summary: - RFE: Increase size of data base entry for instance_info to allow
- configdrives larger than 64KiB
+ [RFE] Stop storing configdrive in the database
Changed in ironic:
status: Invalid → Confirmed
assignee: John L. Villalovos (happycamp) → nobody
description: updated
Changed in ironic:
assignee: nobody → Dmitry Tantsur (divius)
status: Confirmed → In Progress
Changed in ironic:
assignee: Dmitry Tantsur (divius) → Julia Kreger (juliaashleykreger)

Reviewed: https://review.openstack.org/334967
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=8fdf752ae9688c581fe8c977e8a5ffb9432583f3
Submitter: Zuul
Branch: master

commit 8fdf752ae9688c581fe8c977e8a5ffb9432583f3
Author: David Edery <email address hidden>
Date: Sun Sep 4 13:18:53 2016 +0300

    Increase the instance_info column size to LONGTEXT on MySQL/MariaDB

    In MySQL, the maximum size of TEXT is 64KiB. The instance_info column
    is defined as TEXT and can potentially hold big amount of data
    (partly affected by cloud-init scripts for the node).

    This patch-set resizes nodes.instance_info to LONGTEXT which allows up
    to 4GiB of data to be stored in the column. This change is relevant only
    to MySQL and does not affect PostgreSQL or any other DB flavor. The fix
    also addresses the upgrade process of the column.

    Change-Id: Ifb9a49d4258a559cf2175d902e9424a3f98065c5
    Closes-Bug: #1596421

Changed in ironic:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/546551
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=b777d3a517d4b1576a24ec7453eb5351a5893b71
Submitter: Zuul
Branch: stable/queens

commit b777d3a517d4b1576a24ec7453eb5351a5893b71
Author: David Edery <email address hidden>
Date: Sun Sep 4 13:18:53 2016 +0300

    Increase the instance_info column size to LONGTEXT on MySQL/MariaDB

    In MySQL, the maximum size of TEXT is 64KiB. The instance_info column
    is defined as TEXT and can potentially hold big amount of data
    (partly affected by cloud-init scripts for the node).

    This patch-set resizes nodes.instance_info to LONGTEXT which allows up
    to 4GiB of data to be stored in the column. This change is relevant only
    to MySQL and does not affect PostgreSQL or any other DB flavor. The fix
    also addresses the upgrade process of the column.

    Change-Id: Ifb9a49d4258a559cf2175d902e9424a3f98065c5
    Closes-Bug: #1596421
    (cherry picked from commit 8fdf752ae9688c581fe8c977e8a5ffb9432583f3)

tags: added: in-stable-queens

This issue was fixed in the openstack/ironic 10.1.1 release.

This issue was fixed in the openstack/ironic 11.0.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers