Activity log for bug #1596421

Date Who What changed Old value New value Message
2016-06-27 06:24:26 David Edery bug added bug
2016-06-27 06:24:26 David Edery attachment added complete_exception.log https://bugs.launchpad.net/bugs/1596421/+attachment/4690910/+files/complete_exception.log
2016-06-27 06:24:50 David Edery ironic: assignee David Edery (david-edery)
2016-06-28 12:02:25 OpenStack Infra ironic: status New In Progress
2016-07-05 18:33:17 aeva black ironic: importance Undecided Medium
2016-07-05 18:36:22 aeva black summary nodes are not powered-on due to short nodes.instance_info node deployment fails when user passes a long cloud-init script
2016-09-13 16:55:56 John L. Villalovos summary node deployment fails when user passes a long cloud-init script RFE: Increase size of data base entry for instance_info to allow configdrives larger than 64KB
2016-09-13 16:56:07 John L. Villalovos tags rfe
2016-09-13 16:56:53 John L. Villalovos bug added subscriber John L. Villalovos
2016-09-13 17:03:13 John L. Villalovos description By using cloud-init scripts, it's easy to pass the 65K characters limitation of nodes.instance_info (db flavor is mysql, column type is text). The number of lines in the cloud-init scripts is not anywhere near 65K (or 5K for that matter). Exception: 2016-06-20 10:14:12.330 1884 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (pymysql.err.DataError) (1406, u"Data too long for column 'instance_info' at row 1") [SQL: u'UPDATE nodes SET updated_at=%s, provision_state=%s, provision_updated_at=%s, instance_info=%s WHERE nodes.id = %s'] [parameters: (datetime.datetime(2016, 6, 20, 7, 14, 12, 327159), u'deploy failed', datetime.datetime(2016, 6, 20, 7, 14, 12, 326708), '{"root_gb": "40", "display_name": "overcloud-compute-0", "image_source": "8bd9c674-15cc-4886-9b47-be4901a16227", "local_gb": "1862", "capabilities": "{\\"boot_option\\": \\"local\\"}", "memory_mb": "4096", "vcpus": "1", "deploy_key": "G5EHKLCVWTC1GRH1SHIN6L8CGH94B52C", "configdrive": "H4sICMOXZ1cC/3RtcFl1U1NaOQDsvduSG8mSGAie0c6MoDHt2uzZtTXZSIoDNg+AQ9zryupGS9XFIlndxSp2XZpNVpVBCWQClV0AEp0JVLHIoWz1Is2a7QdIetAH6CPW9DV6k+2DnmW7fonIzMgLAF76wp50dheAzAiPCA+PcPe4uOdyGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEGGWSQQQYZZJDBnZ2HjUbzTm5/7+D0e5EOPWfctwfVVtp7hW/7DmHdXlgu/If/5/7yL3P/iR/9p98Hr/93/FPI/Q3/+pvcX+LHX+amf/77/D/+R3/1O5Vf/MJwOLHGx1OjdyV2nNFkNrVEs1Vr1FaqzZo13HC8qfhtw+Pdg73jw72n2493BXwRD9bXG/Unj47Fo739 ... (the complete exception is attached to the bug report) AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/P35P3laHXcACAoA", "swap_mb": "0"}', 3)] In the above case, the "configdrive" data contains ~80K characters. Due to the exception, instance_info is left empty and the nodes are not powered-on/deployed. Suggested solution: Change nodes.instance_info from (mysql's) text to medium/longtext (http://stackoverflow.com/questions/13932750/tinytext-text-mediumtext-and-longtext-maximum-storage-sizes). Handling the above exception, a manual change to longtext solved the issue. In the Ironic database the 'instance_info' field as stored as a TEXT field. With MySQL this has a limit of 64KiB. The configdrive is stored in the instance_info field and so can not be larger than 64KiB. We have reports from multiple users who would like to be able to use configdrives larger than 64KIB. Proposal is to switch to using the MEDIUMTEXT field type for instance_info for MySQL databases, which would allow up to 16MiB for the instance_info. Also propose to have a configuration setting for the maximum size for the configdrive to by default be set to 64KiB. Any configdrive that exceeds the config setting would be rejected. This would maintain the current size limitation and operators would not be surprised by their database growing unexpectedly. Operators could then change the config setting to increase the maximum size of the configdrive.
2016-09-13 17:04:19 John L. Villalovos summary RFE: Increase size of data base entry for instance_info to allow configdrives larger than 64KB RFE: Increase size of data base entry for instance_info to allow configdrives larger than 64KiB
2016-09-13 17:11:18 John L. Villalovos description In the Ironic database the 'instance_info' field as stored as a TEXT field. With MySQL this has a limit of 64KiB. The configdrive is stored in the instance_info field and so can not be larger than 64KiB. We have reports from multiple users who would like to be able to use configdrives larger than 64KIB. Proposal is to switch to using the MEDIUMTEXT field type for instance_info for MySQL databases, which would allow up to 16MiB for the instance_info. Also propose to have a configuration setting for the maximum size for the configdrive to by default be set to 64KiB. Any configdrive that exceeds the config setting would be rejected. This would maintain the current size limitation and operators would not be surprised by their database growing unexpectedly. Operators could then change the config setting to increase the maximum size of the configdrive. In the Ironic database the 'instance_info' field as stored as a TEXT field. With MySQL this has a limit of 64KiB. The configdrive is stored in the instance_info field and so can not be larger than 64KiB. We have reports from multiple users who would like to be able to use configdrives larger than 64KIB. Proposal is to switch to using the MEDIUMTEXT field type for instance_info for MySQL databases, which would allow up to 16MiB for the instance_info. Also propose to have a configuration setting for the maximum size for the configdrive to by default be set to 64KiB. Database entry would still be MEDIUMTTEXT. Any configdrive that exceeds the config setting would be rejected. This would maintain the current size limitation and operators would not be surprised by their database growing unexpectedly. Operators could then change the config setting to increase the maximum size of the configdrive.
2016-09-13 18:05:47 OpenStack Infra ironic: assignee David Edery (david-edery) John L. Villalovos (happycamp)
2016-09-14 14:06:30 Jim Rollenhagen ironic: importance Medium Wishlist
2016-11-21 17:37:55 Jim Rollenhagen ironic: status In Progress Invalid
2018-02-13 12:26:15 Dmitry Tantsur summary RFE: Increase size of data base entry for instance_info to allow configdrives larger than 64KiB [RFE] Stop storing configdrive in the database
2018-02-13 12:26:19 Dmitry Tantsur ironic: status Invalid Confirmed
2018-02-13 12:26:23 Dmitry Tantsur ironic: assignee John L. Villalovos (happycamp)
2018-02-13 12:27:27 Dmitry Tantsur description In the Ironic database the 'instance_info' field as stored as a TEXT field. With MySQL this has a limit of 64KiB. The configdrive is stored in the instance_info field and so can not be larger than 64KiB. We have reports from multiple users who would like to be able to use configdrives larger than 64KIB. Proposal is to switch to using the MEDIUMTEXT field type for instance_info for MySQL databases, which would allow up to 16MiB for the instance_info. Also propose to have a configuration setting for the maximum size for the configdrive to by default be set to 64KiB. Database entry would still be MEDIUMTTEXT. Any configdrive that exceeds the config setting would be rejected. This would maintain the current size limitation and operators would not be surprised by their database growing unexpectedly. Operators could then change the config setting to increase the maximum size of the configdrive. In the Ironic database the 'instance_info' field as stored as a TEXT field. With MySQL this has a limit of 64KiB. The configdrive is stored in the instance_info field and so can not be larger than 64KiB. We have reports from multiple users who would like to be able to use configdrives larger than 64KIB. Proposal is to switch to using the MEDIUMTEXT field type for instance_info for MySQL databases, which would allow up to 16MiB for the instance_info. Also propose to have a configuration setting for the maximum size for the configdrive to by default be set to 64KiB. Database entry would still be MEDIUMTTEXT. Any configdrive that exceeds the config setting would be rejected. This would maintain the current size limitation and operators would not be surprised by their database growing unexpectedly. Operators could then change the config setting to increase the maximum size of the configdrive. Update from dtantsur: since bug #1575935 is fixed, we no longer need to store the configdrive in the database at all.
2018-02-13 12:30:15 OpenStack Infra ironic: status Confirmed In Progress
2018-02-13 12:30:15 OpenStack Infra ironic: assignee Dmitry Tantsur (divius)
2018-02-20 17:25:06 OpenStack Infra ironic: assignee Dmitry Tantsur (divius) Julia Kreger (juliaashleykreger)
2018-02-20 22:29:40 OpenStack Infra ironic: status In Progress Fix Released
2018-02-21 15:46:04 OpenStack Infra tags rfe in-stable-queens rfe