python3-heat-dashboard launch stack fails

[Heather Lemon]

Heat-dashboard fails to upload files.

[Impact]

Users cannot upload heat template files via the Openstack dashboard (see failed ui screenshot)

"A template is used to automate the deployment of infrastructure, services, and applications."

[Test Case]

1) Setup a basic openstack with heat and horizon
- generate a heat-template.yaml deployment (attached template.yaml)
2) After the regular cloud setup, try to upload the file via horizon dashboard -> Project -> Stacks -> Launch stack (+)
- choose the template.yaml
3) Check openstack-dashboard/0 $/var/log/apache2/error.log and it should have the error
```
[Mon Feb 01 19:46:05.787127 2021] [wsgi:error] [pid 13984:tid 140255296943872] [remote 10.5.3.109:59180] File "/usr/lib/python3/dist-packages/heat_dashboard/content/stacks/forms.py", line 149, in clean
[Mon Feb 01 19:46:05.787129 2021] [wsgi:error] [pid 13984:tid 140255296943872] [remote 10.5.3.109:59180] self.clean_uploaded_files('template', _('template'), cleaned, files)
[Mon Feb 01 19:46:05.787132 2021] [wsgi:error] [pid 13984:tid 140255296943872] [remote 10.5.3.109:59180] File "/usr/lib/python3/dist-packages/heat_dashboard/content/stacks/forms.py", line 207, in clean_uploaded_files
[Mon Feb 01 19:46:05.787135 2021] [wsgi:error] [pid 13984:tid 140255296943872] [remote 10.5.3.109:59180] if tpl.startswith('{'):
[Mon Feb 01 19:46:05.787139 2021] [wsgi:error] [pid 13984:tid 140255296943872] [remote 10.5.3.109:59180] TypeError: startswith first arg must be bytes or a tuple of bytes, not str
[Mon Feb 01 19:46:05.787145 2021] [wsgi:error] [pid 13984:tid 140255296943872] [remote 10.5.3.109:59180]
```
---------
# use sshuttle for reverse proxy to your openstack instance
sshuttle -r ubuntu@X.X.X.X X.X.0.0/24

testing with fix
# add my ppa or use debdiff inside of openstack-dashboard/0

# ssh to openstack dashboard container
# use new heat-dashboard deb
# restart apache2
# try and upload template.yaml again
juju ssh openstack-dashboard/0

#remove existing python3-heat-dashboard version
sudo apt-get remove python3-heat-dashboard

sudo add-apt-repository ppa:hypothetical-lemon/heat-dashboard
apt-get update

# make sure the version is newest (heat-dashboard_1.5.0-0ubuntu3~cloud1)
sudo apt-cache policy python3-heat-dashboard

# restart apache2
systemctl restart apache2

# go to error log
cd /var/log/apache2
tail -f error.log

# in a new terminal

on the openstack cli
-----
openstack flavor create test
openstack floating ip create ext_net
openstack server list
#find the associated container for reverse-proxy
| 7b790c15-ccd1-469d-85f6-c20321128c3c | juju-d21df1-sru-2003343-9 | ACTIVE | test_admin_net=x.x.x.131 | auto-sync/ubuntu-bionic-daily-amd64-server-20200826-disk1.img | m1.small
#copy id or name

openstack image list
- create an image if one is not listed
openstack keypair create test-key
openstack keypair list

openstack server add floating ip <name of openstack-dashboard-container> <ip add of new floating ip> #copied cmd from <openstack floating ip list>
# in a separate terminal
sshuttle -r ubuntu@x.x.x.x x.x.x.0/24

# open a private session with brower
copy and paste ip from your floating ip list command into the browser url

#Note: an image must be available for use before attempting a file upload
I did this through the UI, but cli works too

try to upload the file via horizon dashboard -> Project -> Stacks -> Launch stack (+)
- choose the template.yaml
- fill out admin password
- Launch

(the instance will be created)

you can tail the error.logs for apache2 inside of openstack-dashboard/o
under /var/log/apache2/error.log

I saw no errors when uploading the template file.

[Where problems could occur]

The information for the forms page could fail to load properly.

[Other Info]

Patch landed in Train and now going to Stein.
This is apython3 API change, Bionic/Queens are on python2, no fix needed in those releases.

Openstack: Stein
OS: Ubuntu Bionic

# how to find new Openstack dashboard credentials
credentials (leave a comment if you don't know)
-----
# print out openstack login credentials
juju config keystone admin-user
juju config keystone admin-password

#helpful links
[0] https://docs.openstack.org/heat/latest/install/launch-instance.html
[1] https://docs.openstack.org/heat-dashboard/latest/install/index.html
[2] https://docs.openstack.org/newton/user-guide/cli-create-and-manage-stacks.html

VERIFICATION DONE
I was able to successfully launch a new stack from the template gui.
# enabled proposed
Enabled the bionic-proposed repository in the openstack-dashboard container.
#removed old version
sudo apt-get remove python3-heat-dashboard
# dpkg -l python3-heat-dashboard | cat
Ensured that the version number is latest (python3-heat-dashboard 1.5.0-0ubuntu3~cloud1)
# restart apache
sudo systemctl restart apache2
# created an image from iso via openstack gui
# created a new network and subnet via openstack gui
# launched Orchestration -> Stacks -> +Launch Stack
# seleted template.yaml modified with corrected subnet
# was able to see stack created and no errors in log file
# /var/log/apache2/error.log

[Heather Lemon (hypothetical-lemon)]

screenshot of error thrown in openstack-dashboard

[Heather Lemon (hypothetical-lemon)]

upload an example template.yaml

[Heather Lemon (hypothetical-lemon)]

screenshot of /var/log/apache2/error.log

[Heather Lemon (hypothetical-lemon)]

sample template.yaml for test

[Heather Lemon (hypothetical-lemon)]

adding .mkv video showing fixed issue

[Heather Lemon (hypothetical-lemon)]

this template version 2. the one that works

[Heather Lemon (hypothetical-lemon)]

attached debdiff ready for sru cloud review

[Dr. Jens Harbott (j-harbott)]

This has been fixed upstream in release 1.5.1, why not simply update to the new upstream version? The patch you attached is https://review.opendev.org/c/openstack/heat-dashboard/+/694952 , you should at a minimum show its origin.

[Corey Bryant (corey.bryant)]

Hi Heather,

Thanks for your help on this. I'm making a few minor changes to the patch to drop the DEP-3 headers and use an unchanged 'git cherry-pick' from the upstream stable/stein branch. Some Ubuntu developers prefer DEP-3 headers only. I'm personally ok with an unchanged cherry-pick from upstream or DEP-3 headers.

@Jens, We are currently only able to do point releases for the packages listed at the following unfortunately: https://wiki.ubuntu.com/OpenStack/StableReleaseUpdates

Thanks,
Corey

[Corey Bryant (corey.bryant)]

Hello Heather, or anyone else affected,

Accepted heat-dashboard into stein-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:stein-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-stein-needed to verification-stein-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-stein-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Heather Lemon (hypothetical-lemon)]

Hi Corey,

That's good to know about the DEP-3 headers.

Thank You!
Heather Lemon

[Heather Lemon (hypothetical-lemon)]

Re-tagged as verification-done.

Thanks,
Heather Lemon

[Corey Bryant (corey.bryant)]

The verification of the Stable Release Update for heat-dashboard has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Corey Bryant (corey.bryant)]

This bug was fixed in the package heat-dashboard - 1.5.0-0ubuntu3~cloud1
---------------

 heat-dashboard (1.5.0-0ubuntu3~cloud1) bionic-stein; urgency=medium
 .
   [ Corey Bryant ]
   * d/gbp.conf: Create stable/stein branch.
 .
   [ Chris MacNaughton ]
   * d/control: Update VCS paths for move to lp:~ubuntu-openstack-dev.
 .
   [ Heather Lemon ]
   * d/p/2003343-enforces-unicode-decoding-of-data.patch (LP: #1914100)
     - Enforces the decoding of the returned value as a UTF-8