cloud-init

disco: python37 unittest/tox support

Bug #1813361 reported by Chad Smith on 2019-01-25

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	cloud-init	Fix Released	Undecided	Chad Smith

Bug Description

cloud-init's xenial toxenv falls over on tip of master 7a4696596bbcccfedf5c6b6e25ad684ef30d9cea
in Ubuntu Disco python37 environments. Some of the tox dependencies like httpretty are exhibiting issues with python3.7

Make "all the tox things" work on Disco

type of errors seen running tox -r -e xenial on Disco

======================================================================
ERROR: tests.unittests.test_datasource.test_openstack.TestOpenStackDataSource.test_wb__crawl_metadata_does_not_persist
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/cloud-init/.tox/xenial/lib/python3.7/site-packages/httpretty/core.py", line 1055, in wrapper
    return test(*args, **kw)
  File "/home/ubuntu/cloud-init/tests/unittests/test_datasource/test_openstack.py", line 395, in test_wb__crawl_metadata_does_not_persist
    _register_uris(self.VERSION, EC2_FILES, EC2_META, OS_FILES)
  File "/home/ubuntu/cloud-init/tests/unittests/test_datasource/test_openstack.py", line 126, in _register_uris
    body=get_request_callback)
  File "/home/ubuntu/cloud-init/.tox/xenial/lib/python3.7/site-packages/httpretty/core.py", line 938, in register_uri
    match_querystring)
  File "/home/ubuntu/cloud-init/.tox/xenial/lib/python3.7/site-packages/httpretty/core.py", line 760, in __init__
    self.info = URIInfo.from_uri(uri, entries)
  File "/home/ubuntu/cloud-init/.tox/xenial/lib/python3.7/site-packages/httpretty/core.py", line 730, in from_uri
    result = urlsplit(uri)
  File "/usr/lib/python3.7/urllib/parse.py", line 400, in urlsplit
    url, scheme, _coerce_result = _coerce_args(url, scheme)
  File "/usr/lib/python3.7/urllib/parse.py", line 123, in _coerce_args
    return _decode_args(args) + (_encode_result,)
  File "/usr/lib/python3.7/urllib/parse.py", line 107, in _decode_args
    return tuple(x.decode(encoding, errors) if x else '' for x in args)
  File "/usr/lib/python3.7/urllib/parse.py", line 107, in <genexpr>
    return tuple(x.decode(encoding, errors) if x else '' for x in args)
AttributeError: 're.Pattern' object has no attribute 'decode'

----------------------------------------------------------------------

Also of note: python27 interpreter isn't available on Disco, so we need to allow tox to skip py27 env if not present.

See original description

Related branches

~chad.smith/cloud-init:ubuntu/bionic

Merged into cloud-init:ubuntu/bionic at revision 4a1aaa18595fa663e1a38dd3ac8f73231ec69a7f

Server Team CI bot: Approve (continuous-integration) on 2019-01-26

cloud-init Commiters: Pending requested 2019-01-26

Diff: 7789 lines (+3838/-768)

98 files modified

ChangeLog (+54/-0)
HACKING.rst (+2/-2)
bash_completion/cloud-init (+4/-1)
cloudinit/cmd/cloud_id.py (+90/-0)
cloudinit/cmd/devel/logs.py (+23/-8)
cloudinit/cmd/devel/net_convert.py (+10/-5)
cloudinit/cmd/devel/render.py (+24/-11)
cloudinit/cmd/devel/tests/test_logs.py (+37/-6)
cloudinit/cmd/devel/tests/test_render.py (+44/-1)
cloudinit/cmd/main.py (+4/-16)
cloudinit/cmd/query.py (+24/-12)
cloudinit/cmd/tests/test_cloud_id.py (+127/-0)
cloudinit/cmd/tests/test_query.py (+71/-5)
cloudinit/config/cc_disk_setup.py (+1/-1)
cloudinit/config/cc_lxd.py (+1/-1)
cloudinit/config/cc_resizefs.py (+7/-0)
cloudinit/config/cc_set_passwords.py (+1/-1)
cloudinit/config/cc_write_files.py (+6/-1)
cloudinit/config/tests/test_set_passwords.py (+40/-0)
cloudinit/dhclient_hook.py (+72/-38)
cloudinit/handlers/jinja_template.py (+9/-1)
cloudinit/net/__init__.py (+38/-4)
cloudinit/net/dhcp.py (+76/-25)
cloudinit/net/eni.py (+15/-14)
cloudinit/net/netplan.py (+3/-3)
cloudinit/net/sysconfig.py (+61/-5)
cloudinit/net/tests/test_dhcp.py (+47/-4)
cloudinit/net/tests/test_init.py (+51/-1)
cloudinit/sources/DataSourceAliYun.py (+5/-15)
cloudinit/sources/DataSourceAltCloud.py (+22/-11)
cloudinit/sources/DataSourceAzure.py (+82/-31)
cloudinit/sources/DataSourceBigstep.py (+4/-0)
cloudinit/sources/DataSourceCloudSigma.py (+5/-1)
cloudinit/sources/DataSourceConfigDrive.py (+12/-0)
cloudinit/sources/DataSourceEc2.py (+59/-56)
cloudinit/sources/DataSourceIBMCloud.py (+4/-0)
cloudinit/sources/DataSourceMAAS.py (+4/-0)
cloudinit/sources/DataSourceNoCloud.py (+52/-1)
cloudinit/sources/DataSourceNone.py (+4/-0)
cloudinit/sources/DataSourceOVF.py (+36/-26)
cloudinit/sources/DataSourceOpenNebula.py (+9/-1)
cloudinit/sources/DataSourceOracle.py (+4/-0)
cloudinit/sources/DataSourceScaleway.py (+10/-1)
cloudinit/sources/DataSourceSmartOS.py (+3/-0)
cloudinit/sources/__init__.py (+104/-21)
cloudinit/sources/helpers/netlink.py (+250/-0)
cloudinit/sources/helpers/tests/test_netlink.py (+373/-0)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+2/-3)
cloudinit/sources/tests/test_init.py (+83/-3)
cloudinit/sources/tests/test_oracle.py (+8/-0)
cloudinit/temp_utils.py (+2/-2)
cloudinit/tests/test_dhclient_hook.py (+105/-0)
cloudinit/tests/test_temp_utils.py (+17/-1)
cloudinit/tests/test_url_helper.py (+24/-1)
cloudinit/tests/test_util.py (+82/-17)
cloudinit/url_helper.py (+25/-6)
cloudinit/util.py (+25/-3)
cloudinit/version.py (+1/-1)
config/cloud.cfg.tmpl (+11/-1)
debian/changelog (+75/-0)
doc/rtd/topics/datasources.rst (+60/-1)
doc/rtd/topics/datasources/azure.rst (+65/-38)
doc/rtd/topics/instancedata.rst (+137/-46)
doc/rtd/topics/network-config-format-v1.rst (+1/-1)
packages/redhat/cloud-init.spec.in (+1/-0)
packages/suse/cloud-init.spec.in (+1/-0)
setup.py (+2/-1)
systemd/cloud-init.service.tmpl (+1/-2)
templates/sources.list.ubuntu.tmpl (+17/-17)
tests/cloud_tests/releases.yaml (+16/-0)
tests/cloud_tests/testcases/base.py (+15/-3)
tests/cloud_tests/testcases/modules/apt_configure_primary.py (+9/-5)
tests/cloud_tests/testcases/modules/apt_configure_primary.yaml (+0/-7)
tests/unittests/test_builtin_handlers.py (+25/-0)
tests/unittests/test_cli.py (+8/-8)
tests/unittests/test_datasource/test_aliyun.py (+4/-0)
tests/unittests/test_datasource/test_altcloud.py (+67/-51)
tests/unittests/test_datasource/test_azure.py (+262/-79)
tests/unittests/test_datasource/test_cloudsigma.py (+6/-0)
tests/unittests/test_datasource/test_configdrive.py (+3/-0)
tests/unittests/test_datasource/test_ec2.py (+37/-23)
tests/unittests/test_datasource/test_ibmcloud.py (+39/-1)
tests/unittests/test_datasource/test_nocloud.py (+98/-41)
tests/unittests/test_datasource/test_opennebula.py (+4/-0)
tests/unittests/test_datasource/test_ovf.py (+119/-39)
tests/unittests/test_datasource/test_scaleway.py (+72/-4)
tests/unittests/test_datasource/test_smartos.py (+7/-0)
tests/unittests/test_ds_identify.py (+16/-1)
tests/unittests/test_handler/test_handler_lxd.py (+1/-1)
tests/unittests/test_handler/test_handler_resizefs.py (+42/-10)
tests/unittests/test_handler/test_handler_write_files.py (+12/-0)
tests/unittests/test_net.py (+137/-6)
tests/unittests/test_util.py (+6/-0)
tests/unittests/test_vmware_config_file.py (+52/-6)
tools/ds-identify (+32/-6)
tools/run-container (+1/-0)
tox.ini (+2/-2)
udev/66-azure-ephemeral.rules (+17/-1)

~chad.smith/cloud-init:ubuntu/cosmic

Merged into cloud-init:ubuntu/cosmic at revision 6db1a138541f807d103bcee3080d830c71dfab2e

Server Team CI bot: Approve (continuous-integration) on 2019-01-26

cloud-init Commiters: Pending requested 2019-01-26

Diff: 5357 lines (+2757/-473)

69 files modified

ChangeLog (+54/-0)
HACKING.rst (+2/-2)
bash_completion/cloud-init (+4/-1)
cloudinit/cmd/devel/logs.py (+23/-8)
cloudinit/cmd/devel/net_convert.py (+10/-5)
cloudinit/cmd/devel/render.py (+24/-11)
cloudinit/cmd/devel/tests/test_logs.py (+37/-6)
cloudinit/cmd/devel/tests/test_render.py (+44/-1)
cloudinit/cmd/main.py (+4/-16)
cloudinit/cmd/query.py (+24/-12)
cloudinit/cmd/tests/test_query.py (+71/-5)
cloudinit/config/cc_disk_setup.py (+1/-1)
cloudinit/config/cc_lxd.py (+1/-1)
cloudinit/config/cc_resizefs.py (+7/-0)
cloudinit/config/cc_set_passwords.py (+1/-1)
cloudinit/config/cc_write_files.py (+6/-1)
cloudinit/config/tests/test_set_passwords.py (+40/-0)
cloudinit/dhclient_hook.py (+72/-38)
cloudinit/handlers/jinja_template.py (+9/-1)
cloudinit/net/__init__.py (+34/-2)
cloudinit/net/dhcp.py (+76/-25)
cloudinit/net/eni.py (+15/-14)
cloudinit/net/netplan.py (+3/-3)
cloudinit/net/sysconfig.py (+61/-5)
cloudinit/net/tests/test_dhcp.py (+47/-4)
cloudinit/net/tests/test_init.py (+51/-1)
cloudinit/sources/DataSourceAzure.py (+74/-31)
cloudinit/sources/DataSourceNoCloud.py (+31/-1)
cloudinit/sources/DataSourceOVF.py (+30/-26)
cloudinit/sources/DataSourceOpenNebula.py (+1/-1)
cloudinit/sources/DataSourceScaleway.py (+10/-1)
cloudinit/sources/helpers/netlink.py (+250/-0)
cloudinit/sources/helpers/tests/test_netlink.py (+373/-0)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+2/-3)
cloudinit/temp_utils.py (+2/-2)
cloudinit/tests/test_dhclient_hook.py (+105/-0)
cloudinit/tests/test_temp_utils.py (+17/-1)
cloudinit/tests/test_url_helper.py (+24/-1)
cloudinit/tests/test_util.py (+66/-17)
cloudinit/url_helper.py (+25/-6)
cloudinit/util.py (+20/-3)
cloudinit/version.py (+1/-1)
config/cloud.cfg.tmpl (+11/-1)
debian/changelog (+70/-0)
doc/rtd/topics/datasources.rst (+60/-1)
doc/rtd/topics/datasources/azure.rst (+65/-38)
doc/rtd/topics/network-config-format-v1.rst (+1/-1)
packages/redhat/cloud-init.spec.in (+1/-0)
packages/suse/cloud-init.spec.in (+1/-0)
systemd/cloud-init.service.tmpl (+1/-2)
tests/cloud_tests/releases.yaml (+16/-0)
tests/unittests/test_builtin_handlers.py (+25/-0)
tests/unittests/test_cli.py (+8/-8)
tests/unittests/test_datasource/test_azure.py (+201/-34)
tests/unittests/test_datasource/test_ec2.py (+24/-16)
tests/unittests/test_datasource/test_nocloud.py (+66/-34)
tests/unittests/test_datasource/test_ovf.py (+79/-43)
tests/unittests/test_datasource/test_scaleway.py (+72/-4)
tests/unittests/test_ds_identify.py (+16/-1)
tests/unittests/test_handler/test_handler_lxd.py (+1/-1)
tests/unittests/test_handler/test_handler_resizefs.py (+42/-10)
tests/unittests/test_handler/test_handler_write_files.py (+12/-0)
tests/unittests/test_net.py (+123/-6)
tests/unittests/test_util.py (+6/-0)
tests/unittests/test_vmware_config_file.py (+52/-6)
tools/ds-identify (+32/-6)
tools/run-container (+1/-0)
tox.ini (+2/-2)
udev/66-azure-ephemeral.rules (+17/-1)

Chad Smith (chad.smith) on 2019-01-25

Changed in cloud-init:
status:	New → In Progress
assignee:	nobody → Chad Smith (chad.smith)

Chad Smith (chad.smith) on 2019-01-25

description:

updated

Revision history for this message

Server Team CI bot (server-team-bot) wrote on 2019-01-25:

This bug is fixed with commit c7248059 to cloud-init on branch master.
To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=c7248059

Changed in cloud-init:
status:	In Progress → Fix Committed

Revision history for this message

Chad Smith (chad.smith) wrote on 2019-05-10: Fixed in cloud-init version 19.1.

This bug is believed to be fixed in cloud-init in version 19.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.