Collect all logs needed to debug curtin/cloud-init for each deployment
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Expired
|
High
|
Unassigned | ||
cloud-init |
Expired
|
Medium
|
Unassigned | ||
cloud-init (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Xenial |
Confirmed
|
Undecided
|
Unassigned | ||
Zesty |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
Re-opening this bug as confirmed because the previous SRU content released only provided only 'cloud-init collect-logs'. A command line tool which tars all cloud-init install logs and artifacts for triage.
However, those fixes did not provide any configuration options for MAAS to request that those logs are automatically published to MAAS upon error.
Cloud-init should provide cloud-config which allows consumers to specify an endpoint and oauth credentials to which cloud-init will automatically POST all compressed cloud-init log artifacts.
=== Original Description ===
According to https:/
- /var/log/
- /run/cloud-init*
- /var/log/cloud
- /tmp/install.log
We need these to be automatically collected by MAAS so we can automatically collect them as artifacts in the case of failures in OIL. curtin/cloud-init issues can be race conditions that are difficult to reproduce manually, so we need to grab the logs required to debug the first time it happens.
=== Begin SRU Template ===
[Impact]
ubuntu-bug cloud-init now collects cloud-init-related information for a bug-report
[Test Case]
# Launch instance under test
$ for release in xenial zesty;
do
ref=
lxc-
lxc launch $ref $name;
sleep 10;
lxc exec $name ubuntu-bug cloud-init # And follow the prompts to report a bogus bug
done
[Regression Potential]
Worst case scenario is the apport wrapper doesn't work and the developer has to file a bug manually instead.
[Other Info]
Upstream commit at
https:/
=== End SRU Template ===
Related branches
- Server Team CI bot: Approve (continuous-integration)
- Scott Moser: Pending requested
-
Diff: 15737 lines (+9583/-1296)273 files modifiedChangeLog (+422/-0)
Makefile (+3/-3)
cloudinit/analyze/__init__.py (+0/-0)
cloudinit/analyze/__main__.py (+155/-0)
cloudinit/analyze/dump.py (+176/-0)
cloudinit/analyze/show.py (+207/-0)
cloudinit/analyze/tests/test_dump.py (+210/-0)
cloudinit/apport.py (+105/-0)
cloudinit/cmd/devel/__init__.py (+0/-0)
cloudinit/cmd/devel/logs.py (+101/-0)
cloudinit/cmd/devel/parser.py (+26/-0)
cloudinit/cmd/devel/tests/__init__.py (+0/-0)
cloudinit/cmd/devel/tests/test_logs.py (+120/-0)
cloudinit/cmd/main.py (+35/-34)
cloudinit/config/cc_bootcmd.py (+60/-30)
cloudinit/config/cc_chef.py (+33/-11)
cloudinit/config/cc_landscape.py (+2/-2)
cloudinit/config/cc_ntp.py (+57/-49)
cloudinit/config/cc_puppet.py (+18/-15)
cloudinit/config/cc_resizefs.py (+87/-70)
cloudinit/config/cc_resolv_conf.py (+1/-1)
cloudinit/config/cc_runcmd.py (+57/-27)
cloudinit/config/cc_snappy.py (+2/-2)
cloudinit/config/cc_ssh_authkey_fingerprints.py (+2/-2)
cloudinit/config/cc_zypper_add_repo.py (+218/-0)
cloudinit/config/schema.py (+181/-43)
cloudinit/distros/__init__.py (+8/-1)
cloudinit/distros/arch.py (+59/-31)
cloudinit/distros/debian.py (+71/-23)
cloudinit/distros/opensuse.py (+212/-0)
cloudinit/distros/sles.py (+5/-155)
cloudinit/helpers.py (+7/-7)
cloudinit/log.py (+5/-0)
cloudinit/net/__init__.py (+21/-30)
cloudinit/net/dhcp.py (+163/-0)
cloudinit/net/eni.py (+3/-0)
cloudinit/net/netplan.py (+12/-28)
cloudinit/net/network_state.py (+84/-18)
cloudinit/net/sysconfig.py (+5/-1)
cloudinit/net/tests/test_dhcp.py (+260/-0)
cloudinit/net/tests/test_init.py (+2/-2)
cloudinit/netinfo.py (+4/-4)
cloudinit/simpletable.py (+62/-0)
cloudinit/sources/DataSourceAliYun.py (+6/-3)
cloudinit/sources/DataSourceAltCloud.py (+2/-2)
cloudinit/sources/DataSourceAzure.py (+7/-3)
cloudinit/sources/DataSourceCloudStack.py (+37/-14)
cloudinit/sources/DataSourceEc2.py (+164/-22)
cloudinit/sources/DataSourceGCE.py (+126/-72)
cloudinit/sources/DataSourceOVF.py (+169/-51)
cloudinit/sources/__init__.py (+8/-1)
cloudinit/sources/helpers/azure.py (+16/-8)
cloudinit/sources/helpers/vmware/imc/config.py (+21/-3)
cloudinit/sources/helpers/vmware/imc/config_nic.py (+130/-71)
cloudinit/sources/helpers/vmware/imc/config_passwd.py (+67/-0)
cloudinit/sources/helpers/vmware/imc/guestcust_util.py (+7/-5)
cloudinit/stages.py (+20/-13)
cloudinit/temp_utils.py (+101/-0)
cloudinit/tests/__init__.py (+0/-0)
cloudinit/tests/helpers.py (+16/-2)
cloudinit/tests/test_simpletable.py (+100/-0)
cloudinit/tests/test_temp_utils.py (+101/-0)
cloudinit/tests/test_url_helper.py (+40/-0)
cloudinit/url_helper.py (+5/-1)
cloudinit/util.py (+41/-42)
cloudinit/version.py (+1/-1)
config/cloud.cfg.tmpl (+9/-5)
debian/apport-launcher.py (+6/-0)
debian/changelog (+117/-2)
debian/rules (+2/-1)
dev/null (+0/-2)
doc/examples/cloud-config-chef.txt (+4/-0)
doc/rtd/index.rst (+1/-0)
doc/rtd/topics/capabilities.rst (+40/-10)
doc/rtd/topics/datasources.rst (+1/-0)
doc/rtd/topics/datasources/gce.rst (+20/-0)
doc/rtd/topics/debugging.rst (+146/-0)
doc/rtd/topics/format.rst (+1/-0)
doc/rtd/topics/modules.rst (+0/-1)
packages/bddeb (+4/-4)
packages/debian/copyright (+10/-15)
packages/debian/dirs (+0/-1)
packages/debian/rules.in (+2/-1)
packages/pkg-deps.json (+0/-3)
packages/redhat/cloud-init.spec.in (+0/-6)
requirements.txt (+0/-3)
setup.py (+4/-4)
systemd/cloud-final.service.tmpl (+3/-1)
systemd/cloud-init-local.service.tmpl (+6/-0)
systemd/cloud-init.service.tmpl (+10/-0)
sysvinit/suse/cloud-config (+113/-0)
sysvinit/suse/cloud-final (+113/-0)
sysvinit/suse/cloud-init (+114/-0)
sysvinit/suse/cloud-init-local (+113/-0)
templates/hosts.opensuse.tmpl (+26/-0)
templates/hosts.suse.tmpl (+0/-3)
templates/sources.list.debian.tmpl (+8/-8)
templates/timesyncd.conf.tmpl (+8/-0)
tests/cloud_tests/__init__.py (+1/-1)
tests/cloud_tests/__main__.py (+4/-1)
tests/cloud_tests/args.py (+2/-2)
tests/cloud_tests/bddeb.py (+10/-9)
tests/cloud_tests/collect.py (+3/-0)
tests/cloud_tests/config.py (+1/-0)
tests/cloud_tests/images/nocloudkvm.py (+88/-0)
tests/cloud_tests/instances/base.py (+7/-5)
tests/cloud_tests/instances/lxd.py (+9/-1)
tests/cloud_tests/instances/nocloudkvm.py (+217/-0)
tests/cloud_tests/platforms.yaml (+4/-0)
tests/cloud_tests/platforms/__init__.py (+2/-0)
tests/cloud_tests/platforms/nocloudkvm.py (+90/-0)
tests/cloud_tests/releases.yaml (+18/-1)
tests/cloud_tests/setup_image.py (+24/-8)
tests/cloud_tests/snapshots/nocloudkvm.py (+74/-0)
tests/cloud_tests/testcases/bugs/README.md (+0/-0)
tests/cloud_tests/testcases/bugs/lp1511485.yaml (+0/-0)
tests/cloud_tests/testcases/bugs/lp1611074.yaml (+0/-0)
tests/cloud_tests/testcases/bugs/lp1628337.yaml (+0/-0)
tests/cloud_tests/testcases/examples/README.md (+0/-0)
tests/cloud_tests/testcases/examples/TODO.md (+0/-0)
tests/cloud_tests/testcases/examples/add_apt_repositories.yaml (+0/-0)
tests/cloud_tests/testcases/examples/alter_completion_message.yaml (+0/-0)
tests/cloud_tests/testcases/examples/configure_instance_trusted_ca_certificates.yaml (+0/-0)
tests/cloud_tests/testcases/examples/configure_instances_ssh_keys.yaml (+0/-0)
tests/cloud_tests/testcases/examples/including_user_groups.yaml (+0/-0)
tests/cloud_tests/testcases/examples/install_arbitrary_packages.yaml (+0/-0)
tests/cloud_tests/testcases/examples/install_run_chef_recipes.yaml (+0/-0)
tests/cloud_tests/testcases/examples/run_apt_upgrade.yaml (+0/-0)
tests/cloud_tests/testcases/examples/run_commands.yaml (+0/-0)
tests/cloud_tests/testcases/examples/run_commands_first_boot.yaml (+0/-0)
tests/cloud_tests/testcases/examples/setup_run_puppet.yaml (+0/-0)
tests/cloud_tests/testcases/examples/writing_out_arbitrary_files.yaml (+0/-0)
tests/cloud_tests/testcases/main/README.md (+0/-0)
tests/cloud_tests/testcases/main/command_output_simple.yaml (+0/-0)
tests/cloud_tests/testcases/modules/README.md (+0/-0)
tests/cloud_tests/testcases/modules/TODO.md (+0/-2)
tests/cloud_tests/testcases/modules/apt_configure_conf.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_disable_suites.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_primary.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_proxy.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_security.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_sources_key.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_sources_list.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_configure_sources_ppa.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_pipelining_disable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/apt_pipelining_os.yaml (+0/-0)
tests/cloud_tests/testcases/modules/bootcmd.yaml (+0/-0)
tests/cloud_tests/testcases/modules/byobu.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ca_certs.yaml (+0/-0)
tests/cloud_tests/testcases/modules/debug_disable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/debug_enable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/final_message.yaml (+0/-0)
tests/cloud_tests/testcases/modules/keys_to_console.yaml (+0/-0)
tests/cloud_tests/testcases/modules/landscape.yaml (+0/-0)
tests/cloud_tests/testcases/modules/locale.yaml (+0/-0)
tests/cloud_tests/testcases/modules/lxd_bridge.yaml (+0/-0)
tests/cloud_tests/testcases/modules/lxd_dir.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ntp.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ntp_pools.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ntp_servers.yaml (+0/-0)
tests/cloud_tests/testcases/modules/package_update_upgrade_install.yaml (+0/-0)
tests/cloud_tests/testcases/modules/runcmd.yaml (+0/-0)
tests/cloud_tests/testcases/modules/salt_minion.yaml (+0/-0)
tests/cloud_tests/testcases/modules/seed_random_command.yaml (+0/-0)
tests/cloud_tests/testcases/modules/seed_random_data.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_hostname.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_hostname_fqdn.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_password.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_password_expire.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_password_list.yaml (+0/-0)
tests/cloud_tests/testcases/modules/set_password_list_string.yaml (+0/-0)
tests/cloud_tests/testcases/modules/snappy.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_auth_key_fingerprints_disable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_auth_key_fingerprints_enable.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_import_id.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_keys_generate.yaml (+0/-0)
tests/cloud_tests/testcases/modules/ssh_keys_provided.yaml (+0/-0)
tests/cloud_tests/testcases/modules/timezone.yaml (+0/-0)
tests/cloud_tests/testcases/modules/user_groups.yaml (+0/-0)
tests/cloud_tests/testcases/modules/write_files.yaml (+0/-0)
tests/cloud_tests/util.py (+43/-0)
tests/unittests/test__init__.py (+1/-1)
tests/unittests/test_atomic_helper.py (+1/-1)
tests/unittests/test_builtin_handlers.py (+1/-1)
tests/unittests/test_cli.py (+146/-4)
tests/unittests/test_cs_util.py (+1/-1)
tests/unittests/test_data.py (+1/-1)
tests/unittests/test_datasource/test_aliyun.py (+7/-6)
tests/unittests/test_datasource/test_altcloud.py (+3/-3)
tests/unittests/test_datasource/test_azure.py (+4/-2)
tests/unittests/test_datasource/test_azure_helper.py (+97/-50)
tests/unittests/test_datasource/test_cloudsigma.py (+1/-1)
tests/unittests/test_datasource/test_cloudstack.py (+83/-7)
tests/unittests/test_datasource/test_common.py (+2/-1)
tests/unittests/test_datasource/test_configdrive.py (+1/-1)
tests/unittests/test_datasource/test_digitalocean.py (+1/-1)
tests/unittests/test_datasource/test_ec2.py (+255/-35)
tests/unittests/test_datasource/test_gce.py (+3/-2)
tests/unittests/test_datasource/test_maas.py (+1/-1)
tests/unittests/test_datasource/test_nocloud.py (+1/-1)
tests/unittests/test_datasource/test_opennebula.py (+1/-1)
tests/unittests/test_datasource/test_openstack.py (+4/-1)
tests/unittests/test_datasource/test_ovf.py (+165/-1)
tests/unittests/test_datasource/test_scaleway.py (+1/-1)
tests/unittests/test_datasource/test_smartos.py (+1/-1)
tests/unittests/test_distros/__init__.py (+21/-0)
tests/unittests/test_distros/test_arch.py (+45/-0)
tests/unittests/test_distros/test_create_users.py (+1/-1)
tests/unittests/test_distros/test_debian.py (+42/-24)
tests/unittests/test_distros/test_generic.py (+17/-1)
tests/unittests/test_distros/test_netconfig.py (+3/-3)
tests/unittests/test_distros/test_opensuse.py (+12/-0)
tests/unittests/test_distros/test_resolv.py (+1/-1)
tests/unittests/test_distros/test_sles.py (+12/-0)
tests/unittests/test_distros/test_sysconfig.py (+1/-1)
tests/unittests/test_distros/test_user_data_normalize.py (+1/-1)
tests/unittests/test_ds_identify.py (+47/-4)
tests/unittests/test_ec2_util.py (+1/-1)
tests/unittests/test_filters/test_launch_index.py (+1/-1)
tests/unittests/test_handler/test_handler_apt_conf_v1.py (+1/-1)
tests/unittests/test_handler/test_handler_apt_configure_sources_list_v1.py (+1/-1)
tests/unittests/test_handler/test_handler_apt_configure_sources_list_v3.py (+1/-1)
tests/unittests/test_handler/test_handler_apt_source_v1.py (+1/-1)
tests/unittests/test_handler/test_handler_apt_source_v3.py (+1/-1)
tests/unittests/test_handler/test_handler_bootcmd.py (+146/-0)
tests/unittests/test_handler/test_handler_ca_certs.py (+1/-1)
tests/unittests/test_handler/test_handler_chef.py (+76/-12)
tests/unittests/test_handler/test_handler_debug.py (+7/-4)
tests/unittests/test_handler/test_handler_disk_setup.py (+1/-1)
tests/unittests/test_handler/test_handler_growpart.py (+1/-1)
tests/unittests/test_handler/test_handler_landscape.py (+130/-0)
tests/unittests/test_handler/test_handler_locale.py (+57/-3)
tests/unittests/test_handler/test_handler_lxd.py (+1/-1)
tests/unittests/test_handler/test_handler_mcollective.py (+1/-1)
tests/unittests/test_handler/test_handler_mounts.py (+1/-1)
tests/unittests/test_handler/test_handler_ntp.py (+102/-5)
tests/unittests/test_handler/test_handler_power_state.py (+2/-2)
tests/unittests/test_handler/test_handler_puppet.py (+142/-0)
tests/unittests/test_handler/test_handler_resizefs.py (+222/-7)
tests/unittests/test_handler/test_handler_rsyslog.py (+1/-1)
tests/unittests/test_handler/test_handler_runcmd.py (+108/-0)
tests/unittests/test_handler/test_handler_seed_random.py (+1/-1)
tests/unittests/test_handler/test_handler_set_hostname.py (+4/-3)
tests/unittests/test_handler/test_handler_snappy.py (+2/-2)
tests/unittests/test_handler/test_handler_spacewalk.py (+1/-1)
tests/unittests/test_handler/test_handler_timezone.py (+1/-1)
tests/unittests/test_handler/test_handler_write_files.py (+1/-1)
tests/unittests/test_handler/test_handler_yum_add_repo.py (+1/-1)
tests/unittests/test_handler/test_handler_zypper_add_repo.py (+237/-0)
tests/unittests/test_handler/test_schema.py (+151/-16)
tests/unittests/test_helpers.py (+1/-1)
tests/unittests/test_log.py (+58/-0)
tests/unittests/test_merging.py (+1/-1)
tests/unittests/test_net.py (+128/-8)
tests/unittests/test_pathprefix2dict.py (+1/-1)
tests/unittests/test_registry.py (+1/-1)
tests/unittests/test_reporting.py (+1/-1)
tests/unittests/test_rh_subscription.py (+1/-1)
tests/unittests/test_runs/test_merge_run.py (+1/-1)
tests/unittests/test_runs/test_simple_run.py (+106/-21)
tests/unittests/test_sshutil.py (+2/-1)
tests/unittests/test_templating.py (+1/-1)
tests/unittests/test_util.py (+13/-2)
tests/unittests/test_version.py (+1/-1)
tests/unittests/test_vmware_config_file.py (+247/-2)
tools/build-on-freebsd (+0/-1)
tools/ds-identify (+6/-0)
tools/make-tarball (+1/-1)
tools/read-version (+1/-1)
tools/render-cloudcfg (+3/-2)
tools/xkvm (+664/-0)
tox.ini (+31/-11)
- Server Team CI bot: Approve (continuous-integration)
- Scott Moser: Approve
-
Diff: 439 lines (+354/-6)7 files modifiedcloudinit/apport.py (+105/-0)
cloudinit/cmd/devel/logs.py (+101/-0)
cloudinit/cmd/devel/tests/__init__.py (+0/-0)
cloudinit/cmd/devel/tests/test_logs.py (+120/-0)
cloudinit/cmd/main.py (+10/-1)
packages/debian/rules.in (+1/-0)
tests/unittests/test_cli.py (+17/-5)
Changed in maas: | |
milestone: | next → 2.3.0 |
Changed in cloud-init: | |
importance: | Undecided → Wishlist |
Changed in cloud-init: | |
status: | New → Confirmed |
Changed in maas: | |
milestone: | 2.3.0 → 2.3.x |
Changed in maas: | |
milestone: | 2.3.x → 2.5.x |
tags: | added: cdo-qa foundations-engine |
Changed in maas: | |
importance: | Wishlist → High |
Changed in maas: | |
milestone: | 2.5.x → 2.5.0alpha2 |
Changed in maas: | |
milestone: | 2.5.0alpha2 → 2.5.0beta1 |
Changed in maas: | |
milestone: | 2.5.0beta1 → 2.5.0beta2 |
Changed in maas: | |
milestone: | 2.5.0beta2 → 2.5.0 |
Changed in maas: | |
milestone: | 2.5.0 → 2.5.x |
Changed in maas: | |
milestone: | 2.5.x → 2.6.0 |
no longer affects: | maas/2.4 |
Changed in cloud-init: | |
status: | Fix Released → Confirmed |
Changed in cloud-init (Ubuntu): | |
status: | Fix Released → Confirmed |
Changed in cloud-init (Ubuntu Xenial): | |
status: | Fix Released → Confirmed |
Changed in cloud-init: | |
importance: | Wishlist → Medium |
assignee: | Chad Smith (chad.smith) → nobody |
description: | updated |
Changed in cloud-init (Ubuntu Zesty): | |
status: | Fix Released → Won't Fix |
MAAS stores install.log. Maas/Rsyslog/ )
MAAS doesn't store cloud unit logs because cloud-Init doesn't send them. However, you have rsyslog a that are what cloud-I it logs, although, I think it may not be logging correctly (/var/log/