clout init error when running native scripts at start up

[James Flint]

AWS AMI ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-*

ubuntu@ip-10-90-130-11:~$ cloud-init collect
usage: /usr/bin/cloud-init [-h] [--version] [--file FILES] [--debug] [--force]
                           {init,modules,single,dhclient-hook,features,analyze,devel,collect-logs,clean,status}
                           ...
/usr/bin/cloud-init: error: argument subcommand: invalid choice: 'collect' (choose from 'init', 'modules', 'single', 'dhclient-hook', 'features', 'analyze', 'devel', 'collect-logs', 'clean', 'status')

When cloud init runs and hits my native scripts it err and there is no Reason, Stdout or Stderr.

These scripts are preforming some string replaces for config files using 'sed'/

[James Flint (fewknow)]

Scripts that are trying to be ran.

[Chad Smith (chad.smith)]

Your scripts look like they are calling 'cloud-init collect' and not 'cloud-init collect-logs'

collect is not a valid command

/usr/bin/cloud-init: error: argument subcommand: invalid choice: 'collect' (choose from 'init', 'modules', 'single', 'dhclient-hook', 'features', 'analyze', 'devel', 'collect-logs', 'clean', 'status')

[James Flint (fewknow)]

Chad,
    That isn't my error...that was just what I got when collecting the logs. I have attached the error along with the logs. I was just follow the protocol for sending logs. The real error isn't invalid syntax. Please look at the attachments.

[James Flint (fewknow)]

This is some of what is in the cloud-init.log

2018-03-15 13:09:23,723 - util.py[DEBUG]: Running command ['/var/lib/cloud/instance/scripts/vault_update.sh'] with allowed return codes [0] (shell=False, capture=False)
2018-03-15 13:09:57,010 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2018-03-15 13:09:57,011 - handlers.py[DEBUG]: finish: modules-final/config-scripts-user: FAIL: running config-scripts-user with frequency once-per-instance
2018-03-15 13:09:57,011 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed
2018-03-15 13:09:57,011 - util.py[DEBUG]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 793, in _run_modules
    freq=freq)
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 54, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 188, in run
    return (True, results)
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 76, in lock
    yield self._acquire(name, freq)
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 187, in run
    results = functor(*args)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py", line 45, in handle
    util.runparts(runparts_path)
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 810, in runparts
    % (len(failed), len(attempted)))

These are the error I am seeing when my native scripts are executed.
Thanks

[Chad Smith (chad.smith)]

James, sorry I misunderstood because of your description on this bug. I had miswritten the protocol "cloud-init bug reporting guidelines:" when you clicked the original https://bugs.launchpad.net/cloud-init/+filebug.

It should have told you to run "sudo cloud-init collect-logs" instead of 'cloud-init collect'. I fixed that in Launchpad.

cloud-init-output.log will contain your stdout & stderr from scripts run if the traceback didn't give you enough into in /var/log/cloud-init.log

Looking in the cloud-init-output.log that you attached I see sed related errors reported:
sed: -e expression #1, char 37: unknown option to `s'

One quick way to diagnose and fix custom scripts you deliver is to run them in the deployed instance. they all get written to /var/lib/cloud/instance/scripts/.
You can look at those scripts to make sure they show the content you expected and iteratively run then and sort the errors with your scripts there to figure out what's wrong.

Setting this back in invalid, unless you have updates that you think indicate a cloud-init bug.

[Chad Smith (chad.smith)]

If there is a bug that needs tracking, please feel free to update the description of this bug appropriately and set it back to New

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3138