Comment 2 for bug 1648380

ok, while my previous analysis was correct, the first stage runs correctly w/o finding a DataSource (at least it looks like it).

But when it runs the init-network phase, things seem to go badly :

Nov 21 10:12:34 ubuntu [CLOUDINIT] handlers.py[DEBUG]: finish: init-network/check-cache: SUCCESS: no cache found
Nov 21 10:12:34 ubuntu [CLOUDINIT] util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance
Nov 21 10:12:34 ubuntu [CLOUDINIT] stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.ubuntu.Distro'>
Nov 21 10:12:34 ubuntu [CLOUDINIT] __init__.py[DEBUG]: Looking for for data source in: ['CloudSigma'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM', 'NETWORK']
Nov 21 10:12:34 ubuntu [CLOUDINIT] __init__.py[DEBUG]: Searching for network data source in: []
Nov 21 10:12:34 ubuntu [CLOUDINIT] util.py[WARNING]: No instance datasource found! Likely bad things to come!
Nov 21 10:12:34 ubuntu [CLOUDINIT] util.py[DEBUG]: No instance datasource found! Likely bad things to come!

cloud-init.log:Nov 21 10:12:34 ubuntu [CLOUDINIT] util.py[DEBUG]: No instance datasource found! Likely bad things to come!
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 247, in main_init
    init.fetch(existing=existing)
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 358, in fetch
    return self._get_data_source(existing=existing)
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 268, in _get_data_source
    pkg_list, self.reporter)
  File "/usr/lib/python3/distpackages/cloudinit/sources/__init__.py", line 318, in find_source
    raise DataSourceNotFoundException(msg)
cloudinit.sources.DataSourceNotFoundException: Did not find any data source, searched classes: ()

The cleaner backtrace comes from /var/log/syslog.3.gz :

Nov 21 10:12:35 ubuntu cloud-init[974]: Can not apply stage config, no datasource found! Likely bad things to come!
Nov 21 10:12:35 ubuntu cloud-init[974]: ------------------------------------------------------------
Nov 21 10:12:35 ubuntu cloud-init[974]: Traceback (most recent call last):
Nov 21 10:12:35 ubuntu cloud-init[974]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 351, in main_modules
Nov 21 10:12:35 ubuntu cloud-init[974]: init.fetch(existing="trust")
Nov 21 10:12:35 ubuntu cloud-init[974]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 358, in fetch
Nov 21 10:12:35 ubuntu cloud-init[974]: return self._get_data_source(existing=existing)
Nov 21 10:12:35 ubuntu cloud-init[974]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 268, in _get_data_source
Nov 21 10:12:35 ubuntu cloud-init[974]: pkg_list, self.reporter)
Nov 21 10:12:35 ubuntu cloud-init[974]: File "/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 318, in find_source
Nov 21 10:12:35 ubuntu cloud-init[974]: raise DataSourceNotFoundException(msg)
Nov 21 10:12:35 ubuntu cloud-init[974]: cloudinit.sources.DataSourceNotFoundException: Did not find any data source, searched classes: ()
Nov 21 10:12:35 ubuntu cloud-init[974]: ------------------------------------------------------------
Nov 21 10:12:35 ubuntu systemd[1]: cloud-config.service: Main process exited, code=exited, status=1/FAILURE
Nov 21 10:12:35 ubuntu systemd[1]: Failed to start Apply the settings specified in cloud-config.

As mentionned in my previous comment that failure happens when this is called:

cfg_list = ['CloudSigma']
pkg_list = ['', 'cloudinit.sources']
ds_deps = ['FILESYSTEM', 'NETWORK']

ds_list = list_sources(cfg_list, ds_deps, pkg_list)
ds_names = [type_utils.obj_name(f) for f in ds_list]

ds_list & ds_names will be empty.

I suspect that the following change between cloud-init_0.7.7~bzr1212 (version in 20160627 image) and cloud-init_0.7.8-1-g3705bb5 (version in 20161020) is the reason for that failure :

@@ -119,17 +109,13 @@
         return self.metadata['uuid']

-class DataSourceCloudSigmaNet(DataSourceCloudSigma):
- def __init__(self, sys_cfg, distro, paths):
- DataSourceCloudSigma.__init__(self, sys_cfg, distro, paths)
- self.dsmode = 'net'
-
+# Legacy: Must be present in case we load an old pkl object
+DataSourceCloudSigmaNet = DataSourceCloudSigma

 # Used to match classes to dependencies. Since this datasource uses the serial
 # port network is not really required, so it's okay to load without it, too.
 datasources = [
     (DataSourceCloudSigma, (sources.DEP_FILESYSTEM)),
- (DataSourceCloudSigmaNet, (sources.DEP_FILESYSTEM, sources.DEP_NETWORK)),
 ]

And most particularly the removal of :
(DataSourceCloudSigmaNet, (sources.DEP_FILESYSTEM, sources.DEP_NETWORK))

In order to verify this assertion, I added the removed element in the list & ran list_sources() with the same parameters (from within the downloaded image) :

>>> import sources
>>> from sources import DataSourceCloudSigma as DataSourceCloudSigma
>>> import type_utils
>>> cfg_list = ['CloudSigma']
>>> pkg_list = ['', 'cloudinit.sources']
>>> ds_deps = ['FILESYSTEM', 'NETWORK']
>>> ds_list = sources.list_sources(cfg_list, ds_deps, pkg_list)
>>> ds_names = [type_utils.obj_name(f) for f in ds_list]
>>> ds_list
[<class 'cloudinit.sources.DataSourceCloudSigma.DataSourceCloudSigma'>]
>>> ds_names
['DataSourceCloudSigma']

This time, the proper datasource is correctly found.

Now I'm off to find out why it got removed