Cloud init running module Ansible failed

[Wilfred Arambhan]

I'm trying to run an ansible-pull on cloud-init 'final' stage. And I'm seeing the below error. This was previously working. Also attaching my cloud-init yaml file.

Setting up build-essential (12.8ubuntu1.1) ...
Setting up python3-dev (3.8.2-0ubuntu2) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.12) ...
Cloud-init v. 23.3.1-0ubuntu1~20.04.1 running 'modules:final' at Fri, 17 Nov 2023 10:00:20 +0000. Up 23.05 seconds.
2023-11-17 10:01:46,514 - util.py[WARNING]: Running module ansible (<module 'cloudinit.config.cc_ansible' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ansible.py'>) failed
Cloud-init v. 23.3.1-0ubuntu1~20.04.1 finished at Fri, 17 Nov 2023 10:01:46 +0000. Datasource DataSourceEc2Local. Up 108.93 seconds

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: cloud-init 23.3.1-0ubuntu1~20.04.1
ProcVersionSignature: Ubuntu 5.15.0-1049.54~20.04.1-aws 5.15.126
Uname: Linux 5.15.0-1049-aws x86_64
ApportVersion: 2.20.11-0ubuntu27.27
Architecture: amd64
CasperMD5CheckResult: skip
Date: Fri Nov 17 10:16:57 2023
Ec2AMI: ami-0c7090bd76537f73d
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: ap-south-1a
Ec2InstanceType: t2.micro
Ec2Kernel: unavailable
Ec2Ramdisk: unavailable
PackageArchitecture: all
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
SourcePackage: cloud-init
UpgradeStatus: No upgrade log present (probably fresh install)
cloud-init-log-warnings:
 2023-11-17 10:00:06,278 - DataSourceEc2.py[DEBUG]: strict_mode: warn, cloud_name=aws cloud_platform=ec2
 2023-11-17 10:01:46,514 - util.py[WARNING]: Running module ansible (<module 'cloudinit.config.cc_ansible' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ansible.py'>) failed
     raise ProcessExecutionError(
 cloudinit.subp.ProcessExecutionError: Unexpected error while running command.

[Chad Smith (chad.smith)]

Thank you for filing a bug and making cloud-init better. Looking at the traceback, this is a duplicate of the following bug: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/2040291

It is known to affect cloud-init version 23.3.1 and was addressed upstream here https://github.com/canonical/cloud-init/pull/4403 and landed in this commit https://github.com/canonical/cloud-init/commit/13938f749f7393eaa47a9d5ddd286e675ff2944d.

This release is currently queued a -proposed as an upload 23.3.3 in Ubuntu Focal, Jammy, Lunar and Mantic (and already released in Ubuntu Nobel, the devel series).

We expect to see this fix published next week. Thanks again!

=== Recognized traceback on invalid optional --break-system-packages param.

    raise ProcessExecutionError(
cloudinit.subp.ProcessExecutionError: Unexpected error while running command.
Command: ['su', '-', 'ansible', '-c', 'env PATH=$PATH /usr/bin/python3 -m pip install --break-system-packages --user --upgrade pip']
Exit code: 2
Reason: -
Stdout:
Stderr:
        Usage:
          /usr/bin/python3 -m pip install [options] <requirement specifier> [package-index-options] ...
          /usr/bin/python3 -m pip install [options] -r <requirements file> [package-index-options] ...
          /usr/bin/python3 -m pip install [options] [-e] <vcs project url> ...
          /usr/bin/python3 -m pip install [options] [-e] <local project path> ...
          /usr/bin/python3 -m pip install [options] <archive url/path> ...

        no such option: --break-system-packages