pro-client decodes utf-8 strictly, which causes problems in some locales

[sistemas_grupo-cps]

[ Impact ]

If for any reason the user's system has any non-utf8 character as part of any subprocess command executed, the Pro Client will fail to parse the return output and break.

Independent of why this happens, the Pro Client should not be relying on CLI output when there are Python bindings available to perform the same task - and python3-apt provides ways to list installed packages.

The fix to this bug is to ignore utf-8 errors in the occasions where it needs to subprocess. Even other software than APT may include non-utf8 output by chance. To avoid corner case scenarios, setting LANG=C when executing any subprocess should mitigate the presence of non-utf8 anyway.

[ Test Plan ]

Subprocessing is part of the regular operation of the Pro Client in many scenarios covered by the integration / acceptance tests.

For this particular issue:
- Launch a container for the release to be tested
- Install a non-utf-8 locale and set it as the default. e.g.
    - sudo locale-gen fr_FR
    - sudo dpkg-reconfigure locales # select fr_FR
    - # log out and log in
- pro status
- see it fail
- install the package from -proposed
- see it does not fail anymore

[ Where problems could occur ]

Ignoring non-utf8 characters when decoding will not prevent the Client from reading all the actual utf-8 present in the output. The pieces we parse from the commands we execute are not involving any possible non-utf8 text. If we would have assessed this wrong, this could cause all sorts of regressions.

To be extra careful, we are setting LANG=C.UTF-8 - so we have way better expectation management on the outputs from any external command we run on a system.

[ Original description ]

["2024-04-10T08:28:17.952", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/os-release", {}]
["2024-04-10T08:28:17.953", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /proc/version_signature", {}]
["2024-04-10T08:28:18.575", "ERROR", "ubuntupro.cli", "wrapper", 1768, "Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent call
 last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File \"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 1475, in action_status\n status, ret = actions.status(\n File \"/usr/lib/python3/dist-package
s/uaclient/actions.py\", line 173, in status\n status = ua_status.status(cfg=cfg, show_all=show_all)\n File \"/usr/lib/python3/dist-packages/uaclient/
status.py\", line 418, in status\n response = _unattached_status(cfg)\n File \"/usr/lib/python3/dist-packages/uaclient/status.py\", line 283, in _unat
tached_status\n resources = get_available_resources(cfg)\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 680, in get_available_re
sources\n resources = client.available_resources()\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 114, in available_resources\n
   activity_info = self._get_activity_info()\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in _get_activity_info\n \"deskt
op\": system.is_desktop(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 397, in is_desktop\n for package in apt.get_installed_pa
ckages():\n File \"/usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in get_installed_packages\n out, _ = system.subp([\"apt\", \"list\", \"
--installed\"])\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n out, err = _subp(\n File \"/usr/lib/python3/dist-pa
ckages/uaclient/system.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode b
yte 0xe1 in position 109: invalid continuation byte"}]
["2024-04-10T08:30:25.404", "DEBUG", "ubuntupro.config", "parse_config", 625, "Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}
]
["2024-04-10T08:30:25.408", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2024-04-10T08:30:25.409", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/user-config.json but file does not exis
t", {}]
["2024-04-10T08:30:25.421", "DEBUG", "ubuntupro.http", "configure_web_proxy", 146, "Setting no_proxy: 169.254.169.254,[fd00:ec2::254],metadata", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.http", "configure_web_proxy", 154, "Setting global proxy dict", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.cli", "main", 1825, "Executed with sys.argv: ['/usr/bin/pro', 'attach', '<REDACTED>']", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.lock", "__enter__", 77, "spin lock starting for pro attach", {}]
["2024-04-10T08:30:25.423", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmparyyoirn", {}]
["2024-04-10T08:30:25.423", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /var/lib/ubuntu-advantage/lock atomically via tempfile /var/lib/
ubuntu-advantage/tmpn1yxl91o", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmpwqyq85v1", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/machine-id", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/os-release", {}]
["2024-04-10T08:30:25.425", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /proc/version_signature", {}]
["2024-04-10T08:30:25.970", "DEBUG", "ubuntupro.system", "ensure_file_absent", 576, "Removed file: /run/ubuntu-advantage/notices/60-operation_in_progress"
, {}]
["2024-04-10T08:30:25.970", "DEBUG", "ubuntupro.system", "ensure_file_absent", 576, "Removed file: /var/lib/ubuntu-advantage/lock", {}]
["2024-04-10T08:30:25.971", "ERROR", "ubuntupro.cli", "wrapper", 1768, "Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent call
 last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File \"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 257, in new_f\n return f(args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uac
lient/cli/__init__.py\", line 202, in new_f\n return f(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 186,
in new_f\n retval = f(*args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1305, in action_attach\n a
ctions.attach_with_token(cfg, token=token, allow_enable=allow_enable)\n File \"/usr/lib/python3/dist-packages/uaclient/actions.py\", line 65, in attach_w
ith_token\n new_machine_token = contract_client.add_contract_machine(\n File \"/usr/lib/python3/dist-packages/uaclient/util.py\", line 92, in decorato
r\n return f(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 92, in add_contract_machine\n activity_info = se
lf._get_activity_info()\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in _get_activity_info\n \"desktop\": system.is_deskt
op(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 397, in is_desktop\n for package in apt.get_installed_packages():\n File \"/
usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in get_installed_packages\n out, _ = system.subp([\"apt\", \"list\", \"--installed\"])\n Fi
le \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n out, err = _subp(\n File \"/usr/lib/python3/dist-packages/uaclient/syste
m.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position
109: invalid continuation byte"}]
["2024-04-10T08:38:29.763", "DEBUG", "ubuntupro.config", "parse_config", 625, "Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}
]
["2024-04-10T08:38:29.767", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2024-04-10T08:38:29.768", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/user-config.json but file does not exis
t", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.http", "configure_web_proxy", 146, "Setting no_proxy: 169.254.169.254,[fd00:ec2::254],metadata", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.http", "configure_web_proxy", 154, "Setting global proxy dict", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.cli", "main", 1825, "Executed with sys.argv: ['/usr/bin/pro', 'attach', '<REDACTED>']", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.lock", "__enter__", 77, "spin lock starting for pro attach", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmplx6ehdag", {}]
["2024-04-10T08:38:29.782", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /var/lib/ubuntu-advantage/lock atomically via tempfile /var/lib/
ubuntu-advantage/tmp6ml_ltfj", {}]
["2024-04-10T08:38:29.782", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmpvxf_b1v0", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/machine-id", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/os-release", {}]
["2024-04-10T08:38:29.784", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /proc/version_signature", {}]
["2024-04-10T08:38:30.366", "DEBUG", "ubuntupro.system", "ensure_file_absent", 576, "Removed file: /run/ubuntu-advantage/notices/60-operation_in_progress"
, {}]
["2024-04-10T08:38:30.367", "DEBUG", "ubuntupro.system", "ensure_file_absent", 576, "Removed file: /var/lib/ubuntu-advantage/lock", {}]
["2024-04-10T08:38:30.367", "ERROR", "ubuntupro.cli", "wrapper", 1768, "Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent call
 last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File \"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 257, in new_f\n return f(args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uac
lient/cli/__init__.py\", line 202, in new_f\n return f(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 186,
in new_f\n retval = f(*args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1305, in action_attach\n a
ctions.attach_with_token(cfg, token=token, allow_enable=allow_enable)\n File \"/usr/lib/python3/dist-packages/uaclient/actions.py\", line 65, in attach_w
ith_token\n new_machine_token = contract_client.add_contract_machine(\n File \"/usr/lib/python3/dist-packages/uaclient/util.py\", line 92, in decorato
r\n return f(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 92, in add_contract_machine\n activity_info = se
lf._get_activity_info()\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in _get_activity_info\n \"desktop\": system.is_deskt
op(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 397, in is_desktop\n for package in apt.get_installed_packages():\n File \"/
usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in get_installed_packages\n out, _ = system.subp([\"apt\", \"list\", \"--installed\"])\n Fi
le \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n out, err = _subp(\n File \"/usr/lib/python3/dist-packages/uaclient/syste
m.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position
109: invalid continuation byte"}]

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: ubuntu-advantage-tools 31.2~20.04
ProcVersionSignature: Ubuntu 5.15.0-101.111~20.04.1-generic 5.15.143
Uname: Linux 5.15.0-101-generic x86_64
NonfreeKernelModules: falcon_lsm_serviceable falcon_nf_netcontain falcon_kal falcon_lsm_pinned_16407 falcon_lsm_pinned_16206
ApportVersion: 2.20.11-0ubuntu27.27
Architecture: amd64
CasperMD5CheckResult: skip
Date: Wed Apr 10 08:28:11 2024
PackageArchitecture: all
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 LANG=es_ES
 SHELL=/bin/bash
SourcePackage: ubuntu-advantage-tools
UpgradeStatus: No upgrade log present (probably fresh install)
apparmor_logs.txt:

cloud-id.txt-error: Invalid command specified 'cloud-id'.
livepatch-status.txt-error: Invalid command specified '/snap/bin/canonical-livepatch status'.
ua-status.json-error: Failed running command 'pro status --format json' [exit(1)]. Message:
uaclient.conf:
 contract_url: https://contracts.canonical.com
 log_level: debug

[Grant Orndorff (orndorffgrant)]

Hi sistemascps!

Thank you for the bug report and sorry for the delay in responding. It looks like a non-UTF-8 character is present in the output of `apt list --installed`.

The fix will be to replace the usage of `apt list --installed` with using the python3-apt library to get the list of installed packages.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 33

---------------
ubuntu-advantage-tools (33) oracular; urgency=medium

  * d/apparmor: adjust the esm_cache apparmor profile to allow reading of dpkg
    data directory (LP: #2067810) (GH: #3137)
  * New upstream release 33 (LP: #2069237)
    - apt: use Python bindings instead of apt CLI to query for installed
      packages (LP: #2060769) (LP: #2068744)
    - beta: drop support for beta services
    - contracts: add support for contracts which target a specific series
    - fips: change enable functionality to ensure all packages with a FIPS
      candidate are upgraded to the FIPS version (GH: #2667)
    - fix:
      + add the current_status field to the plan api return object
      + change recommended attach method to magic attach (GH: #3040)
    - livepatch: prefer the term 'coverage' instead of 'support' in messaging
      (GH: #3063)
    - realtime:
      + auto-select the raspi variant when appropriate
      + inform the user when auto-selecting a variant

 -- Renan Rodrigo <email address hidden> Thu, 13 Jun 2024 00:19:54 -0300

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33~24.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33~22.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33~20.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33~18.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33~16.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Jean-Baptiste Lallement (jibel)]

I tried on noble and it doesn't fix the problem.

* Firstly the upgrade to the package in proposed doesn't work if I do not force the locale.
```
Il est n cessaire de prendre 250 ko dans les archives.
Apr s cette op ration, 12,3 ko d'espace disque suppl mentaires seront utilis s.
R ception de :1 http://archive.ubuntu.com/ubuntu noble-proposed/main amd64 ubuntu-pro-client-l10n amd64 33~24.04 [19,3 kB]
R ception de :2 http://archive.ubuntu.com/ubuntu noble-proposed/main amd64 ubuntu-pro-client amd64 33~24.04 [231 kB]
250 ko r ceptionn s en 0s (938 ko/s)
(Lecture de la base de donn es... 40702 fichiers et r pertoires d j install s.)
Pr paration du d paquetage de .../ubuntu-pro-client-l10n_33~24.04_amd64.deb ...
D paquetage de ubuntu-pro-client-l10n (33~24.04) sur (32.3~24.04) ...
Pr paration du d paquetage de .../ubuntu-pro-client_33~24.04_amd64.deb ...
Traceback (most recent call last):
  File "/usr/bin/py3clean", line 210, in <module>
    main()
  File "/usr/bin/py3clean", line 196, in main
    pfiles = set(dpf.from_package(options.package))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/python3/debpython/files.py", line 54, in from_package
    stdout = str(stdout, 'utf-8')
             ^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 584: invalid continuation byte
dpkg: avertissement: le sous-processus ancien paquet ubuntu-pro-client script pre-removal a renvoy un tat de sortie d'erreur 1
```

* Secondly, once the package is installed, pro status is still failing:
```
u@drenec:~$ locale
LANG=fr_FR
LANGUAGE=
LC_CTYPE="fr_FR"
LC_NUMERIC="fr_FR"
LC_TIME="fr_FR"
LC_COLLATE="fr_FR"
LC_MONETARY="fr_FR"
LC_MESSAGES="fr_FR"
LC_PAPER="fr_FR"
LC_NAME="fr_FR"
LC_ADDRESS="fr_FR"
LC_TELEPHONE="fr_FR"
LC_MEASUREMENT="fr_FR"
LC_IDENTIFICATION="fr_FR"
LC_ALL=
u@drenec:~$ apt policy ubuntu-pro-client
ubuntu-pro-client:
  Install�: 33~24.04
  Candidat�: 33~24.04
 Table de version�:
 *** 33~24.04 100
        100 http://archive.ubuntu.com/ubuntu noble-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     32.3.1~24.04 500
        500 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages
     31.2.3 500
        500 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages
u@drenec:~$ pro status
An unexpected error occurred: 'utf-8' codec can't decode byte 0xa0 in position 18: invalid start byte
For more details, see the log: /home/u/.cache/ubuntu-pro/ubuntu-pro.log
If you think this is a bug, please run: ubuntu-bug ubuntu-advantage-tools
```

From the log the error is:
```
["2024-07-10T12:14:41.978", "ERROR", "ubuntupro.cli", "wrapper", 1360, "Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent call last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1297, in wrapper\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1434, in main\n return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init...

[Christian Ehrhardt  (paelzer)]

Interesting, the team will later have a deeper look.
But I'm confused as you can see in the following:

---

First confirming the original problem

root@n:~# locale
LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="fr_FR"
LC_NUMERIC="fr_FR"
LC_TIME="fr_FR"
LC_COLLATE="fr_FR"
LC_MONETARY="fr_FR"
LC_MESSAGES="fr_FR"
LC_PAPER="fr_FR"
LC_NAME="fr_FR"
LC_ADDRESS="fr_FR"
LC_TELEPHONE="fr_FR"
LC_MEASUREMENT="fr_FR"
LC_IDENTIFICATION="fr_FR"
LC_ALL=fr_FR
root@n:~# pro status
An unexpected error occurred: 'utf-8' codec can't decode byte 0xe9 in position 65: invalid continuation byte
For more details, see the log: /var/log/ubuntu-advantage.log
If you think this is a bug, please run: ubuntu-bug ubuntu-advantage-tools

---

Then I can sadly confirm that upgrading indeed is broken the same way you mentioned.

root@n:~# apt install ubuntu-pro-client
Lecture des listes de paquets... Fait
Construction de l'arbre des d�pendances... Fait
Lecture des informations d'�tat... Fait
ubuntu-pro-client est d�j� la version la plus r�cente (32.3.1~24.04).
ubuntu-pro-client pass� en ��install� manuellement��.
Les paquets suivants ont �t� install�s automatiquement et ne sont plus n�cessaires�:
  libaio1 libsoup2.4-1 libsoup2.4-common libvpx8 python3-boto
Veuillez utiliser ��apt autoremove�� pour les supprimer.
0 mis � jour, 0 nouvellement install�s, 0 � enlever et 3 non mis � jour.
root@n:~# apt install ubuntu-pro-client -t noble/proposed
Lecture des listes de paquets... Fait
E: La valeur ��noble/proposed�� n'est pas valable pour APT::Default-Release car cette version ne fait pas partie des sources disponibles.
root@n:~# apt install ubuntu-pro-client=33~24.04
Lecture des listes de paquets... Fait
Construction de l'arbre des d�pendances... Fait
Lecture des informations d'�tat... Fait
Les paquets suivants ont �t� install�s automatiquement et ne sont plus n�cessaires�:
  libaio1 libsoup2.4-1 libsoup2.4-common libvpx8 python3-boto
Veuillez utiliser ��apt autoremove�� pour les supprimer.
Paquets recommand�s�:
  ubuntu-pro-client-l10n
Les paquets suivants seront ENLEV�S�:
  ubuntu-pro-client-l10n
Les paquets suivants seront mis � jour�:
  ubuntu-pro-client
1 mis � jour, 0 nouvellement install�s, 1 � enlever et 3 non mis � jour.
Il est n�cessaire de prendre 231 ko dans les archives.
Apr�s cette op�ration, 81,9 ko d'espace disque seront lib�r�s.
Souhaitez-vous continuer�? [O/n] Y
R�ception de�:1 http://archive.ubuntu.com/ubuntu noble-proposed/main amd64 ubuntu-pro-client amd64 33~24.04 [231 kB]
231 ko r�ceptionn�s en 0s (1�210 ko/s)
(Lecture de la base de donn�es... 110216 fichiers et r�pertoires d�j� install�s.)
Suppression de ubuntu-pro-client-l10n (32.3.1~24.04) ...
(Lecture de la base de donn�es... 110215 fichiers et r�pertoires d�j� install�s.)
Pr�paration du d�paquetage de .../ubuntu-pro-client_33~24.04_amd64.deb ...
Traceback (most recent call last):
  File "/usr/bin/py3clean", line 210, in <module>
    main()
  File "/usr/bin/py3clean", line 196, in main
    pfiles = set(dpf.from_package(options.package))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/python3/debpython/files.py", line 54, in from_package
    stdout = str(stdout, 'utf-8'...

[Jean-Baptiste Lallement (jibel)]

The only difference I noticed between you case and mine, is that your machine is not attached and mine is.

```
$ LC_ALL=C pro status
SERVICE ENTITLED STATUS DESCRIPTION
anbox-cloud yes disabled Scalable Android in the cloud
esm-apps yes enabled Expanded Security Maintenance for Applications
esm-infra yes disabled Expanded Security Maintenance for Infrastructure
landscape yes disabled Management and administration tool for Ubuntu

For a list of all Ubuntu Pro services, run 'pro status --all'
Enable services with: pro enable <service>

     Account: <my account>
Subscription: Ubuntu Pro - free personal subscription
```

[Christian Ehrhardt  (paelzer)]

Out of a discussion with Renan.

On the system that worked, this triggers it again as it will run apt-cache policy.

root@n:~# pro attach C...
An unexpected error occurred: 'utf-8' codec can't decode byte 0xa0 in position 18: invalid start byte
For more details, see the log: /var/log/ubuntu-advantage.log
If you think this is a bug, please run: ubuntu-bug ubuntu-advantage-tools

root@n:~# pro detach
An unexpected error occurred: 'utf-8' codec can't decode byte 0xa0 in position 18: invalid start byte
For more details, see the log: /var/log/ubuntu-advantage.log
If you think this is a bug, please run: ubuntu-bug ubuntu-advantage-tools

The problem is that the problem that this was is fixed "apt list --installed".
But this turns out to now be other calls, in that case apt-cache policy.

root@n:~# apt-cache policy
Fichiers du paquet�:
 100 /var/lib/dpkg/status
...

You see the bad char here.
So it is more that all kind of calls could return unparseable, based on the language settings, and hit a bug quite similar to this.

So it seems:
- the original bug is actually fixed
- at least one more similar case is open still
- If we can we will try to ignore parsing errors kind of everywhere.
- If not, then this would need to become a new issue as it would need a bigger fix.

[Renan Rodrigo (renanrodrigo)]

Given the wider range of problems represented in the bug, I have changed the title/description to accomodate any subprocess returning non-utf8. There is also a new fix plan. I will implement it and we will upload a 33.1 version to proposed fixing this.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (33~16.04) for xenial have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/unknown (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 33.1

---------------
ubuntu-advantage-tools (33.1) oracular; urgency=medium

  * New upstream release 33.1: (LP: #2060769)
    - system:
      + always pass C.UTF8 as the language when calling a subprocess
      + ignore utf-8 decode errors on subprocess output

 -- Renan Rodrigo <email address hidden> Wed, 10 Jul 2024 16:43:02 -0300

[Robie Basak (racb)]

> setting LANG=C when executing any subprocess should mitigate the presence of non-utf8 anyway

This alarmed me so I looked deeper but I think it's just a mistake and should say UTF-8. But then I noticed:

> + always pass C.UTF8 as the language when calling a subprocess
> To be extra careful, we are setting LANG=C.UTF-8...

And the code:

> + set_lang = {"LANG": "C.UTF8", "LC_ALL": "C.UTF8"}

Is it UTF8 or UTF-8? This led me down a rabbithole.

On a Xenial system `locale -a` reports C.UTF-8 and en_GB.utf8 as available, but not C.UTF8.

On a Noble system `locale -a` reports C.utf8.

So I guess it's all fine and will be treated as equivalent, including back all the way to Xenial. But I wonder if there's a way to test for this.

And now that I'm thinking about it, perhaps we want to set LC_ALL instead of LANG?

Sorry I didn't notice this in my previous review. I was focused on the SRU specifics and not thinking about it from a general review perspective.

Anyway, I think it's all fine, so accepting. I only mention the above in case it identified an area needing attention to someone else.

[Robie Basak (racb)]

> And now that I'm thinking about it, perhaps we want to set LC_ALL instead of LANG?

Oh, sorry, you are.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.1~24.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.1~22.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.1~20.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.1~18.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.1~16.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (33~16.04) for xenial have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/unknown (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (33~24.04) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

software-properties/unknown (amd64, arm64, armhf, i386, ppc64el, s390x)
ubuntu-advantage-tools/unknown (armhf)
update-motd/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (33.1~24.04) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

software-properties/unknown (amd64, arm64, armhf, i386, ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.2~24.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.2~22.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.2~20.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.2~18.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello sistemas_grupo-cps, or anyone else affected,

Accepted ubuntu-advantage-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/33.2~16.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (33.1~24.04) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

software-properties/unknown (amd64, arm64, armhf, i386, ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (33.2~24.04) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

software-properties/unknown (amd64, i386, ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (33.2~22.04) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/unknown (armhf)
update-motd/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Renan Rodrigo (renanrodrigo)]

The Pro Client test suite now contains an integration test that covers the scenario described in this bug (Pro client's commands run successfully in a non-utf8 locale).

I have performed the test steps described above for all valid releases, and was able to see the bug is fixed using the package from proposed.

There is a problem, as pointed out in #9 and #10 here, when upgrading the package. But that happens when running prerm scripts for the older version of the Client, and debpython expects utf-8 output as well. This means:
- this problem when upgrading without utf-8 support may happen to other packages, which use debpython, and
- we can't fix it (:

Gievn we can now run all Pro Client commands even if a non-utf8 locale is set, I am marking this bug verification-done for all releases.

[Robie Basak (racb)]

> - we can't fix it (:

I'm not sure about this.

> But that happens when running prerm scripts for the older version of the Client...

If this is "old-prerm upgrade" that is returning a non-zero exit status then we can avoid a dpkg failure by providing "new-prerm failed-upgrade".

Perhaps I've misunderstood. I think it's worth making sure that we aren't going to cause a dpkg failure on users as a result of this SRU.

[Robie Basak (racb)]

I looked into this with Renan.

We think this is a separate issue and I have filed bug 2075337.

It would seem to affect any package that uses dh_python3 which inserts py3clean into the package prerm.

For SRU purposes, I would have preferred to hold publication of package updates that will trigger the issue even though we know that it isn't caused by this package.

However, doing that in this case would seem on balance to be too onerous. Users are already affected by this issue in other cases. We'd have to stop SRUs of all affected packages pending the fix. That might have been appropriate if this affected all UTF-8 using users, but seems a bit too far to go for users of other character sets.

We have already had one SRU of ubuntu-advantage-tools with no reports of failures. I therefore think it's OK to release this SRU now, without waiting for bug 2075337 to be fixed. The workaround is to use UTF-8 as is the general recommendation.

[Robie Basak (racb)]

(not strictly for this bug but it doesn't seem worth trying to add another one that won't be linked SRU-process-wise)

Update: Didier is concerned about potential implications for WSL due to bug 2075337, so we're waiting on an ack from that side.

For now, I'm going to change the tag back to verification-needed to ensure that there isn't an accidental release. But this bug is actually verified - it's bug 2075337 that's the real issue.

[Christian Ehrhardt  (paelzer)]

Ack to Renan

"this bug is actually verified - it's bug 2075337 that's the real issue"

Yes, and that isn't a ubuntu advantage tools bug btw, it is a generic potentially all python package issue.
And while it isn't a ubuntu advantage tools bug, being installed on all systems might cause it more widespread than a rare leaf package - hence we want some confidence on bug 2075337 implications first.

"going to change the tag back to verification-needed to ensure that there isn't an accidental release"

I think to even better represent what is going on, we should use the "block-proposed" tag [1] here.

[1]: https://wiki.ubuntu.com/ProposedMigration#Migrating_packages_from_-proposed_to_release

[Robie Basak (racb)]

> I think to even better represent what is going on, we should use the "block-proposed" tag [1] here.

No, that's not right. That's for the development release only, and doesn't show on the pending SRU report.