UnicodeDecodeError in add_repo_add_key_and_install_app when a appname has special chars in it

[Gary Lasker]

[TEST CASE]
Unfortunately, there is currently not a convenient way to test this error condition as it specifically requires an application in the for-purchase archive to contain a unicode character in its name. Currently, no such application is available. However, a unit test that reproduces the error and verifies the fix directly is included in the source code for this version of Software Center (see branch details below), and this has been run at build time.

It is still very useful to test for regressions as follows:
1. Install and launch the version of software-center in precise-proposed.
2. Click the down arrow next for "All Software" and select the "For Purchase" entry.
3. Navigate in the list and make sure that all of the titles are displayed correctly. View the details view for various items and verify the titles there as well.

[REGRESSION POTENTIAL]
This is a fix that simply corrects a unicode decode error in a debug log statement, so the risk of regression is negligible.

---

The esteemed davmor2 discovered a new UnicodeDecodeError while testing a new app for the For Purchase section. This application appears to be the first one we've deployed For Purchase that uses a unicode character in its title, which explains why we have not seen this particular one before.

The title of the associated application is "Linux Shell 01 - Manual Práctico del Shell".

Further details from davmor2:

Bug Description:
A recent magazine came through with a unicode character in the title of the application and USC/SCA combo was unable to handle it.

Version:
n/a

OS:
64bit Oneiric

USC Version:
5.0.4

Steps To Reproduce:
1. Goto myapps
2. Create an app with a unicode title
3. Myapps will publish this
4. USC will fail to download/install this

Expected Results:
I expect to be able to create an app and publish it. That or we need to document that only non unicode characters may be applied to a title.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: software-center 5.0.4
ProcVersionSignature: Ubuntu 3.0.0-15.25-generic 3.0.13
Uname: Linux 3.0.0-15-generic x86_64
NonfreeKernelModules: fglrx
ApportVersion: 1.23-0ubuntu4
Architecture: amd64
Date: Tue Jan 24 12:52:52 2012
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release amd64 (20111011)
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_GB:en
 PATH=(custom, no user)
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SoftwareCenterLog:
 2012-01-24 12:41:14,289 - softwarecenter.backend - INFO - looking for '/var/lib/apt/lists/private-ppa.launchpad.net_commercial-ppa-uploaders_android-user-issue-201202_ubuntu_dists_oneiric_Release.gpg'
 2012-01-24 12:41:14,355 - softwarecenter.backend - INFO - install_package()
 2012-01-24 12:41:15,186 - softwarecenter.backend - INFO - run_transaction()
 2012-01-24 12:42:56,237 - softwarecenter.utils - WARNING - Could not determine the installed desktop file path for app-install desktop file: 'software-center-agent'
SourcePackage: software-center
UpgradeStatus: No upgrade log present (probably fresh install)

Here is the stacktrace:

Traceback (most recent call last):
  File "/usr/share/software-center/softwarecenter/ui/gtk3/widgets/apptreeview.py", line 452, in _cell_data_func_cb
    model.load_range(indices, 5)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/models/appstore2.py", line 441, in load_range
    if self[(i,)][0]: continue
  File "/usr/lib/python2.7/dist-packages/gi/overrides/Gtk.py", line 732, in __getitem__
    raise IndexError("could not find tree path '%s'" % key)
IndexError: could not find tree path '197'
Traceback (most recent call last):
  File "/usr/share/software-center/softwarecenter/ui/gtk3/widgets/apptreeview.py", line 452, in _cell_data_func_cb
    model.load_range(indices, 5)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/models/appstore2.py", line 441, in load_range
    if self[(i,)][0]: continue
  File "/usr/lib/python2.7/dist-packages/gi/overrides/Gtk.py", line 732, in __getitem__
    raise IndexError("could not find tree path '%s'" % key)
IndexError: could not find tree path '199'
Traceback (most recent call last):
  File "/usr/share/software-center/softwarecenter/ui/gtk3/widgets/apptreeview.py", line 452, in _cell_data_func_cb
    model.load_range(indices, 5)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/models/appstore2.py", line 441, in load_range
    if self[(i,)][0]: continue
  File "/usr/lib/python2.7/dist-packages/gi/overrides/Gtk.py", line 732, in __getitem__
    raise IndexError("could not find tree path '%s'" % key)
IndexError: could not find tree path '201'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/defer/__init__.py", line 474, in _inline_callbacks
    result = gen.send(result)
  File "/usr/share/software-center/softwarecenter/backend/installbackend_impl/aptd.py", line 505, in add_repo_add_key_and_install_app
    app))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 26: ordinal not in range(128)

[Michael Vogt (mvo)]

This looks very similar to bug #871088 but that got fixed in 5.0 (r2443.2.25) so it either a different incarnation or a regression.

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-center - 5.3.2

---------------
software-center (5.3.2) quantal-proposed; urgency=low

  [ Gary Lasker ]
  * lp:~gary-lasker/software-center/recommendations-string-change-lp986437-for-5.2
    - cherry pick string change LP: #986437

  [ Natalia Bidart ]
  * lp:~nataliabidart/software-center/lost-in-translation:
    - Pass the missing "self" argument to the super(ish) call to fix
      crash in the installed pane

  [ Michael Vogt ]
  * merge fixes from the 5.2 branch:
    - LP: #873104, #1003954, #967036, #920741
 -- Michael Vogt <email address hidden> Thu, 28 Jun 2012 11:12:02 +0200

[Brian Murray (brian-murray)]

Hello Gary, or anyone else affected,

Accepted software-center into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/software-center/5.2.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Gary Lasker (gary-lasker)]

I have verified that software-center version 5.2.4 in precise-proposed fixes this bug per the test case given in the description.

Thanks!

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-center - 5.2.4

---------------
software-center (5.2.4) precise-proposed; urgency=low

  [ Michael Vogt ]
  * lp:~mvo/software-center/lp970627:
    - fix intermittent crash when expunging the cache (LP: #970627)
  * lp:~mvo/software-center/lp971776:
    - fix crash when a downloaded image is invalid for whatever
      reason (network issues, proxy issues, pay-wall in between)
      (LP: #971776)
  * lp:~mvo/software-center/lp967036-2:
    - fix unicode error crash for people running with the
      LANGUAGE=zh_CN.UTF-8 or zh_TW.UTF-8 (LP: #967036)
  * lp:~mvo/software-center/cookie-jar:
    - fix paypal support in the purchases window (LP: #1018347)

  [ Gary Lasker ]
  * debian/control:
    - update vcs link to point to the 5.2 branch
  * lp:~gary-lasker/software-center/previous-purchase-sorting-lp873104:
    - return correct results when sorting the list of previous
      purchases (LP: #873104)
  * lp:~gary-lasker/software-center/recommendations-string-change-lp986437-for-5.2:
    - switch to the updated recommendations opt-in string now that the
      corresponding language pack updates are released and available
      (LP: #986437)
  * lp:~gary-lasker/software-center/fix-lp920741:
    - fix UnicodeDecodeError when a commercial app's title contains
      a unicode character (LP: #920741)

  [ Natalia Bidart ]
  * lp:~nataliabidart/software-center/cant-stop-the-music:
    - Stop the video if user navigates away from an app details page
      (LP: #1003954).
 -- Michael Vogt <email address hidden> Thu, 28 Jun 2012 10:35:55 +0200

[Rolf Leggewie (r0lf)]

oneiric has seen the end of its life and is no longer receiving any updates. Marking the oneiric task for this ticket as "Won't Fix".