software-center crashed with UnicodeDecodeError in get_label(): 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128) for apps with hardware requirements

[mastaiza]

[TEST CASE]
1. install chinese support via gnome-language-selector
2. run LANGUAGE=zh_CN.UTF-8 software-center ryzom
3. verify the crash (and that the UI looks chinese)
4. use software-center from precise-proposed (once its there)
5. run LANGUAGE=zh_CN.UTF-8 software-center ryzom
6. verify that it works (and that all UI looks chinese)

[REGRESSION POTENTIAL]
Extremely low, this fix is a simple unicode conversion.

ProblemType: Crash
DistroRelease: Ubuntu 12.04
Package: software-center 5.1.13.2
ProcVersionSignature: Ubuntu 3.2.0-20.33-generic 3.2.12
Uname: Linux 3.2.0-20-generic i686
NonfreeKernelModules: nvidia
ApportVersion: 1.95-0ubuntu1
Architecture: i386
CrashCounter: 1
Date: Wed Mar 28 18:25:54 2012
ExecutablePath: /usr/share/software-center/software-center
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Beta i386 (20110901)
InterpreterPath: /usr/bin/python2.7
PackageArchitecture: all
ProcCmdline: /usr/bin/python /usr/bin/software-center
PythonArgs: ['/usr/bin/software-center']
SourcePackage: software-center
Title: software-center crashed with UnicodeDecodeError in get_label(): 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)
UpgradeStatus: Upgraded to precise on 2012-01-15 (72 days ago)
UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[zubozrout (zubozrout)]

Same on Intel 64bit processor.

[Bernard Banko (beernarrd)]

With todays updates I have repeteadely got a lot of crashes: software center, compiz, ocrfeed, tvtube. Hard to say more or wheather they are linked or not.

[Bernard Banko (beernarrd)]

Apport complains about old libraries so it cannot report (libatk1.0-0, libatk1.0-data, libglib2.0-0, libgs9-common, libpango1.0-0)

[Gary Lasker (gary-lasker)]

Hi Bernard, thanks for your info. This particular bug is not yet fix-released, so it may occur with current updates.

Beta 2 of Precise Pangolin was released just hours ago. I'd suggest doing a full update in a couple of hours and hopefully the other crashes that you are seeing will settle down.

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-center - 5.1.14

---------------
software-center (5.1.14) precise; urgency=low

  [ Michael Vogt ]
  * lp:~mvo/software-center/lp962580:
    - add locking to the expunge helper process to fix bugs that are triggered
      if multiple expunge cache processes are run (LP: #962580)
  * lp:~mvo/software-center/cache-refresh-glitch:
    - ensure that we get a full refresh if a pkg was not available before
      show_app is called
  * lp:~mvo/software-center/lp940482:
    - fix crash if the debfile does not return proper utf8 for the
      description (LP: #940482)
  * lp:~mvo/software-center/lp966514:
    - properly handle network disconnect conditions with the Ubuntu
      SSO dialog (LP: #966514)
  * lp:~mvo/software-center/lp966879:
    - fix for crashes in the installed view treeview (LP: #966879,
      LP: #950899)
  * lp:~mvo/software-center/lp846204:
    - fix ValueError crashes in get_iter due to invalid tree paths
      (LP: #846204)
  * lp:~mvo/software-center/lp964433:
    - disconnect the model from the view before calling set_from_matches
      (LP: #964433)
  * lp:~mvo/software-center/treeview-keep-state-on-db-cache-change:
   - restore the state of the installed view treeview when the
     the db or cache changes, such as on an app install or remove
  * lp:~mvo/software-center/946393:
   - fix installing multiple apps when in a custom list view (LP: #946393)
  * lp:~mvo/software-center/lp969050:
   - disconnect the view when the model is cleared to avoid a furry of
     cursor_changed signals as the rows get removed (LP: #969050)

  [ Gary Lasker ]
  * lp:~gary-lasker/software-center/installed-pane-refresh:
    - avoid rebuilding the treeview in the installedpane if its not
      required (LP: #828887)
  * lp:~gary-lasker/software-center/fix-crash-lp967036:
    - Small branch to fix a crash due to a UnicodeDecodeError when accessing
      the short description for H/W requirements (LP: #967036)
  * lp:~gary-lasker/software-center/fix-crash-lp935930:
    - fix a crash due to a UnicodeDecodeError (LP: #935930)
 -- Michael Vogt <email address hidden> Fri, 30 Mar 2012 18:00:50 +0200

[Jactry Zeng (jactry)]

Intel 64 bit

[Nicolás Abel Carbone (nicocarbone)]

I am still having this problem. It happens when trying to open the info page of Darwina.

[Stéphane Guillou (stephane-guillou)]

Just got this problem with an up-to-date 12.04 beta while navigating in the software centre.

[David Callé (davidc3)]

Happened when trying to open the page for "Sistema de facturacion e Inventario Akane".

[Gary Lasker (gary-lasker)]

Thanks for reporting back, everyone! Clearly this fix was not sufficient so I'm setting this back to confirmed.

[Michael Vogt (mvo)]

It turns out this is due to str/unicode python2.x issues. The problem is that we get a utf8 encoded string and uses that as input for %s - then python will try to convert it to a string without a encoding and that fails. I added a explicit convertion back to unicode now. The alternative would be to return a unicode translated string in get_hw_missing_long_description() but that is also not great. Fortunately python3 will safe us at some point :)

[mazhar ozdal (local1907)]

software center >search ryzom > details and crashed software center.

http://desmond.imageshack.us/Himg820/scaled.php?server=820&filename=ekrangrnts2012041213413.png&res=landing

Ubuntu 12.04 64bit

[Gary Lasker (gary-lasker)]

Thank you for grabbing this one, Michael!

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-center - 5.2

---------------
software-center (5.2) precise; urgency=low

  [ Michael Vogt ]
  * debian/control:
    - recommend xz-lmza instead of lzma (LP: #878354)
  * lp:~mvo/software-center/track-axi-changes:
    - add a monitor for changes in the apt-xapian-index to ensure
      the DB is reopened if the cron job changes the apt-xapian-index
      database (LP: #507836)
  * lp:~mvo/software-center/lp970342:
    - do not crash if the parent goes away (and therefore the pipe)
      and the token can not be obtained (LP: #970342)
  * lp:~mvo/software-center/lp967036:
    - fix UnicodeDecodeError crash for apps with hardware
      requirements in the details view (LP: #967036)
  * lp:~mvo/software-center/list-a11y:
    - restore the a11y support for orca/accerciser/ldtp for
      the listview and treeview
  * lp:~mvo/software-center/lp979013:
    - fix a bug in the review display for certain language
      environments (LP: #979013)
  * lp:~mvo/software-center/appview-tweaks:
    - code cleanup in the sortmode code
  * lp:~mvo/software-center/lp772549:
    - fix a bug that can result in a blank software center
      screen on startup (LP: #772549)
  * lp:~mvo/software-center/lp914393:
    - adds an explicit sys.exit() to ensure all gtk event
      processing is stopped on application quit (LP: #914393)
  * lp:~mvo/software-center/lp976169:
    - do not crash if apt-xapian-index is not installed
      (LP: #976337)
    - ignore xapian.DatabaseOpeningError here (happens when
      there is no a-x-i so we don't care) (LP: #976169)
  * lp:~mvo/software-center/lp976525:
    - do not show "upgradable" button when there should be "remove"
      (LP: #976525)

  [ Gary Lasker ]
  * lp:~gary-lasker/software-center/sorting-fix-lp969215:
    - fix sort combo box inconsistencies (LP: #969215)

  [ Dave Morley ]
  * lp:~davmor2/software-center/remove_two_lines:
    - small fixes for test script USC-start-stop-times.py

  [ Kiwinote ]
  * sc/ui/gtk3/widgets/apptreeview.py:
    - everyone tries to expand a category in the installed pane by clicking
      on the text rather than the expander arrow (LP: #877130)
 -- Michael Vogt <email address hidden> Fri, 13 Apr 2012 18:35:13 +0200

[Anthony Wong (anthonywong)]

I have to re-open this bug because I still have this problem even though my software-center is 5.2.1, problem occurs when the "More Info" button of Ryzom is pressed when locale is in zh_TW.UTF-8, but does not happen in en_US.UTF-8.

$ dpkg -l software-center | grep software-center
ii software-center 5.2.1 Utility for browsing, installing, and removing software

Traceback (most recent call last):
  File "/usr/share/software-center/softwarecenter/utils.py", line 118, in wrapper
    f(*args, **kwargs)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/panes/availablepane.py", line 751, in on_application_activated
    self.display_details_page)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/session/viewmanager.py", line 190, in display_page
    callback(page, view_state)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/panes/availablepane.py", line 685, in display_details_page
    SoftwarePane.display_details_page(self, page, view_state)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/panes/softwarepane.py", line 464, in display_details_page
    self.app_details_view.show_app(view_state.application)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/views/appdetailsview.py", line 1733, in show_app
    self._update_minimal(self.app_details)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/views/appdetailsview.py", line 1567, in _update_minimal
    self._update_pkg_info_table(app_details)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/views/appdetailsview.py", line 1491, in _update_pkg_info_table
    self.hardware_info.set_value(app_details.hardware_requirements)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/views/appdetailsview.py", line 540, in set_value
    self.value_label.set_hardware_requirements(value)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/widgets/labels.py", line 105, in set_hardware_requirements
    label.set_hardware_requirement(tag, sup)
  File "/usr/share/software-center/softwarecenter/ui/gtk3/widgets/labels.py", line 84, in set_hardware_requirement
    self._label.set_markup(self.get_label())
  File "/usr/share/software-center/softwarecenter/ui/gtk3/widgets/labels.py", line 78, in get_label
    "hardware": unicode(short_descr, "utf8", "ignore")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 19: ordinal not in range(128)

[Michael Vogt (mvo)]

I can reproduce this with LANGUAGE=zh_CN.UTF-8 as well. As this affects a lot of users, setting this to critical.

[Michael Vogt (mvo)]

It turns out this caused by a translation:

The string for the hardware is translated as:

original: '%(sym)s%(hardware)s,'
translation: '%(sym)s%(hardware)s，'

The "," is different in the translation causing the crash.

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-center - 5.3.2

---------------
software-center (5.3.2) quantal-proposed; urgency=low

  [ Gary Lasker ]
  * lp:~gary-lasker/software-center/recommendations-string-change-lp986437-for-5.2
    - cherry pick string change LP: #986437

  [ Natalia Bidart ]
  * lp:~nataliabidart/software-center/lost-in-translation:
    - Pass the missing "self" argument to the super(ish) call to fix
      crash in the installed pane

  [ Michael Vogt ]
  * merge fixes from the 5.2 branch:
    - LP: #873104, #1003954, #967036, #920741
 -- Michael Vogt <email address hidden> Thu, 28 Jun 2012 11:12:02 +0200

[Brian Murray (brian-murray)]

Hello mastaiza, or anyone else affected,

Accepted software-center into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/software-center/5.2.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Anthony Wong (anthonywong)]

Confirmed software-center 5.2.4 in precise-proposed fixes the bug.

[Gary Lasker (gary-lasker)]

I have verified that software-center version 5.2.4 in precise-proposed fixes this bug per the test case given in the description.

Thanks!

[Gary Lasker (gary-lasker)]

Thank you for your verification, Anthony!

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-center - 5.2.4

---------------
software-center (5.2.4) precise-proposed; urgency=low

  [ Michael Vogt ]
  * lp:~mvo/software-center/lp970627:
    - fix intermittent crash when expunging the cache (LP: #970627)
  * lp:~mvo/software-center/lp971776:
    - fix crash when a downloaded image is invalid for whatever
      reason (network issues, proxy issues, pay-wall in between)
      (LP: #971776)
  * lp:~mvo/software-center/lp967036-2:
    - fix unicode error crash for people running with the
      LANGUAGE=zh_CN.UTF-8 or zh_TW.UTF-8 (LP: #967036)
  * lp:~mvo/software-center/cookie-jar:
    - fix paypal support in the purchases window (LP: #1018347)

  [ Gary Lasker ]
  * debian/control:
    - update vcs link to point to the 5.2 branch
  * lp:~gary-lasker/software-center/previous-purchase-sorting-lp873104:
    - return correct results when sorting the list of previous
      purchases (LP: #873104)
  * lp:~gary-lasker/software-center/recommendations-string-change-lp986437-for-5.2:
    - switch to the updated recommendations opt-in string now that the
      corresponding language pack updates are released and available
      (LP: #986437)
  * lp:~gary-lasker/software-center/fix-lp920741:
    - fix UnicodeDecodeError when a commercial app's title contains
      a unicode character (LP: #920741)

  [ Natalia Bidart ]
  * lp:~nataliabidart/software-center/cant-stop-the-music:
    - Stop the video if user navigates away from an app details page
      (LP: #1003954).
 -- Michael Vogt <email address hidden> Thu, 28 Jun 2012 10:35:55 +0200