For-purchase apps not showing up on Oneiric

Bug #938736 reported by Michael Nelson on 2012-02-22
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
software-center (Ubuntu)
High
Unassigned
Oneiric
High
Unassigned
Precise
High
Unassigned

Bug Description

TEST CASE:
- there is no really good test case except for a regression test, the issue happens with a certain ~/.cache/software-center/scaclient file only that was updated at a specific (broken) time and never times out so we need to expunge it manually

There have been reports recently (from jpugh, davmor2 and a dev) of for-purchase apps not displaying in Software Center (the most recent being https://apps.ubuntu.com/cat/applications/oneiric/nanoparticles/ ).

Apparently it was discussed recently (I wasn't involved) and diagnosed as some type of cache issue, but I'll let those in the know update this description with something more useful.

Related branches

John Pugh (jpugh) wrote :
Download full text (3.2 KiB)

>> The issue is that we have 3 applications (all PDFs) that show up fine in
>> apps.ubuntu.com.
>>
>> https://apps.ubuntu.com/cat/applications/oneiric/fullcircle-issue-57/
>> https://apps.ubuntu.com/cat/applications/oneiric/fullcircle-it-spec-3/
>> https://apps.ubuntu.com/cat/applications/oneiric/fullcircle-it-issue-56/
>>
>> However all three of these cannot be found in the Software Center client
>> (5.04) in Oneiric.
>>
>> Michael V is not able to reproduce and I believe that Anthony is unable
>> to reproduce the problem either. However the developer, Dave M and I can
>> reproduce it.
> There is a error when the client tries to update the data from the
> software-center agent server. With the help of you and Dave I found
> that the cache of httplib2 contains the follwing in e.g.:
> ~/.cache/software-center/scaclient/software-center.ubuntu.com,api,2.0,applications,en,ubuntu,oneiric,i386,,d8cf7fa67c82edc14789f34f01dad0df
>
> """
> content-location: https://software-center.ubuntu.com/api/2.0/applications/en/ubuntu/oneiric/i386/
> -x-permanent-redirect-url:
> http://software-center.ubuntu.com/api/2.0/applications/en/ubuntu/oneiric/i386/
> """
> This causes the httplib2 code to immediately raise a error.
>
> As a workaround we need to remove the "-x-permanent-redirect-url" from
> the servers response and then figure out if/how to fix httplib2.
- At first I thought it was httplib2 that was storing the https and
  http responses under the same file, but then I noticed that the cache
  filename includes an md5 sum of the complete url, including the
  scheme.
- Confirmed by looking at a dump of davmor's cache, that there were
  *two* files starting with
  software-center.ubuntu.com,api,2.0,applications,en,ubuntu,oneiric,i386
  (jpugh, could you confirm that you have two as well?), one for http
  and one for https, and they're redirecting the browser between each
  other, causing the infinite loop.
- As mentioned in my previous email, requesting available apps via https
  bounces you over to http, but requesting them via http returns the
  actual list of apps, so this didn't add up with the contents of
  davmor's cache.
- With RT #50628 we added a squid cache in front of our 2.0 API. Until
  then, it seems the list of apps was being served over http*s*, and
  requesting over http would bounce you over (exactly opposite to how
  it is now. This was a bug that was fixed during this RT.
- This RT was set up on production on February 2. Checking the dates in
  the responses in davmor's cache, the redirect from http->https is from
  December (before the RT), and the redirect for https->http is from
  February 8 (after the RT), so that makes sense.
- The thing with 301 responses is that they *never* get expired by
  httplib2's cache. They're permanent redirects, so that's technically
  the correct thing to do.

So I don't know how many users will be affected by this, but it seems
like the way to work around this would be to either:
a) Modify httplib2 a bit to expire permanent redirect responses from the
cache every once in a while, or
b) zap the entire cache and start over every once in a while.

Option a) would be tidier I think. As for a short term ...

Read more...

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in software-center (Ubuntu):
status: New → Confirmed
Michael Vogt (mvo) on 2012-03-05
Changed in software-center (Ubuntu Oneiric):
status: New → In Progress
Changed in software-center (Ubuntu Precise):
status: Confirmed → In Progress
Changed in software-center (Ubuntu Oneiric):
importance: Undecided → High
Changed in software-center (Ubuntu Precise):
importance: Undecided → High
description: updated
Michael Vogt (mvo) wrote :

This is fixed in precise with the following commits:
...
revno: 2830
committer: Gary Lasker <email address hidden>
branch nick: trunk
timestamp: Thu 2012-03-08 12:35:19 -0500
message:
  * lp:~mvo/software-center/top-rated-refresh:
  * lp:~mvo/software-center/apptreeview-tweaks:
    - fix 'load_range" errors when navigating the installed view, and
      do some nice needed refactoring/cleanup
...
revno: 2597 [merge]
fixes bug(s): https://launchpad.net/bugs/803028 https://launchpad.net/bugs/88669
8
committer: Michael Vogt <email address hidden>
branch nick: software-center
timestamp: Tue 2011-12-13 17:41:50 +0100
message:
  * lp:~gary-lasker/software-center/catalog-published-date-lp803028:
    - add date_published value from the software-center-agent server
      for use with cataloged time so that for-purchase items appear
      correctly and in the proper order in What's New; include unit
      tests for the new functionality (LP: #803028, LP: #886698
...

Changed in software-center (Ubuntu Precise):
status: In Progress → Fix Released

Hello Michael, or anyone else affected,

Accepted software-center into oneiric-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in software-center (Ubuntu Oneiric):
status: In Progress → Fix Committed
tags: added: verification-needed
James Westby (james-w) wrote :

I had the caching bug, and opening s-c twice with the fixed version allows
me to see the apps again.

My understanding is that having to open twice is expected.

Thanks,

James

tags: added: verification-done
removed: verification-needed
Michael Vogt (mvo) wrote :

Thanks for the verification James. You are correct the fix as we have it currently will only work on the second open of software-center.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package software-center - 5.0.6

---------------
software-center (5.0.6) oneiric-proposed; urgency=low

  [ Gary Lasker ]
  * lp:~gary-lasker/software-center/catalog-published-date-lp803028i-for-5.0:
    - use date_published value from the software-center-agent server
      so that for-purchase items appear correctly and in the proper order
      in What's New; include corresponding unit test (LP: #803028)

  [ Michael Vogt ]
  * backport fix from lp:~mvo/software-center/fix-refresh-of-whats-new
    to ensure that new apps from e.g. the software-center-agent are
    immediately displayed in the "Whats new" section of the lobby
    (LP: #949725)

software-center (5.0.5) oneiric-proposed; urgency=low

  [ Gary Lasker ]
  * lp:~gary-lasker/software-center/fix-lp913756-for-5.0:
    - do not offer to add an icon to the Unity launcher for packages
      that do not have an Exec entry in their corresponding desktop
      file, e.g. ubuntu-restricted-extras, wine (LP: #913756)
  * lp:~gary-lasker/software-center/staging-certs-2-for-5.0:
    - add SOFTWARE_CENTER_FORCE_DISABLE_CERTS_CHECK to allow QA easier
      testing (LP: #918746)

  [ Michael Vogt ]
  * lp:~mvo/software-center/fix-server-pagination:
    - reset server side review "page" when switching to a different
      app
  * lp:~mvo/software-center/expunge-cache-5.0:
    - fix invalid httplib cache entries (LP: #938736)

  [ Kiwinote ]
  * grab exhibits for the current series only (LP: #899257)
 -- Michael Vogt <email address hidden> Thu, 08 Mar 2012 09:07:20 +0100

Changed in software-center (Ubuntu Oneiric):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers