User agent doesn't include Ubuntu in it so apt.ubuntu.com doesn't work

Bug #709125 reported by Jean-Baptiste Lallement on 2011-01-28
78
This bug affects 10 people
Affects Status Importance Assigned to Milestone
AptUrlRedirector
Critical
Marco Ceppi
Ubufox Extension
Undecided
Unassigned
ubufox (Ubuntu)
High
Chris Coulson
Natty
High
Chris Coulson

Bug Description

Binary package hint: software-center

With Firefox 4.0~b10+build1+nobinonly-0ubuntu2 the test case sc-001 Follow the link fails and the web page displays the following message:
=====
You don't seem to be running Ubuntu
=====

It works correctly with Chromium.

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: software-center 3.1.10
ProcVersionSignature: Ubuntu 2.6.37-12.26-generic 2.6.37
Uname: Linux 2.6.37-12-generic i686
Architecture: i386
Date: Fri Jan 28 10:27:31 2011
EcryptfsInUse: Yes
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_US:en
 PATH=(custom, user)
 LANG=en_US.utf8
 LC_MESSAGES=en_US.utf8
 SHELL=/bin/bash
SourcePackage: software-center

Jean-Baptiste Lallement (jibel) wrote :
summary: - Test Case "sc-01 - Follow the link" fails with Firefox 4.0 Beta 9
+ Test Case "sc-001 - Follow the link" fails with Firefox 4.0 Beta 10

Isn't this a bug with apt.ubuntu.com and not software center itself?

Brian Murray (brian-murray) wrote :

It seems to a bug with Firefox as the HTTP_USER_AGENT doesn't indicate Ubuntu at all. From the logs on apt.ubuntu.com:

[28/Jan/2011:22:39:50 +0000] "GET /p/python-wxtools HTTP/1.0" 200 343 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:2.0b10) Gecko/20100101 Firefox/4.0b10"

affects: software-center (Ubuntu) → firefox (Ubuntu)
Brian Murray (brian-murray) wrote :

You can test this yourself at: http://whatsmyuseragent.com/

summary: - Test Case "sc-001 - Follow the link" fails with Firefox 4.0 Beta 10
+ User agent doesn't include Ubuntu in it so apt.ubuntu.com doesn't work
Changed in firefox (Ubuntu):
importance: Undecided → High
Brian Murray (brian-murray) wrote :

For what it is worth about:config says this though:

general.useragent.vendor;Ubuntu

Jean-Baptiste Lallement (jibel) wrote :

I confirm the investigation from Brian, and get the same usergent here.

Changed in firefox (Ubuntu Natty):
status: New → Confirmed
Chris Coulson (chrisccoulson) wrote :

The general.useragent.vendor* preferences weren't really supported, and have been dropped entirely as part of the UA string refactoring (see http://blog.mozilla.com/dwitte/2010/08/24/user-agent-string-changes-coming-in-firefox-4/). There is currently no mechanism for adding the vendor name.

The vendor details at the end of our UA string has also recently been a source of bugs where websites don't render correctly on Ubuntu, but do work correctly using upstream Mozilla builds.

I'm not sure of any way around this yet. "navigator.vendor" is also just a stub in Firefox 4, and returns an empty string if you query it with JS.

Changed in firefox (Ubuntu Natty):
milestone: none → natty-alpha-3
Chris Coulson (chrisccoulson) wrote :

I've spoken to somebody from Mozilla, and this won't be fixed by adding vendor information back to the user agent string (see https://bugzilla.mozilla.org/show_bug.cgi?id=572650 for some idea of why this has changed, and this is not the sort of thing we'll be vendor patching).

In addition to that, as I pointed out above - the vendor identification in the UA string has been a constant source of problems where websites misdetect the browser on Ubuntu and display incorrect content (ie, mobile content).

At best, I can probably add an extra header to the HTTP packets when communicating with apt.ubuntu.com (and this is what has been recommended to me) - but this would also need a server-side fix too.

Brian Murray (brian-murray) wrote :

I'm not wedded to the idea of having this fixed in Firefox, the right thing may be to fix apt.ubuntu.com, but it still should be sorted out.

Greg A (etulfetulf) on 2011-02-22
Changed in apturlredirector:
status: New → Confirmed
importance: Undecided → Critical
assignee: nobody → Marco Ceppi (marco-ceppi)
Marco Ceppi (marcoceppi) wrote :

There's a similar issue going on with Google Chrome at the moment. As simple fix has been included in the apturlredirector project but I'm open to more elaborate ideas (like taking in extra headers, etc).

Marco Ceppi (marcoceppi) on 2011-02-22
Changed in apturlredirector:
milestone: none → 1.1
Martin Pitt (pitti) wrote :

Chris, Michael, is this an apt or firefox bug? Who should this be assigned to? Thanks!

Changed in firefox (Ubuntu Natty):
assignee: nobody → Chris Coulson (chrisccoulson)
Marco Ceppi (marcoceppi) wrote :

Would creating a small extension that's just included in the default installation possibly be "the best" solution for this? Something that can be picked up by a webpage to say "Oh, this is an Ubuntu Machine"

Martin Pitt (pitti) on 2011-03-01
Changed in firefox (Ubuntu Natty):
milestone: natty-alpha-3 → ubuntu-11.04-beta-1
Stefan Nuxoll (snuxoll) wrote :

What is the rationale behind denying UA's without Ubuntu in them access to apt.ubuntu.com? I ask because it would honestly make more sense to get rid of the UA sniffing, Mozilla's change in FF4 is something that I agree with, the less information in a UA string the better, we shouldn't be encouraging browser fingerprinting and wasting TCP packets to send the word "Ubuntu" in the HTTP request. It reeks of the same issue we as an open source community have been fighting against webapps that sniff the UA and deny us access because we're not running Windows or OS X, even though it uses nothing more than Javascript, HTML and CSS and will run fine on any browser.

If we are going to include an extension because there is a valid reason to do UA sniffing on apt.ubuntu.com (I can't think of any myself, so please do tell me the rationale) I think we should make sure it only sends the extra UA part on ubuntu.com URL's, if we are going to make Ubuntu users more identifiable we should limit only to Ubuntu services.

Martin Pitt (pitti) wrote :

As discussed above, this won't be fixed in Firefox.

I agree that apt.ubuntu.com should just drop the UA filter. There is little reason to deny access to non-Ubuntu hosts, after all? Non-Ubuntu OSes won't have apturl and other .deb handlers installed, so it won't do them any harm?

Changed in firefox (Ubuntu Natty):
assignee: Chris Coulson (chrisccoulson) → nobody
milestone: ubuntu-11.04-beta-1 → none
status: Confirmed → Won't Fix
Changed in firefox (Ubuntu):
status: Confirmed → Won't Fix
assignee: Chris Coulson (chrisccoulson) → nobody
Matthew Paul Thomas (mpt) wrote :

apt.ubuntu.com does not "deny access" to non-Ubuntu clients, and nobody has suggested that it should. That's not the issue. The issue is that it has to be able to display OS-dependent information: an installation button for the relevant package if you're running Ubuntu, or if you're running something else, information about Ubuntu and how to get it.

It's possible to tell whether a Web browser is running on Windows or not, or on Mac OS X or not. If Firefox on Ubuntu makes it impossible for us to tell whether it's running on Ubuntu or not, that's putting Ubuntu at an unfair disadvantage.

Chris Coulson (chrisccoulson) wrote :

Well, it's still possible to differentiate an Ubuntu machine from Mac OS X and Windows, it just isn't possible to differentiate it from another Linux machine.

In any case, this isn't going to be changed in Firefox (https://bugzilla.mozilla.org/show_bug.cgi?id=572650 details the reasoning behind this, and this isn't something we are going to support indefinitely as a distro-patch, and I can't see Mozilla ever agreeing to us reverting this change either). In addition to this, as I pointed out above, having "Ubuntu" in the UA string has been a long-standing source of problems such as this: http://askubuntu.com/questions/16014/web-sites-see-my-firefox-as-a-mobile-browser-on-ubuntu (and plenty of bugs reported on Launchpad too).

Matthew Paul Thomas (mpt) wrote :

We can't reasonably assume that every Linux user is using Ubuntu. Market-share-wise, that would be roughly equivalent to assuming that every browser user is using Internet Explorer. :-)

https://bugzilla.mozilla.org/show_bug.cgi?id=572650 describes a worthy goal, but nothing about that requires treating Ubuntu *differently* from other OSes. As long as it is possible to distinguish Windows and Mac OS X users by the UA string, it should be possible to distinguish Ubuntu users too.

Even if naming the kernel in the UA string is important (I don't know why, but maybe it is), Firefox should treat Ubuntu exactly the same as it treats Android and Maemo, which also use Linux as their kernel: that is, have "(Ubuntu; Linux;...)" just like it has "(Android; Linux;...)" and "(Maemo; Linux;...)".

Jorge Castro (jorge) wrote :

mpt,

I think being able to have apt.u.c working for people who use other browsers is a lesser evil.

Sure we lose the chance to tell people about ubuntu and how to get it, but if they're on another OS it won't do them any harm, and in return we get to finally help people fix the "I can't install things right" problem, which is a bigger issue.

Marco Ceppi (marcoceppi) on 2011-03-26
Changed in apturlredirector:
status: Confirmed → In Progress
Chris Coulson (chrisccoulson) wrote :

Just to be clear, I fully intend to add support to ubufox for adding an extra header to the HTTP requests for certain websites (eg, apt.ubuntu.com), which make it possible for those sites to identify an Ubuntu machine. However, this will be restricted only to sites where we really, really need it (it won't be sent out in every request), and it would still require a server-side fix in apt.ubuntu.com too (to detect the new header).

The header will probably be called something like X-Ubuntu, and have a value equal to the release version (eg, 11.04). I'm not sure if we need any extra information.

Marco Ceppi (marcoceppi) wrote :

I've included a design change to the APTURLRedirector project which will bypass the "Switch to Ubuntu" screen if it's detected from the UA that the machine is linux and attempt to launch the apt link. After 5 seconds additional text will display informing the user that you need Ubuntu for this to work and provide the link to learn more. This fix has been committed. I would love to implement the older functionality (Possibly after Chris implements sending the additional header to *.ubuntu.com) for now this change ensures that Firefox users won't need to incur an additional step trying to get packages from apt.ubuntu.com

Changed in apturlredirector:
status: In Progress → Fix Committed
Chris Coulson (chrisccoulson) wrote :

I've fixed this in ubufox now - http://bazaar.launchpad.net/~mozillateam/ubufox/trunk/revision/225

Basically, requests to apt.ubuntu.com now get an extra header (X-Ubuntu=11.04)

affects: firefox (Ubuntu Natty) → ubufox (Ubuntu Natty)
Changed in ubufox (Ubuntu Natty):
assignee: nobody → Chris Coulson (chrisccoulson)
milestone: none → ubuntu-11.04-beta-2
status: Won't Fix → Triaged
Changed in ubufox:
status: New → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubufox - 0.9-0ubuntu1

---------------
ubufox (0.9-0ubuntu1) natty; urgency=low

  * New upstream release v0.9
  * All patches are merged in to the upstream branch now

  * Fix LP: #750305 - Please update links and configs for Natty release.
    Rather than doing this manually for every release, we add a dist.js with
    distro information (populated at build time with lsb_release) and just
    do all this automatically.
  * Fix LP: #752364 - Plugin Finder Service never finds anything. Use the
    release info in dist.js to construct the URL, now that this got dropped
    from Firefox
  * Fix LP: #728826 - Update icons to new versions.
  * Drop the pluginGlue.js module. This didn't work, as we can no longer
    use a modules registerSelf method to use nsICategoryManager at startup,
    due to the XPCOM registration changes in Firefox 4. Instead, add
    category entries with the chrome.manifest
  * Fix LP: #709125 - User agent doesn't include Ubuntu in it so
    apt.ubuntu.com doesn't work. We add an extra X-Ubuntu HTTP header in
    requests to apt.ubuntu.com, rather than send this information out in
    every request in the UA string

  * Add build-depend on lsb-release so we can create dist.js
 -- Chris Coulson <email address hidden> Sat, 09 Apr 2011 00:08:51 +0100

Changed in ubufox (Ubuntu Natty):
status: Triaged → Fix Released
Changed in ubufox:
status: Fix Committed → Fix Released
Jeremy Bicha (jbicha) wrote :

This still doesn't work. Has it been fixed at apt.ubuntu.com yet?

Greg A (etulfetulf) wrote :

No.

Brian Murray (brian-murray) wrote :

It has been fixed at apt.ubuntu.com now as the user agent checking has been removed altogether until the new version of apturlredirector is installed.

Chris Coulson (chrisccoulson) wrote :

Do I need to keep the X-Ubuntu header in the long term then? Or will apt.ubuntu.com not check this in the future?

Brian Murray (brian-murray) wrote :

I'm really not clear on how the apturldirector project works but I'm under the impression it is a community project hosted on Canonical servers and that IS had some concerns with the new version of it. However, as I understand it were if those concerns addressed then the new version would be installed and it'd be looking for the X-Ubuntu header.

Marco Ceppi (marcoceppi) wrote :

Being the person who wrote the APTURLRedirector Project code - I haven't heard of any issues from Canonical IS team. As of right now it doesn't process or look for the X-Ubuntu header, but could if need be. Currently Firefox would be the only browser that supports it - so adaptations to Chrome/Chromium would need to be made to send the X-Ubuntu header for it to be a holistic solution. That being said if Canonical IS team would like to communicate any issues with me they can do so via bug reports or email. I would love to have them addressed immediately.

Marco Ceppi (marcoceppi) on 2011-08-02
Changed in apturlredirector:
status: Fix Committed → Fix Released

@chris: I've just checked for the change you landed with r255, and see it must have been removed at some point.

Is there a way we can have this included for apps.ubuntu.com and software-center.ubuntu.com (or *.ubuntu.com so we can test the functionality on our staging servers too)? Currently we can only provide useful info to Chromium users.

Chris Coulson (chrisccoulson) wrote :

Yes, we could probably add it back for that. Are you ok with the version being in an extra http header like before?

Matthew Paul Thomas (mpt) wrote :

Chris, I think it would be easier for everyone using the string (including those who collect OS statistics, like Wikimedia) if it was in the user-agent string itself.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.