Shiretoko user agent string breaks compatibilty with major websites

[Christopher Blay]

Binary package hint: firefox-3.5

I've also read this bug:
 https://bugs.launchpad.net/bugs/394321

I think we can all agree that sites which test for user agent strings are pretty lame but can we also agree that changing an about:config value in Fi^H^HShiretoko to have it report as Firefox (since that's what it really is) would be a huge benefit to users of this package?

Major sites like facebook.com and (probably most) live.com services do use the user agent string to determine compatibility (more than just including a <video> tag).

This is something that would change behind the scenes so it is not a UI change or branding issue: Simply change the default value for general.useragent.extra.firefox to "Firefox/3.5" instead of "Shiretoko/3.5".

WORKAROUND (Will not be overwritten with upgrades):
1. Install User Agent Switcher Addon
2. Create a New User Agent
3. Just Change Shiretoko at the end of the String to Firefox

[Alexander Sack (asac)]

Could you please come up with a list of high impact websites that break with Shiretoko in user-agent, but work with Firefox?

Also be specific about which features are broken and how we can reproduce.

With that info we will be able to understand better if we agree with you on "changing an about:config value in Fi^H^HShiretoko to have it report as Firefox (since that's what it really is) would be a huge benefit to users" :).

[Christopher Blay (blayde)]

http://www.dailymotion.com/openvideodemo has already been brought up in the other bug report. Since open video is one of the big things that's new in Firefox 3.5 it is very confusing to someone who just installed it to be told they need to upgrade. So confusing, in fact, that there *is* a bug report just for this site.

http://www.facebook.com/ is the site I had problems with. Load up home.php when you are signed in and instead of having integrated chat, there is a 'open facebook chat' that also informs you of using an older browser and suggests you update to Firefox.

Both of these work properly when the user-agent is set to "Firefox/3.5"

I tried testing mail.live.com but it's having problems signing me in... Anyways, I'm sure this is enough to justify this simple change

[Christopher Blay (blayde)]

calendar.live.com stops "Shiretoko/3.5" from using its service but allows "Firefox/3.5"

[Christopher Blay (blayde)]

http://www.bing.com/maps/ displays a cryptic message about "Shiretoko/3.5" being unsupported but works fine with "Firefox/3.5"

[Alexander Sack (asac)]

i contacted all websites you mentioned except facebook - for which i couldn't find any form.

[Ryan (ubuntu-draziw)]

If you use a User Agent Switcher https://www.citicards.com works, otherwise the flash over the login box prevents logging in. (But this isn't specific to 3.5....) Workaround for this site is to manually right click to play the flash without looping, then after it plays once you can login. citicards web staff = not so good...

[Christopher Blay (blayde)]

I think we'll find that many banking websites are also silly enough to check user-agent strings. Unfortunately, I don't have accounts at every bank so I can't test every system.

Also with facebook.com not having any way to contact them, the other sites not having fixed their 'compatibility detection', and the inability to check *every* website for this problem perhaps it would be best to make the suggested change?

[Micah Gersten (micahg)]

I contacted the site owner of the Spanish map page in bug 392872.

[Michael R. Crusoe (misterc)]

Vanguard is another high profile site that parses the user agent when it shouldn't. Example: https://personal.vanguard.com/us/accounttypes/advice/ATSAdviceOnAssetMgmtContent.jsp fails to load flash content for "Shiretoko/3.5" variants but works for "Firefox/3.5" variants.

Possible related: https://bugs.launchpad.net/ubuntu/+source/firefox-3.5/+bug/394467 ?

[Micah Gersten (micahg)]

I contacted Vanguard about the User Agent issue.

[Soenke (s0enke)]

@Micah: Sounds like you're not going to change the UA string but contact every website to accept 'Shiretoko' UA-string? What's the benefit?

[will_in_wi (will-in-wi)]

Confirming. Yes this is a bug. Why Ubuntu has decided to modify the default user agent for a final release of Firefox 3.5 and call it by its code name I don't know. I understood the prereleases, but the final? Please fix this, it is so simple. File this under papercuts if you have to.

[Fernando D. (fdbozzo)]

If sites must update the codename every time that is a new version of Firefox or other browsers, it will be a nightmare.
Why should anyone care about codenames? And what is the Agent string meaningless then?

[Micah Gersten (micahg)]

@Everyone
This problem is bigger than just Ubuntu labeling Firefox 3.5 as Shiretoko. Anyone can take the source of Firefox and brand their own browser and that shouldn't make a difference on the web landscape. Debian does it, they have Iceweasel. Websites do not need to update their code every time a new user agent string is created. Rather, they need to use a better detection system to choose which browser is actually being used and support it.

The problem is the attitude of the larger sites on the web that Windows and/or IE are all that exist, maybe if you're lucky OS X.

Attached is the letter from Vanguard in response to my request to update their code to recognize browsers by capability and not User Agent.

This is not the place to discuss this at length. I will post tomorrow a location that we can discuss this further.

[Michael R. Crusoe (misterc)]

@Micah

Thank you for your work on this bug.

However, I must disagree with you. Being that Mozilla does not ship with the codename in the user agent string Ubuntu should make sure it has a good reason to deviate from upstream.

I think we all agree that web developers should code for functionality and not user-agent string sniffing but that battle is outside the scope of this bug.

What is the value add of including the code name of the release when the user-agent string also has the version number? Debian choosing to make a certain decision is interesting but insufficient in itself for Ubuntu to follow -- isn't that the point of Ubuntu that different decisions are made to be more pragmatic in regards to the needs of a typical user?

In summary: "Shiretoko/3.5" adds no value and leads to a loss of functionality. In the spirit of pragmatism and simplicity Ubuntu should ship Firefox 3.5 in a manner consistent with upstream.

[Piotr Szczepanik (piter75)]

I agree with Michael on that.

The only reason I can think of why Firefox 3.5 is Shiretoko in Ubuntu is because Ubuntu is awaiting for the approval of Mozilla Foundation to use the branding because of the modification it made to the code of Firefox. If that's so then it should be told as the resolution of this bug and as soon as we get the approval the User-Agent string should be changed.

Debian is of course having it's Iceweasel but as far as I remember Ubuntu was standing firm with using Firefox brand and I think it was a good choice and it should stay like that because Ubuntu's target user base is different than Debian's.

On the other hand if we want to make people agree and believe that Ubuntu is "Linux for human beings" then we cannot make life of these human beings harder by trying to fix things that cannot be fixed by us. After you contact the site owner - what do you think they are changing in their website? Do they stop depending on the user-agent string? I believe that most of the time they just add a quirk to their browser recognition code to treat Shiretoko/3.5 as Firefox/3.5.

[NoBugs! (luke32j)]

Is there any way to fix the useragent permanently, instead of setting it with the user agent switcher extension every time?

[Christopher Blay (blayde)]

NoBugs, you can change the value permanently by making the change I originally suggested:

1) Type 'about:config' into the address bar of a new tab
2) Say you 'are sure' or whatever it asks you if it does
3) Type 'general.useragent.extra.firefox' into the search bar on the page
4) Double click "Shiretoko/3.5" and a modal dialogue appears where you can change it to "Firefox/3.5"
5) Close the 'about:config' tab and you don't have to worry about it anymore :)

Funny thing is Ubuntu is supposed to be very easy for an 'average human being' to use but average human beings don't know to do this and don't know why their 'new and improved' browser is unsupported by many of their regularly visited websites :(

[NoBugs! (luke32j)]

I had already tried changing that about:config general.useragent.extra.firefox but it keeps switching back.

[NoBugs! (luke32j)]

Never mind, it was an old incompatible extension messing it up.
Works great now, thanks.

[Kristoffer Lundén (kristoffer-lunden)]

The Swedish bank "Swedbank" (at swedbank.se) displays a scary warning when using Shiretoko, but otherwise lets the user move on (which is kind of a sane approach, it's after all about my money here, not some chat or video). This bank is otherwise great in that it officially supports Linux and Firefox, but they can't be expected to know that Ubuntu has special user-agents...

[jflaming (jay-flaming)]

Another one: comcast.net displays warnings about outdated browsers, and makes you click through a bunch of pages to get to real content.

[Dale Jefferson (dalejefferson-deactivatedaccount-deactivatedaccount)]

http://maps.google.com Google maps my location does not work without changing the user sting to Firefox. Its a cool feature and until I came across this bug I could not get it to work!

[pauliephonic (pauliephonic)]

Before I begin, I will echo what others have said, in that sites that restrict access to certain user-agents are mostly wrong in doing so.

I can also fully understand Ubuntu changing the Firefox application name and branding to Shiretoko, so as differentiate it on a users system, and to allow both to be installed side-by-side.

I do not however, understand the reasoning behind changing the user agent.

The user agent is a valid method of identifying a browsers capabilities to visited websites. Some web-servers and javascript libraries rely on the user-agent to handles how content is rendered and to deal with browser quirks.

The only valid reason I can think of to change the user-agent of what is, in essence, standard Firefox 3.5, would be if the functionality were different, or the rendering engine rendered differently. I don't think that this is the case.

This decision, adds unnecessary difficulties for end users, for no technical reason I can think of (I'll happily stand corrected if I am wrong though :)

Cheers and keep up the good work. P

[Daniel Pope (djpope)]

There is a workaround on the forums that fixes this problem (plus the rest of the branding too):

http://ubuntuforums.org/showthread.php?t=1225754

[Christopher Blay (blayde)]

These sites are still effected by this bug:
  http://www.facebook.com/ - chat loads in popup and tells you to upgrade
  http://calendar.live.com/ - warns that browser is unsupported and tells you to upgrade
  http://www.bing.com/maps/ - displays cryptic message that browser is unsupported and tells you to upgrade
  http://www.swedbank.se/ - warns that browser is unsupported and tells you to upgrade

These sites seem to have fixed their 'compatibility detection':
  http://www.dailymotion.com/openvideodemo
  https://www.citicards.com/
  https://personal.vanguard.com/
  http://www.comcast.net/

I couldn't test this allegedly broken site because I'm not sure what is wrong:
  http://maps.google.com

Also let's remember the Internet has millions of websites which may or may not be silly enough to check user-agent strings for compatibility. Could we make sure our users have the best experience possible with free software and make this simple change to the configuration of Firefox 3.5?

[Christopher Blay (blayde)]

I forgot to mention this site is also still effected by this bug:
  http://callejero.paginasamarillas.es/vemaps/mapa.asp - icons don't load so user can't do anything

[Edmond Dantes (edantes)]

Google Toobar will not install with Shiretoko/3.5 in the user agent.

http://tools.google.com/firefox/toolbar/FT5/intl/en/index.html

It offers a download of a firefox tarball, but not the xpi for the extension.

Technically this might not be a bug, but such a major annoyance that it turns it into a bona fide bug. Think of the cumulative millions of hours wasted by users all over the world. Most users will not figure it out immediately.

Fix it? Please? Pretty please?

"It's easier to put on slippers than to carpet the whole world"
Senator Stuart Smalley

[Tony_D (dragozet)]

Ok, this "bug" or "feature" has frustrated me too much now to ignore. I know of the whole 'useragent' setting change to make the browser firefox compatible on certain websites, but the frustrating thing is that everytime there is a small update in Synaptic, it grabs the firefox-branding package and resets the useragent.

If we all want free OSes and non-ie browsers to florish/compete, these hickups should not happen. New users to systems like this will be turned off for no good reason.

Does nobody else use http://www.Hotmail.com ? Hotmail might look like it's working, but once you try and send an email, it sends a blank email with a blank subject line. Warning messages also pop up on http://www.pcfinancial.ca/ and a few more that I don't recall. Most Microsoft sites check the browser.

At the very least, stop over writing my 'useragent' settings. Now that is REALLY annoying.

[Harald Albrecht (harald-albrecht)]

Wikipedia with the new (beta) editor toolbar also breaks: it uses a browser switch detecting Firefox, thus failing for Shiretoko.

[Ryan (ubuntu-draziw)]

www.fedex.com after login gives me: http://www.fedex.com/us/browserdetect/help/error.html

[Frank Groeneveld (frankgroeneveld)]

For me live.com email services break. I can type e-mails, but when I send them, the receiver tells me they are empty. (As can also be seen in the Sent folder). It works fine with the firefox user-agent string.

[Christopher Blay (blayde)]

We should probably just close this bug as 'invalid':
 * The maintainers said there would be a backport from Karmic after they updated its branding and all I see is libprotobuf3
 * They obviously don't care about making firefox-3.5 useful for an average user which makes me wonder why they put it in Jaunty's repo to begin with
 * Instead of fixing a non-UI flaw, they would rather attempt to police the Internet

Basically if you want a truly integrated and properly branded Firefox 3.5 you'll have to dist-upgrade to Karmic Alpha 6. Sorry everyone but hopefully they've fixed any crazy dependencies and there aren't any bad drivers that could brick your hardware. I don't really know what else we can do.

[Frank Groeneveld (frankgroeneveld)]

Or you should switch to windows apparently :S
This is not the way to fix bug #1!

[yareckon (yareckon)]

Not all web browsers have the same capabilities. When dealing with widely different classes of browsers, say mobile versus desktop, sniffing the useragent string is a reasonable way to get information from the client to deal with issues like screen sizes etc. Libraries like WURFL ensure that these days, this is not a suicidal treadmill of work, as long as browsers with known capabilities don't change their useragent strings for no reason*. This creates more work for everyone.

It's certainly nuts to be relying heavily on useragent sniffing for anything as fundamental as delivering text content to desktop browsers these days, but the alternative is javascript or even worse in some cases where one has to build richly interactive content that 1) validates and 2) sings and or dances and 3) looks great. Many times delivering different markup to different browsers is not needed, but occasionally it is. The HTML5 transition especially is a time to be explicit about browser capabilities, unless you don't care about page weight for all of the alternative markup / workarounds / hacks you're going to be delivered, or missing the content you're going to see the fallback version of.

Not consistently identifying clients capable of using this new tech is going to keep people from implementing new stuff on the server. Not everyone can upgrade their browsers. If new browsers identify themselves consistently, the server guys can move forward with you without dumping corporate America or that old library computer, or last year's shiny mobile phone. Again 95% of the time, there should be no need to do this, but serving HTML5 or SVG for instance to a mobile browser from 2005 is bad news. Oh, you say we should wrap those nice new elements in javascript to prevent them from displaying to the wrong browsers? Wait, what was the point of plain HTML again?

*I certainly understand if this is a legal issue (trademark) or an ideological one (web standards maybe). However, at the moment, whoever is in charge of this part of the ranch is giving the impression of total unaccountability to users as time passes without an explanation or fix. Either tell us to get lost, tell us you don't have time, explain why you want to fix the internet first, etc... Community - Communication = Failure

[klems (klemsfree)]

http://www.google.fr sees my pc using shiretoko as a mobil phone :/

[Endolith (endolith)]

Breaks WikiAnswers editing, too.

Smooth move, guys. People will definitely be migrating from Windows to Ubuntu with this kind of forethought.

[sirald66 (sirald66)]

http://abc.go.com/watch/flashforward/235637/238333/black-swan

[Mark Curtis (merkinman)]

sirald66 the ABC problem isn't caused by the user agent showing Shiretoko. ABC Doesn't support Linux at all:
http://abc.go.com/vp2/help
 "Linux is currently not supported. Please make sure to check back for any change in compatibility."

As for this bug? When is it just going to be marked "WONTFIX"? Less than two weeks from now the 'workaround' will be to upgrade to Karmic anyway.

[Christopher Blay (blayde)]

closing as invalid since i should hope we can all upgrade to karmic now

thank you ubuntu for leaving us all in the lurch for about 3 months - that's only what, half a release cycle?