Lynx cannot access www.google.com (400 error)

[Jason Ribeiro]

When attempting to access www.google.com using Lynx, I receive a "400 Bad
Request" error. AFter doing some searching I came across
http://tux.oclug.on.ca/pipermail/oclug/2004-May/038971.html which basically says
that Lynx is using the information in /etc/mailcap to send a huge Accept: header
and google doesn't like it. I know that this problems occurs for several other
people (we spoke about this issue on #ubuntu one day and several people confirmed).

Fix: In /etc/lynx.cfg, I edited "GLOBAL_MAILCAP:/etc/mailcap" to
"GLOBAL_MAILCAP:/etc/mailcap.lynx" (line 2304 in my file). Then I touched an
empty /etc/mailcap.lynx. After that, I could access www.google.com with no
problems.

I don't know if this is a problem with lynx or with google (since all other
sites I have tried work fine in Lynx). Still, if everything isn't needed in
/etc/mailcap for lynx to work it should get it's own custom shorter one (or
maybe filter /etc/mailcap so it doesn't send a huge Accept: header). Thanks.

[Thomas Dickey (dickey)]

On Tue, Jun 15, 2004 at 11:10:11AM +0200, Zakaria wrote:
> Package: lynx
> Version: 2.8.5-1
> Severity: important
>
> Is there something wrong with this version suddenly all my attempt
> to google.com (www, groups, news) result in 400 Bad Request. If I try
> using w3m or links it's fine. I even try using lynx in cygwin and still
> work.

It appears to be due to a recent increase in the size of the mime-types
on Debian. I made changes in lynx 2.8.6dev.4 which address this.

The current version of lynx is 2.8.5

It's available at
 ftp://lynx.isc.org/lynx/lynx2.8.5/
2.8.6 Development & patches:
 http://lynx.isc.org/current/index.html

--
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net

[Dave Beckett (dave-dajobe)]

This bug was annoying me so I've made a patch taken from lynx
2.8.6dev.4 as mentioned by Thomas E. Dickey in the bug report.

I took as little as I could from that version, and in particular
didn't taken any of the configuration support for the Accept: header
management but hard-coded in 'mediaOpt3' which allows internal,
configuration and user's mime-types, but not system ones. The
development version has full user-control over that.

Cheers

Dave

[Jason Ribeiro (jrib)]

When attempting to access www.google.com using Lynx, I receive a "400 Bad
Request" error. AFter doing some searching I came across
http://tux.oclug.on.ca/pipermail/oclug/2004-May/038971.html which basically says
that Lynx is using the information in /etc/mailcap to send a huge Accept: header
and google doesn't like it. I know that this problems occurs for several other
people (we spoke about this issue on #ubuntu one day and several people confirmed).

Fix: In /etc/lynx.cfg, I edited "GLOBAL_MAILCAP:/etc/mailcap" to
"GLOBAL_MAILCAP:/etc/mailcap.lynx" (line 2304 in my file). Then I touched an
empty /etc/mailcap.lynx. After that, I could access www.google.com with no
problems.

I don't know if this is a problem with lynx or with google (since all other
sites I have tried work fine in Lynx). Still, if everything isn't needed in
/etc/mailcap for lynx to work it should get it's own custom shorter one (or
maybe filter /etc/mailcap so it doesn't send a huge Accept: header). Thanks.

[Niels (debbugs)]

Package: lynx
Version: 2.8.5-2sarge1
Followup-For: Bug #254515

I just noticed the same problem: lynx doesn't seem to manage urls like
google.nl and w3m retrieves and shows those same urls.

Also I notice in my proxy logs line like:

1135895297.926 5 127.0.0.0 TCP_DENIED/400 1545 GET %20http://google.nl/ - NONE/- text/html

I do not really know what to make of it, but i hope it helps :)

cheers, niels

-- System Information:
Debian Release: testing/unstable
  APT prefers oldstable
  APT policy: (500, 'oldstable'), (500, 'testing'), (500, 'stable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.6.8-2-386
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

Versions of packages lynx depends on:
ii libbz2-1.0 1.0.2-11 high-quality block-sorting file co
ii libc6 2.3.5-8 GNU C Library: Shared libraries an
ii libgnutls11 1.0.16-14 GNU TLS library - runtime library
ii libncursesw5 5.5-1 Shared libraries for terminal hand
ii zlib1g 1:1.2.3-8 compression library - runtime

Versions of packages lynx recommends:
ii mime-support 3.35-1 MIME files 'mime.types' & 'mailcap

-- no debconf information

[Thomas Dickey (dickey)]

On Thu, Dec 29, 2005 at 11:40:11PM +0100, Niels wrote:
> Package: lynx
> Version: 2.8.5-2sarge1
> Followup-For: Bug #254515
>
> I just noticed the same problem: lynx doesn't seem to manage urls like
> google.nl and w3m retrieves and shows those same urls.

I didn't test the fix copied into this package (am assuming that is not
the problem).

"lynx -trace" should show the header information, and perhaps give enough
context to make sense of the log information.

> Also I notice in my proxy logs line like:
>
> 1135895297.926 5 127.0.0.0 TCP_DENIED/400 1545 GET %20http://google.nl/ - NONE/- text/html
>
> I do not really know what to make of it, but i hope it helps :)

--
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net

[Daniel Robitaille (robitaille)]

I can confirm this bug in Breezy. According to the Debian bug report, this is due to the size of the mime types used with lynx.

But in Dapper, lynx works fine and can access google.com without any problems.

I suspect it's not because the bug has been solved, but simply because the default /etc/mailcap is smaller in Dapper compared to Breezy:

-rw-r--r-- 1 daniel daniel 18167 2006-02-14 22:44 mailcap.breezy
-rw-r--r-- 1 daniel daniel 16021 2006-02-14 22:54 mailcap.dapper

[Wolf Wiegand (wolf-kondancemilch)]

Package: lynx
Version: 2.8.5-2sarge1
Followup-For: Bug #254515

Hi,

this problem is caused by too many "Accept: ..."-headers in the HTML GET
request. The headers sent seem to be derived from entries in
/etc/mailcap and ~/.mailcap. After configuring a stripped down global
mailcap in /etc/lynx.cfg and removing ~/.mailcap, the problems with
displaying google were gone. I don't know if this is google's or
lynx's problem, though.

Regards,

Wolf

-- System Information:
Debian Release: testing/unstable
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/dash
Kernel: Linux 2.6.16
Locale: LANG=C, LC_CTYPE=de_DE@euro (charmap=ISO-8859-15)

Versions of packages lynx depends on:
ii libbz2-1.0 1.0.3-2 high-quality block-sorting file co
ii libc6 2.3.6-15 GNU C Library: Shared libraries
ii libgnutls11 1.0.16-14+b1 GNU TLS library - runtime library
ii libncursesw5 5.5-2 Shared libraries for terminal hand
ii zlib1g 1:1.2.3-11 compression library - runtime

Versions of packages lynx recommends:
ii mime-support 3.36-1 MIME files 'mime.types' & 'mailcap

-- no debconf information

[Benjamin Hawkes-Lewis (benjamin-hawkeslewis)]

I'm using Dapper, and I can't access www.google.com without the workaround described in the first comment.

[Zephaniah E. Hull (warp-aehallh)]

found 254515 2.8.5-2sarge2.1
tags 254515 + patch

stop

This isn't actually fixed, removing duplicates and shadowed mime types
helps a little bit, but it's still quite possible to get past the limit
where google starts giving a 400 error.

A dpatch is attached which, instead of including a list of mime types,
just hands off */* like Mozilla, this solves the privacy leakage as well
as the whole google not working problem.

Zephaniah E. Hull.

--
   1024D/E65A7801 Zephaniah E. Hull <email address hidden>
    92ED 94E4 B1E6 3624 226D 5727 4453 008B E65A 7801
     CCs of replies from mailing lists are requested.

> My kid brother tells me Visual Age for Java is the cat's pajamas

I'm not a cat person, but I can just imagine the reaction of your
average feline to someone's attempt to stuff it into a pair of
pajamas.

Now picture your hard disk after the thing installs.
 -- Berry Kercheval and Graham Reed on ASR.

[Thomas Dickey (dickey-his)]

tag 67184 fixed-upstream
tag 99400 fixed-upstream
tag 132674 fixed-upstream
tag 254515 fixed-upstream
tag 40435 fixed-upstream

--
Thomas E. Dickey <email address hidden>
http://invisible-island.net
ftp://invisible-island.net

[Daniel Robitaille (robitaille)]

This is still a problem in Feisty

[Zephaniah E. Hull (warp)]

Source: lynx
Source-Version: 2.8.6-1

We believe that the bug you reported is fixed in the latest version of
lynx, which is due to be installed in the Debian FTP archive:

lynx_2.8.6-1.diff.gz
  to pool/main/l/lynx/lynx_2.8.6-1.diff.gz
lynx_2.8.6-1.dsc
  to pool/main/l/lynx/lynx_2.8.6-1.dsc
lynx_2.8.6-1_amd64.deb
  to pool/main/l/lynx/lynx_2.8.6-1_amd64.deb
lynx_2.8.6.orig.tar.gz
  to pool/main/l/lynx/lynx_2.8.6.orig.tar.gz

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to <email address hidden>,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Zephaniah E. Hull <email address hidden> (supplier of updated lynx package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing <email address hidden>)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Tue, 01 May 2007 01:43:17 -0400
Source: lynx
Binary: lynx
Architecture: source amd64
Version: 2.8.6-1
Distribution: unstable
Urgency: low
Maintainer: Zephaniah E. Hull <email address hidden>
Changed-By: Zephaniah E. Hull <email address hidden>
Description:
 lynx - Text-mode WWW Browser
Closes: 40435 67184 99400 120451 121520 132674 137480 141158 147287 152810 157088 171312 184482 188415 193205 204994 240237 244871 248092 252915 254515 265031 268264 271048 304989 313789 315853 318034 325478 343049 344275 374388 390918
Changes:
 lynx (2.8.6-1) unstable; urgency=low
 .
   * Hijack the package. I might not be great at it, but I do use it daily.
   * New upstream release.
     Closes: #254515, #137480, #67184, #99400, #132674, #141158, #40435,
     #120451, #157088, #204994, #244871, #248092, #268264, #271048, #318034,
     #343049, #390918, #240237, #313789, #171312, #193205, #252915, #265031,
     #121520, #152810, #188415, #344275, #374388, #184482, #315853
   * Uses the new upstream defaults. Closes: #325478, #147287.
   * Update 01_default-config.dpatch. (Offset changes only.)
   * Update 02_default-key-bindings.dpatch. (Upstream formatting changes.)
   * Kill 03_newer_gnutls.dpatch entirely.
     This was fixed upstream. But this is also a GPL violation as we only ship
     the patch to configure, and not to configure.in, the source file.
   * Kill 04_CVE-2004-1617.dpatch. (Merged into upstream.)
   * Disable 05_FTBFS_on_GNUHurd_and_GNUkBSD (Upstream changes, file new bug if
     we FTBFS again.)
   * Removed configure arguments:
     --enable-8bit-toupper - Removed, no longer exists.
     --enable-persistent-cookies - Enabled by default.
     --enable-prettysrc - Enabled by default.
     --enable-source-cache - Enabled by default.
     --enable-read-eta - Enabled by default.
   * Added configure arguments:
     --enable-nsl-fork - fork NSL requests, allowing them to be aborted
     --enable-justify-elts - use element-justification logic
   * Update the contents and location of lynx.desktop. Closes: 304989.
   * Other things will be handled by later uploads, patches welcome.
Files:
 5...

[Rolf Leggewie (r0lf)]

What more important page than google? Not everyone will find this workaround. BTW, how about just /dev/null instead of creating an otherwise unnecessary /etc/mailcap.lynx

[Henrik Nilsen Omma (henrik)]

google.com in lynx works fine on Gutsy.

[phaedral (beau-oblios-cap)]

Debian/stable user, but you folks had the bug listed when I searched google for it.

If I use lynx from a virtual console to access google all works fine.

If I use xterm in gnome, ssh to local host, then use lynx to access google all works fine.

If I use xterm in gnome and point lynx to google without first ssh-ing to local host I get the 400 bad request page.

This suggests there is something in the gnome environment variables causing the problem...but I'm not the guy who can sleuth that out.

I have not tried the workaround listed in the initial report.

This affects *blogsite.com blogs now that google runs blogger...

HTH

[Martin Pool (mbp)]

On 6/1/07, Henrik Nilsen Omma <email address hidden> wrote:
> google.com in lynx works fine on Gutsy.

It is still broken in feisty. (I'm just stating this for anyone who
looks at the bug, not insisting on a backport.)

--
Martin

[phaedral (beau-oblios-cap)]

In debian/etch you can fix this with

<blockquote><pre>root# apt-get install lynx-cur</pre></blockquote>

This brings you to lynx version 2.8.7dev.4; hth