Bugs behind proxy server with/without authentication

[antok.tm]

Binary package hint: pidgin

I'm running pidgin behind a proxy server (squid) that has a firewall (shorewall). The basic functions run great. But the buddy icon (Yahoo! protocol) can't show up. This problem has been haunting my network since gaim has buddy icon feature (I'm not sure which version).

If I use the Yahoo Messenger app (v8) , the buddy icons appear.
Actually I'm not sure if this really a bug or incorrect setting in the proxy server

[antok.tm (antok-elect-eng)]

Here is my pidgin's debug window output when trying to get buddy icon :

(14:58:33) util: Request: 'GET /msgr/buddy_name/.friend_icon.png?msIlQRHB7KbknkWi HTTP/1.0
Connection: close
User-Agent: Mozilla/4.0 (compatible; MSIE 5.0)
Accept: */*
Host: f3.yahoofs.com
'
(14:58:33) util: Response headers: 'HTTP/1.0 400 Bad Request
Server: squid/2.6.STABLE16
Date: Wed, 21 Nov 2007 07:58:02 GMT
Content-Type: text/html
Content-Length: 1361
Expires: Wed, 21 Nov 2007 07:58:02 GMT
X-Squid-Error: ERR_INVALID_REQ 0
X-Cache: MISS from my.proxyserver.com
X-Cache-Lookup: NONE from my.proxyserver.com:8080
Via: 1.0 my.proxyserver.com:8080 (squid/2.6.STABLE16)
Proxy-Connection: close

[antok.tm (antok-elect-eng)]

Looks like the host not resolved by my proxyserver correctly. Exact response also recived when accessing address.yahoo.com at the beginning of connection. This debugging windows is really handy....
And also if I try to view an user info, it gets 'Access Denied', seems like the proxy server is not authenticated properly.

[antok.tm (antok-elect-eng)]

Here is what I got when trying to 'Get Info' :

(18:03:30) util: requested to fetch (http://profiles.yahoo.com/my_buddy), full=1, user_agent=((null)), http11=0
(18:03:30) dns: DNS query for 'my.dnsserver.com' queued
(18:03:30) dns: Created new DNS child 7011, there are now 1 children.
(18:03:30) dns: Successfully sent DNS request to child 7011
(18:03:31) dns: Got response for 'my.dnsserver.com'
(18:03:31) dnsquery: IP resolved for my.dnsserver.com
(18:03:31) proxy: Attempting connection to 1.2.3.4
(18:03:31) proxy: Connecting to profiles.yahoo.com:80 via my.proxyserver.com:8080 using HTTP
(18:03:31) proxy: Connection in progress
(18:03:31) proxy: HTTP proxy connection established
(18:03:31) util: Request: 'GET http://profiles.yahoo.com/my_buddy HTTP/1.0

Connection: close

Accept: */*

Host: profiles.yahoo.com

'
(18:03:31) util: Response headers: 'HTTP/1.0 407 Proxy Authentication Required

Server: squid/2.6.STABLE16

Date: Wed, 21 Nov 2007 11:02:58 GMT

Content-Type: text/html

Content-Length: 1378

Expires: Wed, 21 Nov 2007 11:02:58 GMT

X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0

Proxy-Authenticate: Basic realm="my.proxyserver.com"

X-Cache: MISS from my.proxyserver.com

X-Cache-Lookup: NONE from my.proxyserver.com:8080

Via: 1.0 my.proxyserver.com:8080 (squid/2.6.STABLE16)

Proxy-Connection: close

[Adrianus (adrianus-kurnadi)]

Hi,
I also experience the same bugs. Somehow when connected to a proxy pidgin sent incorrect url when fetching yahoo buddy icon. It strips the protocol and website.
I heavily suspect this is a pidgin bug, because my other friends using Kopete and YM client in windows has no problem fetching the buddy icon through the exact same proxy. My office proxy (squid) was not setup to require authentication so I doubt the problem is in that direction.
Note that getting profile is working, only fetching yahoo buddy icon is not.

Best Regards
Adrianus

[antok.tm (antok-elect-eng)]

I'm actually unsure who's to be blame here : pidgin or squid. But one thing for sure : those two doesn't work together pretty well. I've already tested pidgin on a commercial OS , but the results are the same.
Some weeks ago I also mail this bug on pidgin's mailing list but they haven't respond to this problem yet.
And I'm pretty sure this bug also affect file transfer.

Antok

[Stuart Rossiter (monsieurrigsby)]

I'm having similar problems (Gutsy, Pidgin 2.2.1) but possibly different root cause (devs - please advise if you'd prefer a new bug).

Basically, both Yahoo buddy icon gets and file transfers file behind a proxy server because: I have to set "No proxy" for the basic port 5050 connection, and Pidgin isn't honouring the global (Gnome) HTTP proxy connections for its separate port 80 connections for file transfers and buddy icon uploads/downloads.

It's strange because the debug window shows that it *is* honouring the Gnome HTTP proxy settings for the address.yahoo.com connection, but not the filetransfer.msg.yahoo.com (which it appears to use for both buddy icon and other file transfers). Extracts of debug log below confirming this (ist:3128 is the Squid proxy I'm using).

OK main connection with no proxy:

(10:46:01) proxy: Attempting connection to 66.163.181.187
(10:46:01) proxy: Connecting to scs.msg.yahoo.com:5050 with no proxy
(10:46:01) proxy: Connection in progress
(10:46:01) proxy: Connected to scs.msg.yahoo.com:5050.

No Gnome proxy settings used for filetransfer connections (hence they fail - first one for buddy icons, second for actual file transfer):

(10:46:01) dns: Got response for 'filetransfer.msg.yahoo.com'
(10:46:01) dnsquery: IP resolved for filetransfer.msg.yahoo.com
(10:46:01) proxy: Attempting connection to 209.191.110.214
(10:46:01) proxy: Connecting to filetransfer.msg.yahoo.com:80 with no proxy
(10:46:01) proxy: Connection in progress
(10:46:04) proxy: Connected to filetransfer.msg.yahoo.com:80.
(10:46:04) proxy: Error connecting to filetransfer.msg.yahoo.com:80 (No route to host).
(10:46:04) proxy: Connection attempt failed: No route to host
(10:46:04) yahoo: Buddy icon upload failed: No route to host

(10:55:59) proxy: Attempting connection to 209.191.110.214
(10:55:59) proxy: Connecting to filetransfer.msg.yahoo.com:80 with no proxy
(10:55:59) proxy: Connection in progress
[never completes]

Note separate bug #52801 advocating *not* using the global proxy settings:
https://bugs.launchpad.net/ubuntu/+source/pidgin/+bug/52801

If this is an issue as well then presumably the best fix is to add an option to the Yahoo settings to be able to set separate proxy settings for the main (port 5050) connection and the HTTP connections. Otherwise, it should honour the global Gnome HTTP proxy settings I guess.

Apologies if this pollutes what looks a slightly different bug for others (as mentioned earlier, can put in another bug if advised), but this seems a prime root cause problem for all Yahoo proxy related problems.

[Stuart Rossiter (monsieurrigsby)]

Doh - meant "file transfers *fail*" in second paragraph.

[AliRezaTaleghani (alirezataleghani)]

Yep, this is a problem that i have faced with for a long time.
it seems the problem should be for the algoritm that pidgin use for yahoo account , depending on the way we connect to INTERNET, (Proxy || NAT/FIREWALL) .

[Daniel T Chen (crimsun)]

Is this symptom still reproducible in 8.10?

[antok.tm (antok-elect-eng)]

The latest version of Pidgin still doesn't fetch the buddy icons behind a Squid proxy server.
Haven't tried it on my 8.10 system lately, but I'm quite sure it still doesn't.

Some months ago I managed to get a fix on this by hacking into the pidgin source code - the yahoo protocol part.
But after two or three pidgin updates, I get lazy to hack the source code again.

So basicly when connecting behind a proxy server, the newer version of pidgin already got a workaround. It has a different approach when it connects behind a proxy or not. But this gives a new problem for squid :
When fetching a buddy icons, Squid needs to know what is the full url of the image is. Pidgin only gives:
                the host and then the filename.
And so Squid fails to understand where the actual image location is, because it needs to know :
                the host and the full url.

Maybe I could post the source-code hack later.
I hope my explanation is clear enough ...

[antok.tm (antok-elect-eng)]

A few correction for my previous comment :
Pidgin did use the full-url workaround if the user are using it behind a proxy server. So the problem should be fixed by now if you are connecting behind a normal proxy server.

In my university's network, the ports for Yahoo Messenger are open. So the users don't have to connect to the proxy server to use YM. But because the buddy icons is fetched not through the YM ports, the users still need to use the proxy server for this.

Luckily, the settings for the buddy-icons-fetching are not account specific. It depends on the "tools"->"preferences"->"network"->"configure proxy" settings. So the users can set their YM account to not using proxy, and set the buddy-icons-fetching activites to use proxy.

The problem is, Pidgin still looks into the account specific setting to decide wether the user is using proxy or not.... So in my case, the buddy icon is still fetched without the full-url.

Here's my workaround. On the source code, find the file : libpurple/protocols/yahoo/yahoo_picture.c. There should be a line containing :

                     gboolean use_whole_url = FALSE;

change this to :

                     gboolean use_whole_url = TRUE;

this way, pidgin doesn't care if you use proxy or not, it always use full url to fetch buddy icons. I don't understand why they don't simply always use a full url for this code. Compile it and replace the file : /usr/lib/purple-2/libyahoo.so .

But anyway, I think that workaround is only needed if you're facing the same rare cases as I do.

[Sebastien Bacher (seb128)]

the bug should be sent to the software writters on pidgin.im by somebody having the issue

[Stuart Rossiter (monsieurrigsby)]

OK, as per Sebastien's comment, have raised a bug on Pidgin (#9552):
http://developer.pidgin.im/ticket/9552

Delayed since Pidgin's development server kept crashing on me. Included all the pertinent detail from here. Let's see if anyone picks this up...

[Pedro Villavicencio (pedro)]

Thank you for taking the time to report this bug and helping to make Ubuntu better.You reported this bug a while ago and there hasn't been any activity in it recently. We were wondering is this still an issue for you? Do you get the same with Lucid or Maverick? Thanks in advance.

[Vish (vish)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. We are closing this bug report because it lacks the information we need to investigate the problem, as described in the previous comments. Please reopen it if you can give us the missing information, and don't hesitate to submit bug reports in the future.
To reopen the bug report you can click on the current status, under the Status column, and change the Status back to "New".