the %{remote_ip} output format is broken on proxied connections

[Bill Yikes]

This is how a Tor user would use cURL to grab a header, and also expect to be told which IP address was contacted:

curl --ssl --socks4a 127.0.0.1:9050 -L --head -w '(effective URL => "%{url_effective} @ %{remote_ip}")' "$target_url"

It's broken because the "remote_ip" is actually just printed as the 127.0.0.1 (likely that of the proxy server not the remote target host).

tested on curl ver 7.52.1

[Bill Yikes (yik3s)]

According to the SOCKS4a spec:

  https://www.openssh.com/txt/socks4.protocol
  https://www.openssh.com/txt/socks4a.protocol

With SOCKS4 cURL *must* do DNS resolution and pass the selected IP to the SOCKS server. OTOH, SOCKS4a gives cURL the option to resolve. If cURL fails at DNS resolution, it's expected to send the hostname and 0.0.0.x. So generally cURL should succeed at DNS resolution and thus have the IP of the target server. Yet it's not sharing that info with the user.

SOCKS4a was a bad test case because we can't know whether or not cURL did the DNS resolution. A better test case is with SOCKS4 as follows:

curl --ssl --socks4 127.0.0.1:9050 -L --head -w '(effective URL => "%{url_effective} @ %{remote_ip}")' "$target_url"

We are assured per the SOCKS4 protocol that cURL *must* do DNS resolution, so cURL must know the remote IP address. Yet it still neglects to correctly set the %{remote_ip} value. This is certainly a bug.

Secondary bug--

the manpage states: "remote_ip The remote IP address of the most recently done connection - can be either IPv4 or IPv6 (Added in 7.29.0)"

The man page is ambiguous. The /rule of least astonishment/ would have the user naturally expecting "remote IP" to be the target IP whenever cURL knows it. Since the behavior is non-intuitive, the man page should state in detail what the user should expect to receive for the %{remote_ip} value.

[Bill Yikes (yik3s)]

The socks.c code shows that cURL does not even attempt DNS resolution on SOCKS4a. Strictly speaking, the SOCKS4a spec expects apps to /attempt/ DNS resolution before contacting the socks server. I won't complain on this point though because the status quo is favorable to Tor users (as it protects them from DNS leaks).

The fallout is that the SOCKS server does not give feedback to the app on the IP it settles on in the socks4a scenario. This means cURL has no possible way of knowing which IP to express in the %{remote_ip} output.

After seeing the code I'm calling out these bugs:

bug 1) SOCKS4a: Considering that Curl_resolve() is unconditionally bypassed, when a user supplies both --resolve and also demands socks4a cURL will neglect to honor the --resolve option even though the two options are theoretically compatible. This is a minor bug because socks4 can be used instead as a workaround. But certainly the man page should at a minimum disclose the artificial incompatibility between socks4a and --resolve.

bug 2) SOCKS4a docs: cURL has some discretion whether to attempt DNS resolution or not. Yet the docs do not clarify. Users should get reassurance in the man page that using socks4a unconditionally refrains from internal DNS resolution.

bug 3) SOCKS4: Since cURL *must* do DNS resolution, cURL must also know what the target IP is. Thus cURL should properly return the %{remote_ip} value.

bug 4) The docs for %{remote_ip} should tell users what to expect for that value. The man page is vague enough to be useless.

enhancement 1) Introduce a new option (e.g. --no-dns-lookup) so users can explicitly make it clear that cURL is not to perform external lookups. This would serve as a DNS leak fail-safe for users who use --resolve or socks4a. It would most often help prevent users from self-inflicted harm (they use socks4 and neglect to pass in a --resolve, a mistake easily made in the TclCurl API because it's not always obvious which options are sticky when the handle is reused).

Workaround: if proxy users need to know which IP cURL connected to, they must do their own DNS resolution manually outside of cURL (e.g. using dig), supply the IP & hostname via --resolve, and use SOCKS4 or SOCKS5 (not SOCKS4a).