config files should allow and use proxy declarations

[Kirill Danilov]

Hi.
I have HTTPS repository with authentication behind with proxy with authentication. It seems Bazaar for windows detects proxy and asks for proxy auth automatically with ignoring special proxy settings. After this it asks for repository authentication. The problem is that host shown in dialog is exactly the same although proxy and repository are on different domains. So only one difference I have its realm. But I can't point it in auth config file for automatic authentication.
Please, add this parameter to config.

[Vincent Ladeuil (vila)]

>>>>> Kirill Danilov writes:

    > Public bug reported:
    > Hi.
    > I have HTTPS repository with authentication behind with proxy
    > with authentication.

Ok.

    > It seems Bazaar for windows detects proxy and asks for proxy auth
    > automatically with ignoring special proxy settings.

That's weird, so it be nice to understand what is precisely happening here.

    > After this it asks for repository authentication. The problem is
    > that host shown in dialog is exactly the same although proxy and
    > repository are on different domains.

That's even more weird.

    > So only one difference I have its realm.

But realm comes *back* from the server, so it defeats half of the
feature of providing credentials upfront to avoid one round-trip (which
is why realm is not available in authentication.conf to select
credentials for a given host).

    > But I can't point it in auth config file for automatic
    > authentication. Please, add this parameter to config.

Now, may be we can understand what is happening in your case and fix the
problem there.

If you add -Dhttp to your bzr command, your .bzr.log file (see 'bzr
version' to find where it's located) will contain debug information
that would help diagnose the issue.

Be aware that this will include a (poorly) encrypted form of your
credentials in some headers (mainly Authorization and
Proxy-authorization), please clean them up (replace their *value* with
XXXX), but don't delete the headers themselves.

[Kirill Danilov (donz4all)]

User authenticates not on host but on realm. And one host can have many realms. I don't understand why did you decide to use indirect signs for authentication (you try to detect realm via host) instead of direct signs. Why don't you want to add one more optional parameter 'realm' to authentication.conf?
Ok. I did two test, they are in attach. Both of them had the same output and logs with and without http_proxy params in bazaar.conf.
First one without any info in authentication.conf.
Second one with following sections:

[subversion.mycompany.net]
scheme=https
host=subversion.mycompany.net
user=danilov
password=xxx

[subversion.mycompany.net2]
scheme=http
host=subversion.mycompany.net
user=danilov
password=xxx

[https_proxy]
scheme=https
host=wpad
port=8080
user=KDanilov
password=xxx

[http_proxy]
scheme=http
host=wpad
port=8080
user=KDanilov
password=xxx

[Vincent Ladeuil (vila)]

>>>>> Kirill Danilov writes:

    > User authenticates not on host but on realm.

No, users need both, http by itself requires credentials on every
request.

    > And one host can have many realms.

Yes, at different paths, paths are available for filtering. But you
can't have several realms on the same path.

    > I don't understand why did you decide to use indirect signs for
    > authentication (you try to detect realm via host)

We don't detect them, we *receive* them. That's the point. We have to
issue a first request that will *fail* (401 auth required) to *get* a
realm.

The intent is to *avoid* this request by providing the credentials for
the first request. So we need to predict them based on information
already available: a host, a port, a user, a path but no realm.

    > instead of direct signs. Why don't you want to add one more
    > optional parameter 'realm' to authentication.conf?

Because we don't know the realm before contacting the host so they are
not a meaningful way to predict which credentials should be sued.

    > Ok. I did two test, they are in attach. Both of them had the same
    > output and logs with and without http_proxy params in bazaar.conf.
    > First one without any info in authentication.conf. Second one
    > with following sections:

subversion use a different mean to remember credentials (and don't query
bzr to get them). You should issue a successful request with svn to make
it store these credentials and reuse them.

[Kirill Danilov (donz4all)]

       >Yes, at different paths, paths are available for filtering. But you can't have several realms on the same path.

Right. But why do I receive the same information about host during both authentication? Settings for wpad:8080 are ignored if I point them in authentication.conf. So for a now I have no way to make authentication automatic. I have to input two logins and two passwords every time I need communication with this server.

      >Because we don't know the realm before contacting the host so they are not a meaningful way to predict which credentials should be sued.

But you know it after first fail. After that bzr can get right credentials from authentication.conf depart from realm. This will be enough for my case.

       >subversion use a different mean to remember credentials (and don't query bzr to get them). You should issue a successful >request with svn to make it store these credentials and reuse them.

My local repository is Bazaar type. Subversion is only on remote server. And I don't understand how proxy and http authentication are related to subversion. Bzr has no information about destination repository during auth phase.
I have already done successful requests to this server from subversion but on another local repository. I can't find out how to link my bazaar local repository with subversion config files.

[Vincent Ladeuil (vila)]

> But why do I receive the same information about host during both authentication?

That is a bug. You should be queried for the proxy credentials with a prompt for the proxy host and authentication.conf should be queried for this host first.

Then you should be queried with a prompt for the remote host (and authentication.conf should be queried for this second host too). Except that this last part is going through bzr-svn which talks to svn directly without involving authentication.conf which is why svn needs to be involved so the credentials are saved in svn credential cache.

> Subversion is only on remote server.

That's where you need to authenticate then.

> And I don't understand how proxy and http authentication are related to subversion.

You access the subversion repository via http, so you need to authenticate for the remote server and to get there you need to authenticate for the proxy.

> Bzr has no information about destination repository during auth phase.

It knows the url and it knows the proxy. You're asked for the remote server credentials because access there is required for the operation you're asking.

> Settings for wpad:8080 are ignored

That's weird, are you sure you use:
[http_proxy]
scheme=http
host=wpad
port=8080
user=KDanilov
password=xxx

not host=wpad:8080 ?

Using -Dauth in addition to -Dhttp should help understand what is going on here...

(and note that I'm not discussing adding realm in auth.conf because your use case *should* work without it).

Also, the log you pasted says:

  Proxy HTTPS subversion.mycompany.net, Realm: 'PRV-ISA-01.mycompany.ru' username: KDanilov
  Proxy HTTPS <email address hidden>, Realm: 'PRV-ISA-01.mycompany.ru' password:
  HTTPS subversion.mycompany.net, Realm: 'Main Subversion repository' username: danilov
  HTTPS <email address hidden>, Realm: 'Main Subversion repository' password:

but the associated .bzr.log says:

  0.294 > CONNECT subversion.mycompany.net:443
  0.295 > Host: wpad:8080

i.e. the prompt should really be:

Proxy HTTPS wpad:8080, Realm: 'PRV-ISA-01.mycompany.ru' username: KDanilov

Also, you said: "with and without http_proxy params in bazaar.conf" but http_proxy should be specified as an environment variable, we never look for it into bazaar.conf (may be we should), so it's unclear to me what you're doing exactly there :-/

[Kirill Danilov (donz4all)]

>That is a bug. You should be queried for the proxy credentials with a prompt for the proxy host and authentication.conf should be queried for this host first.

Ok. I understand your arguments about realm. Will this bug be fixed in next release?

>That's weird, are you sure you use:
[http_proxy]
scheme=http
host=wpad
port=8080
user=KDanilov
password=xxx

>not host=wpad:8080 ?

Yes, I'm sure that I point address and port separately. I think it's right way, isn't it?

>i.e. the prompt should really be:
>Proxy HTTPS wpad:8080, Realm: 'PRV-ISA-01.mycompany.ru' username: KDanilov

Should be, but wasn't :) I made copy-paste without any changes except passwords and company name

>Also, you said: "with and without http_proxy params in bazaar.conf" but http_proxy should be specified as an environment variable, we never look for it into bazaar.conf (may be we should), so it's unclear to me what you're doing exactly there :-/

Yes, right. Now I remembered that proxy should be set via environmental variable. Some time ago I set proxy via this approach too, but I have the same behavior.
But I think it's good idea to add http_proxy parameters to config files, because it is more usual way.

[Vincent Ladeuil (vila)]

>>>>> Kirill Danilov writes:

    >> That is a bug. You should be queried for the proxy credentials with a
    >> prompt for the proxy host and authentication.conf should be queried for
    >> this host first.

    > Ok. I understand your arguments about realm. Will this bug be fixed in
    > next release?

I need to understand what is happening for you first :-/

    >> That's weird, are you sure you use:
    >> [http_proxy]
    >> scheme=http
    >> host=wpad
    >> port=8080
    >> user=KDanilov
    >> password=xxx

    >> not host=wpad:8080 ?

    > Yes, I'm sure that I point address and port separately. I think it's
    > right way, isn't it?

Yes. host and port should be separated.

The weird thing here is that your auth.conf file seems correct and
should work.

My suspicion is that the proxy is not configured correctly (or bzr can't
recognize it properly).

Then, the proxy intercepts the request anyway and bzr got confused.

Or something along those lines.

The alternative is that bzr proxy support have been broken but I'd be
really surprised if you were the first to encounter the issue...

    >> i.e. the prompt should really be:
    >> Proxy HTTPS wpad:8080, Realm: 'PRV-ISA-01.mycompany.ru' username: KDanilov

    > Should be, but wasn't :)

That's the problem. I wonder if your http_proxy env variable is
recognized...

    > I made copy-paste without any changes except passwords and company
    > name

... and obfuscate the auth headers, that's good, thanks for that.

    >> Also, you said: "with and without http_proxy params in bazaar.conf" but
    >> http_proxy should be specified as an environment variable, we never look
    >> for it into bazaar.conf (may be we should), so it's unclear to me what
    >> you're doing exactly there :-/

    > Yes, right. Now I remembered that proxy should be set via
    > environmental variable. Some time ago I set proxy via this
    > approach too, but I have the same behavior.

Can you try again to set the env variable, run the bzr command with both
-Dhttp and -Dauth and attach the result.

Also, I'd be interested to know:

- if you can connect successfully with subversion itself,

- what prompts you get

- whether you have to specify the proxy and how,

- if the credentials are cached so you don't have to type them for every
  connection.

AFAIK, a first successful connection with svn itself is all that is
needed (and a working proxy configuration of course).

    > But I think it's good idea to add http_proxy parameters to config
    > files, because it is more usual way.

On windows yes, there is a work in progress that would make this easier.

[Kirill Danilov (donz4all)]

Hi.

Sorry for delay. After inspecting bazaar configs I found that proxy was set all this time and my previous logs were made with this setting. It's my inattention but I think I'm not alone in suggesting that all settings should be in one place.
And the second surprise for me was correctly working authorization when I removed proxy variables! So for my case problem is solved. It seems bzr can use system settings and domain authorization for ISA proxy.
Thank you for showing me the right way.
But IMHO you still have bug with incorrect host determining with proxy settings. I did two logs (one with proxy and another one without) with command:
bzr pull https://subversion.mycompany.net/repos/java/myproject/trunk --remember -Dhttp -Dauth

Good luck!

[Vincent Ladeuil (vila)]

So, It's unclear to me from your last comment that the right hosts were used to query authentication.conf or how you did provide the credentials.

But anyway, I'm happy it worked for you in the end.

I'll keep this bug opened but will re-targeted it at adding support for the proxy variables in the bazaar config files (as opposed to recognizing only the environment variables) as that's what seems the more appropriate (if I understand you correctly).

Thank you for your persistent feedback !

[Kirill Danilov (donz4all)]

>So, It's unclear to me from your last comment that the right hosts were used to query authentication.conf or how you did provide the credentials.

I set credentials only for suberversion.mycompany.net host in authorization.conf and that's all. Bzr detected proxy and made domain authorization automatically.

And for my understanding you still have bug with wrong host detection when using http_proxy environment variables.

Thank you for helping me, good luck!

[Kirill Danilov (donz4all)]

Sorry for wrong information.
It seems I don't need proxy for accessing host subversion.mycompany.net in my network. So Bazaar didn't do any automatic actions with proxy. My bad.