wish: persistent client-side per-branch authentication

Bug #622011 reported by Martitza
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Bazaar
Invalid
Undecided
Unassigned
Bazaar Explorer
Confirmed
Wishlist
Unassigned
QBzr
Confirmed
Wishlist
Unassigned

Bug Description

There may already be a better solution for this. I'd like to know. Consider this wish a topic for discussion.

Bzr 2.2 and earlier appear to open a separate connection for every ssh operation on a remote branch. These connections become particularly noticeable when a GUI client like Bazaar Explorer is used, because most people can click icons faster than they can type full commands. For example, just "opening" the working tree of an existing checkout invokes four short-lived ssh sessions. Now get a 'log' and do 'annotate' a few times. You get the idea. Lots of ssh connections hitting the same credentials.

Here's why it matters. IP-specific tools like fail2ban may not be much use against DDoS. So some professional paranoids (like my IT department) also throttle ssh connection requests, regardless of whether they succeed or fail. I just found out that if my department firewall sees more than 20 *total* inbound ssh connection requests in a minute, our firewall locks out *all* ssh connections for ten minutes. What a pain! But I don't control security, and those guys have a job to do which is just as important as mine. I tried to explain to them that this is a *guaranteed* DoS, because all someone has to do to shut down *all* of our ssh traffic is hit us with 20 ssh requests every ten minutes. They gave me a dirty look and I left. :)

This is not theoretical. It saw it happen today on a test box while I was using bzr explorer. I'm going to have to be careful not to annoy those IT guys. It probably could be avoided if every command which can accept a location could also accept an option to use an existing connection if it exists and leave the connection open if it has to open one. Of course, there should be a way to drop all open connections. And the connection should be set on a per-location or per-branch basis, since multiple locations might be in use at once.

Some people might object to the idea of leaving a connection open. But it's no more rude than leaving a ssh login terminal open for hours. And the IT guys would rather kill a few stale connections than figure out how to deal with a flood of short-lived ones which look like something they want to stop. My problem might go away, now that I've pointed out that the ssh throttle is more likely to hurt them than help them. But until they do something smarter, I will be thinking more before clicking.

Tags: multiprocess
Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 622011] [NEW] wish: persistent client-side per-branch authentication

openssh supports persistent 'master' connections; you could use that
(define a control channel, start the master by hand). I'm not sure how
we'd glue management of such a thing into bzr, which has a short-lived
process model.

Revision history for this message
Martitza (martitzam) wrote :
Download full text (3.2 KiB)

@Robert : I did not know that. Looks like it might work. Thanks!

On Sat, Aug 21, 2010 at 4:46 PM, Robert Collins
<email address hidden>wrote:

> openssh supports persistent 'master' connections; you could use that
> (define a control channel, start the master by hand). I'm not sure how
> we'd glue management of such a thing into bzr, which has a short-lived
> process model.
>
> --
> wish: persistent client-side per-branch authentication
> https://bugs.launchpad.net/bugs/622011
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Bazaar Version Control System: New
>
> Bug description:
>
> There may already be a better solution for this. I'd like to know.
> Consider this wish a topic for discussion.
>
> Bzr 2.2 and earlier appear to open a separate connection for every ssh
> operation on a remote branch. These connections become particularly
> noticeable when a GUI client like Bazaar Explorer is used, because most
> people can click icons faster than they can type full commands. For
> example, just "opening" the working tree of an existing checkout invokes
> four short-lived ssh sessions. Now get a 'log' and do 'annotate' a few
> times. You get the idea. Lots of ssh connections hitting the same
> credentials.
>
> Here's why it matters. IP-specific tools like fail2ban may not be much use
> against DDoS. So some professional paranoids (like my IT department) also
> throttle ssh connection requests, regardless of whether they succeed or
> fail. I just found out that if my department firewall sees more than 20
> *total* inbound ssh connection requests in a minute, our firewall locks out
> *all* ssh connections for ten minutes. What a pain! But I don't control
> security, and those guys have a job to do which is just as important as
> mine. I tried to explain to them that this is a *guaranteed* DoS, because
> all someone has to do to shut down *all* of our ssh traffic is hit us with
> 20 ssh requests every ten minutes. They gave me a dirty look and I left.
> :)
>
> This is not theoretical. It saw it happen today on a test box while I was
> using bzr explorer. I'm going to have to be careful not to annoy those IT
> guys. It probably could be avoided if every command which can accept a
> location could also accept an option to use an existing connection if it
> exists and leave the connection open if it has to open one. Of course,
> there should be a way to drop all open connections. And the connection
> should be set on a per-location or per-branch basis, since multiple
> locations might be in use at once.
>
> Some people might object to the idea of leaving a connection open. But
> it's no more rude than leaving a ssh login terminal open for hours. And the
> IT guys would rather kill a few stale connections than figure out how to
> deal with a flood of short-lived ones which look like something they want to
> stop. My problem might go away, now that I've pointed out that the ssh
> throttle is more likely to hurt them than help them. But until they do
> something smarter, I will be thinking more before clicking.
>
> To unsubscribe from this bug, go to:
> https://bugs.lau...

Read more...

Revision history for this message
Vincent Ladeuil (vila) wrote :

bzr is able to reuse connections but AFAIK bzr-explorer relies on qbzr and spawn a process for each command so we're doomed on this front.

Alternatively ssh tunnels may address your problem (or master connections as Robert said but I don't use
them so I can't provide direct feedback there).

There are a bunch of utilities that help define and manage ssh tunnels presumably for all platorms
(it's a bit easier on unices where ~/.ssh/config can be used to define them).

Revision history for this message
Gary van der Merwe (garyvdm) wrote : Re: [Bug 622011] Re: wish: persistent client-side per-branch authentication

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 23/08/2010 09:35, Vincent Ladeuil wrote:
> bzr is able to reuse connections but AFAIK bzr-explorer relies on qbzr
> and spawn a process for each command so we're doomed on this front.

Alexandra and I discussed this, and we want to change this. But it may
take a while.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkxyJ3cACgkQd/3EdwGKOh2TgQCgwLkyE0ixKXNRD0DKug4Llehm
VIgAoJYua+4WbkQPC4Hs1g78O9rrOVWU
=3Ocl
-----END PGP SIGNATURE-----

Revision history for this message
Gary van der Merwe (garyvdm) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 23/08/2010 09:35, Vincent Ladeuil wrote:
> bzr is able to reuse connections but AFAIK bzr-explorer relies on qbzr
> and spawn a process for each command so we're doomed on this front.

Alexander and I discussed this, and we want to change this. But it may
take a while.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkxyJ40ACgkQd/3EdwGKOh0nPACbBzaMp4ywItJIn/AO9TOpoC4c
+xoAn1m2TVqsiKCu0VSPcB1VqFzZTA7h
=iciO
-----END PGP SIGNATURE-----

Revision history for this message
Martitza (martitzam) wrote :

On Mon, Aug 23, 2010 at 12:35 AM, Vincent Ladeuil <<email address hidden>
> wrote:

> bzr is able to reuse connections but AFAIK bzr-explorer relies on qbzr
> and spawn a process for each command so we're doomed on this front.
>
> Alternatively ssh tunnels may address your problem (or master connections
> as Robert said but I don't use
> them so I can't provide direct feedback there).
>
> There are a bunch of utilities that help define and manage ssh tunnels
> presumably for all platorms
> (it's a bit easier on unices where ~/.ssh/config can be used to define
> them).
>
> --
> wish: persistent client-side per-branch authentication
> https://bugs.launchpad.net/bugs/622011
>

Hi Vincent,

I thought about master connections. That would work for me, but I would not
want to have to train a bunch of people to do that. I'm trying to convince
them that bzr is easy. A pool of clear cool water in the desert is
wonderful, unless you have to climb a mountain to drink it. Technically
valid, but socially nonviable for my team. Not a bzr problem. And I
personally use ssh tunnels several times every day. But again I can't ask
my developers (or doc writers!) to use tunnels. Especially on Windows,
where everything useful seems to be much harder than on unix.

I did not know that bzr can reuse connections. I guess that explains why I
never had to think about it. :)

Maybe we are not doomed with qbzr. Perhaps qbzr can be taught to reuse
connections also. I have added qbzr to the list of affected projects.

Thanks!
~M

Revision history for this message
John A Meinel (jameinel) wrote : Re: [Bug 622011] Re: wish: persistent client-side per-branch authentication

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

...

> Hi Vincent,
>
> I thought about master connections. That would work for me, but I would not
> want to have to train a bunch of people to do that. I'm trying to convince
> them that bzr is easy. A pool of clear cool water in the desert is
> wonderful, unless you have to climb a mountain to drink it. Technically
> valid, but socially nonviable for my team. Not a bzr problem. And I
> personally use ssh tunnels several times every day. But again I can't ask
> my developers (or doc writers!) to use tunnels. Especially on Windows,
> where everything useful seems to be much harder than on unix.
>
> I did not know that bzr can reuse connections. I guess that explains why I
> never had to think about it. :)
>
> Maybe we are not doomed with qbzr. Perhaps qbzr can be taught to reuse
> connections also. I have added qbzr to the list of affected projects.
>
> Thanks!
> ~M

If you use heavyweight checkouts instead of lightweight checkouts,
you'll have far fewer ssh connections.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkxzHz4ACgkQJdeBCYSNAAMragCgpKlY0F26pmUxborSGVQ+UDlT
WOgAoKCQ6FXg41ELz/ILeGqv9SeCvn8L
=E3gz
-----END PGP SIGNATURE-----

Revision history for this message
Gary van der Merwe (garyvdm) wrote : Re: [Bug 622011] Re: wish: persistent client-side per-branch authentication

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 24/08/2010 01:02, Martitza wrote:
> Maybe we are not doomed with qbzr. Perhaps qbzr can be taught to reuse
> connections also. I have added qbzr to the list of affected projects.

As mentioned before, this would be possible if everything ran in the
same process. Right now we have:

Bazaar Explorer -> QBzr Dialog -> bzr
process boundary 1 ^ process boundary 2 ^

Process boundary 1 would be quite easy to get rid of, but it will
probably mean losing the ability for BE to launch bzr-gtk dialogs. No
one is working on this at the moment.

Process boundary 2 only exists in dialogs like commit, pull, push, etc..
and not in log, diff, annotate. It was done so that we could easily keep
the ui interactive while a long running bzr command was running. In
order to get rid of this process boundary, I'm working on a branch were
we will have 2 threads, one for the ui, and one for everything else.

Gary

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkxznvIACgkQd/3EdwGKOh2ZgwCeMHCIXyo6PJAw68yc9qe4XenZ
BDAAoLjM+NuidekGYJzR/OdxUyIf6tqf
=H2z4
-----END PGP SIGNATURE-----

Revision history for this message
Martin Pool (mbp) wrote :

I'm going to close the bzr side of this for the moment, unless it turns out there is something to do there.

At the moment qbzr starts multiple processes, and it's not easy for us to hold connections open across that. We could reinvent something like ssh connection sharing, and perhaps we should, but that seems a bit far from our core business. It seems to me a cleaner way to do this would be to have qbzr either reuse the same process for multiple operations, or for it to just do fewer bzr operations.

Changed in bzr:
status: New → Invalid
Revision history for this message
Alexander Belchenko (bialix) wrote :

My idea for Bazaar Explorer is to create a separate process for each opened branch to work with bzr. So Explorer will be able to connect to remote branch once and keep the connection open. But this is very major change and will require to change qbzr as well. So I'm not quite sure when.

Changed in bzr-explorer:
status: New → Confirmed
importance: Undecided → Wishlist
Changed in qbzr:
importance: Undecided → Wishlist
status: New → Confirmed
tags: added: multiprocess
Revision history for this message
Alexander Belchenko (bialix) wrote :
Revision history for this message
Martitza (martitzam) wrote :
Download full text (3.2 KiB)

Yes. I think it makes sense to combine this issue with
https://bugs.launchpad.net/bzr-explorer/+bug/430311 where John said
essentially the same thing and more.

~M

On Mon, May 9, 2011 at 4:37 AM, Alexander Belchenko <email address hidden> wrote:

> See also https://bugs.launchpad.net/bzr-explorer/+bug/430311
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/622011
>
> Title:
> wish: persistent client-side per-branch authentication
>
> Status in Bazaar Version Control System:
> Invalid
> Status in Bazaar Explorer:
> Confirmed
> Status in Qt frontend for Bazaar:
> Confirmed
>
> Bug description:
>
> There may already be a better solution for this. I'd like to know.
> Consider this wish a topic for discussion.
>
> Bzr 2.2 and earlier appear to open a separate connection for every ssh
> operation on a remote branch. These connections become particularly
> noticeable when a GUI client like Bazaar Explorer is used, because
> most people can click icons faster than they can type full commands.
> For example, just "opening" the working tree of an existing checkout
> invokes four short-lived ssh sessions. Now get a 'log' and do
> 'annotate' a few times. You get the idea. Lots of ssh connections
> hitting the same credentials.
>
> Here's why it matters. IP-specific tools like fail2ban may not be
> much use against DDoS. So some professional paranoids (like my IT
> department) also throttle ssh connection requests, regardless of
> whether they succeed or fail. I just found out that if my department
> firewall sees more than 20 *total* inbound ssh connection requests in
> a minute, our firewall locks out *all* ssh connections for ten
> minutes. What a pain! But I don't control security, and those guys
> have a job to do which is just as important as mine. I tried to
> explain to them that this is a *guaranteed* DoS, because all someone
> has to do to shut down *all* of our ssh traffic is hit us with 20 ssh
> requests every ten minutes. They gave me a dirty look and I left. :)
>
> This is not theoretical. It saw it happen today on a test box while I
> was using bzr explorer. I'm going to have to be careful not to annoy
> those IT guys. It probably could be avoided if every command which
> can accept a location could also accept an option to use an existing
> connection if it exists and leave the connection open if it has to
> open one. Of course, there should be a way to drop all open
> connections. And the connection should be set on a per-location or
> per-branch basis, since multiple locations might be in use at once.
>
> Some people might object to the idea of leaving a connection open.
> But it's no more rude than leaving a ssh login terminal open for
> hours. And the IT guys would rather kill a few stale connections than
> figure out how to deal with a flood of short-lived ones which look
> like something they want to stop. My problem might go away, now that
> I've pointed out that the ssh throttle is more likely to hurt them
> than help them. But until they do something smarter, I will be
> thinking more before c...

Read more...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.