Comment 27 for bug 1227575

Revision history for this message
CloudDon (sriramhere) wrote : Re: [Openstack-security] [Bug 1227575] Re: DoS style attack on noVNC server can lead to service interruption or disruption

Thanks for the comments Nathan.

I wanted to check if references to 3rd party tools are allowed in the
reference section.

On Sat, Jan 4, 2014 at 10:11 AM, Nathan Kinder <email address hidden> wrote:

> I would slightly reword the "Summary" section to be a bit more clear.
> I'd suggest something like this:
>
> -------------
> There is currently no limitation on the number of VNC sessions that can be
> established for a single user's VNC token. This enables one to cause a
> Denial of Service (DoS) style attack by establishing many VNC sessions
> against a single instance through a noVNC proxy. This can cause timeouts
> for other users who are trying to access the same instance through VNC.
> -------------
>
> In the "Discussion" section, I don't think the first sentence is needed
> ("NoVNC Proxy is explained here"). There are also some areas where I
> would suggest slight rewording:
>
> -------------
> Once a user gets a token to access an instance's VNC console, there is no
> restriction on the frequency that the user can attempt to connect to the
> instance's VNC console using that token. There is also no restriction on
> the number of simultaneous VNC sessions that the user can establish against
> the instance using a single token. If many connection requests are made,
> any subsequent connection requests made by other users may time out. This
> could also impact other user's currently established VNC sessions to the
> instance. The overall responsiveness of other Nova services running on the
> noVNC host.
>
> By taking advantage of the lack of any VNC connection limiting, a single
> user could cause the noVNC proxy endpoint to be non-responsive or
> unreachable. This results in a DoS attack. It should be noted that there
> is no amplification effect.
> -------------
>
> I think the OSSN needs to mention Havana as well. It currently only
> indicates that Grizzly is affected in the "Affected Services" and
> "Recommended Actions" sections.
>
> You have a few spots that use "NoVNC" or "novnc". These should be
> "noVNC".
>
> In the "Recommended Actions" section, you start to make a reference to
> some best practices but there is no reference. Were you intending to
> refer to the Security Guide?
>
> --
> You received this bug notification because you are a member of OpenStack
> Security Group, which is subscribed to OpenStack.
> https://bugs.launchpad.net/bugs/1227575
>
> Title:
> DoS style attack on noVNC server can lead to service interruption or
> disruption
>
> Status in OpenStack Compute (Nova):
> In Progress
> Status in OpenStack Security Notes:
> New
>
> Bug description:
> There is no limiting on the number of VNC sessions that can be created
> for a single user's VNC token.
> Any attempt to create multiple (say hundreds or thousands) of websocket
> connections to the VNC server
> results in many connection timeouts. Due to these connection timeout
> error, other users trying to access their
> VM's VNC console cannot do so.
>
> A sample script that tries to create 100,000 connections to Nova noVNC
> proxy, shows timeout errors
> Script: http://paste.openstack.org/show/47254/
>
> Script output.... connections get timed out after a while
> -------------------
> ....
> ..
>
> Creating Connection
> Receiving...
> Received 'RFB 003.008
> '
> Creating Connection
> Receiving...
> Received 'RFB 003.008
> '
> Creating Connection
> Receiving...
> Received 'RFB 003.008
> '
> Creating Connection
> Receiving...
> Received 'RFB 003.008
> '
> Creating Connection
> Receiving...
> Received 'RFB 003.008
> '
> Creating Connection
> Receiving...
> Received 'RFB 003.008
> '
> Creating Connection
> Receiving...
> timed out
> Creating Connection
> Receiving...
> timed out
> Creating Connection
> Receiving...
> timed out
> Creating Connection
> Receiving...
> timed out
> Creating Connection
> Receiving...
> timed out
> --------------------
>
> Impact:
> 1. Many of the sessions timeout. Any attempt to open other sessions also
> intermittently fail.
> This can cause serious problems to users already having a running VNC
> session or trying to create new sessions.
>
> 2. The overall performance and response times of other nova services
> running on the novnc host, using tcp protocol
> also gets affected after the connection timeout errors.
>
> For example:
> Before running the sumulate thousands of connections program:
> $ time nova get-vnc-console c1b093a3-f53b-4282-b89c-e68f0fa1b6e5 novnc
>
> +-------+---------------------------------------------------------------------------------+
> | Type | Url
> |
>
> +-------+---------------------------------------------------------------------------------+
> | novnc |
> http://10.2.3.102:6080/vnc_auto.html?token=e776dd33-422f-4b56-9f98-e317410d0212|
>
> +-------+---------------------------------------------------------------------------------+
>
> real 0m0.751s
> user 0m0.376s
> sys 0m0.084s
>
> rohit@precise-dev-102:~/tools/websocket-client-0.7.0$
>
> After running the program, the response time is quite high:
> $ time nova get-vnc-console c1b093a3-f53b-4282-b89c-e68f0fa1b6e5 novnc
>
>
> +-------+---------------------------------------------------------------------------------+
> | Type | Url
> |
>
> +-------+---------------------------------------------------------------------------------+
> | novnc |
> http://10.2.3.102:6080/vnc_auto.html?token=6865d675-d852-478b-b1ee-457b092f11b9|
>
> +-------+---------------------------------------------------------------------------------+
>
> real 3m9.231s
> user 0m0.424s
> sys 0m0.108s
>
>
> Possible solutions:
> 1. Allow just 1 session per instance, and raise a new exception, say,
> VNCSessionAlreadyExists to reject multiple
> connections for the same token, and return an error code to the user.
> 2. Make the number of sessions allowed per instance configurable,
> limited by some count of sessions.
>
> However, both of these solutions may need to override and modify the
> do_proxy() method of websockify's WebSocketProxy class,
> which can lead to maintenance issues.
>
> Another possible solution would be to implement some kind of callback
> function in websockify, to which we can pass the token
> for reconnection. This would first need contribution to the websockify
> project code, and then update Nova.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/1227575/+subscriptions
>
> _______________________________________________
> Openstack-security mailing list
> <email address hidden>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-security
>

--
Thanks,
-Sriram