OpenStack Compute (Nova)

Console-Output does not escape XML correctly

Reported by justinsb on 2012-02-23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Medium
justinsb

Bug Description

The console output does not escape characters in the output correctly, when XML output is selected.

Here's an example error stack trace from Java, the important bit is "An invalid XML character (Unicode: 0x1b) was found in the element content of the document.]":

Error running command: get-consoleoutput adb2a214-8728-45d8-ad4d-4ee28a9124f6
javax.ws.rs.WebApplicationException: javax.xml.bind.UnmarshalException
 - with linked exception:
[org.xml.sax.SAXParseException; lineNumber: 335; columnNumber: 36; An invalid XML character (Unicode: 0x1b) was found in the element content of the document.]
 at com.sun.jersey.core.provider.jaxb.AbstractRootElementProvider.readFrom(AbstractRootElementProvider.java:113)
 at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:554)
 at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
 at com.sun.jersey.api.client.WebResource.handle(WebResource.java:674)
 at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
 at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:563)
 at org.openstack.client.compute.ServerResource.executeAction(ServerResource.java:361)
 at org.openstack.client.compute.ServerResource.getConsoleOutput(ServerResource.java:355)
 at org.openstack.client.cli.commands.GetConsoleOutput.runCommand(GetConsoleOutput.java:21)
 at com.fathomdb.cli.CliSimpleRepl.executeCommand(CliSimpleRepl.java:152)
 at com.fathomdb.cli.CliSimpleRepl.runCommand(CliSimpleRepl.java:90)
 at com.fathomdb.cli.CliBase.mainWithReturnCode(CliBase.java:138)
 at com.fathomdb.cli.CliBase.nailMain(CliBase.java:61)
 at org.openstack.client.cli.OpenstackCli.nailMain(OpenstackCli.java:43)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:601)
 at com.martiansoftware.nailgun.NGSession.run(NGSession.java:298)
Caused by: javax.xml.bind.UnmarshalException
 - with linked exception:
[org.xml.sax.SAXParseException; lineNumber: 335; columnNumber: 36; An invalid XML character (Unicode: 0x1b) was found in the element content of the document.]
 at javax.xml.bind.helpers.AbstractUnmarshallerImpl.createUnmarshalException(AbstractUnmarshallerImpl.java:335)
 at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.createUnmarshalException(UnmarshallerImpl.java:523)
 at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:220)
 at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:189)
 at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:140)
 at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:123)
 at com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider.readFrom(XMLRootElementProvider.java:115)
 at com.sun.jersey.core.provider.jaxb.AbstractRootElementProvider.readFrom(AbstractRootElementProvider.java:111)
 ... 18 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 335; columnNumber: 36; An invalid XML character (Unicode: 0x1b) was found in the element content of the document.
 at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198)
 at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
 at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)
 at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368)
 at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1375)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2854)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:607)
 at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:116)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:488)
 at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:835)
 at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
 at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:123)
 at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1210)
 at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:568)
 at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:216)
 ... 23 more

Thierry Carrez (ttx) on 2012-02-24
Changed in nova:
importance: Undecided → Medium
status: New → Confirmed

Fix proposed to branch: master
Review: https://review.openstack.org/5545

Changed in nova:
assignee: nobody → justinsb (justin-fathomdb)
status: Confirmed → In Progress
justinsb (justin-fathomdb) wrote :

Vishy and I discussed this on IRC.

It's down to a bug in MiniDom, and affects all XML output.
Any string that outputs characters < 32 should really be escaped. Minidom doesn't do this: http://bugs.python.org/issue5752
I don't believe this has any e.g. security consequences other than the fact that we produce invalid XML. i.e. this isn't as bad as escape in HTML.
We probably want to move to lxml anyway. This may fix the problem for free.
It doesn't seem like a good idea to mess with all XML output at this stage.
Console output is particularly problematic because ANSI sequences start with 0x1B.
The patch therefore masks the problem for console-output (only)
Personally, I don't mind if the patch goes in or not; I can easily switch between using JSON and using XML in my binding.
In Folsom, we should fix this properly (with lxml or otherwise)

Reviewed: https://review.openstack.org/5545
Committed: http://github.com/openstack/nova/commit/13ca1b923e479905f9d1debdea6125be92f44808
Submitter: Jenkins
Branch: master

commit 13ca1b923e479905f9d1debdea6125be92f44808
Author: Justin Santa Barbara <email address hidden>
Date: Mon Mar 19 16:33:16 2012 -0700

    Strip out characters that should be escaped from console output

    Minidom has a bug where it doesn't escape characters < 32 correctly.
    That is http://bugs.python.org/issue5752

    We probably want to move to lxml anyway.

    Console-output is particularly likely to have invalid characters,
    because ANSI output include the ESC character (0x1B). Strip out
    invalid characters for this case (although the problem is still
    present anywhere we write a string in the XML)

    Bug #939386

    Change-Id: I4689c0f26fb4887867a8d5b6f5e5fb5e2590be91

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2012-03-20
Changed in nova:
milestone: none → essex-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in nova:
milestone: essex-rc1 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.