Enumeration fails due to non-UTF8 characters returned

[Christopher Dearborn]

We've encountered a case where the iDRAC can return non-UTF8 characters in response to an enumeration query resulting in the following exception:

  File "/usr/lib/python2.7/site-packages/dracclient/client.py", line 371, in list_raid_controllers
    return self._raid_mgmt.list_raid_controllers()
  File "/usr/lib/python2.7/site-packages/dracclient/resources/raid.py", line 171, in list_raid_controllers
    doc = self.client.enumerate(uris.DCIM_ControllerView)
  File "/usr/lib/python2.7/site-packages/dracclient/client.py", line 633, in enumerate
    filter_query, filter_dialect)
  File "/usr/lib/python2.7/site-packages/dracclient/wsman.py", line 162, in enumerate
    resp_xml = ElementTree.fromstring(resp.content)
  File "lxml.etree.pyx", line 2993, in lxml.etree.fromstring (src/lxml/lxml.etree.c:68230)
  File "parser.pxi", line 1617, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:101481)
  File "parser.pxi", line 1495, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:100175)
  File "parser.pxi", line 1011, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:96380)
  File "parser.pxi", line 577, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:91687)
  File "parser.pxi", line 676, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:92962)
  File "parser.pxi", line 616, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:92190)
XMLSyntaxError: Input is not proper UTF-8, indicate encoding !
Bytes: 0xC0 0x26 0x23 0x31, line 2, column 1998

Investigation has shown that this is due to DCIM_ControllerView.DriverVersion containing characters that are not valid UTF8. This was specifically seen in the DriverVersion field of a BOSS card.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/579673
Committed: https://git.openstack.org/cgit/openstack/python-dracclient/commit/?id=0de8b417683527132f7ae6bc9a403a5a83b99488
Submitter: Zuul
Branch: master

commit 0de8b417683527132f7ae6bc9a403a5a83b99488
Author: Christopher Dearborn <email address hidden>
Date: Mon Jul 2 15:33:34 2018 -0400

    Filter out non-ASCII characters on invalid UTF8

    When an enumerate is done, it is possible that the iDRAC may return
    invalid UTF8 that contains non-ASCII characters. This causes an
    XMLSyntaxError to be thrown. This fix detects that situation and
    filters out all non-ASCII characters to bypass the error.

    See the following bug for further details:
    https://bugs.launchpad.net/python-dracclient/+bug/1779412

    Closes-Bug: #1779412
    Change-Id: I5003785dee922920dcdd95c8d7e2a26e0bf97a7d

[Niels Huylebroeck (red15)]

Would've been a simpler and cleaner solution if you had used:

`resp_xml = ElementTree.fromstring(resp.content.decode('utf8', errors='ignore')`