Functional gate job broken due to os-log-merger crashing on null-bytes

[Manjeet Singh Bhatia]

gate-neutron-dsvm-functional-ubuntu-xenial seems to be failing even though tests are being passed.

1. http://logs.openstack.org/53/319353/3/check/gate-neutron-dsvm-functional-ubuntu-xenial/b36e01e/
2. http://logs.openstack.org/25/338625/33/check/gate-neutron-dsvm-functional-ubuntu-xenial/fc680a7/

[Manjeet Singh Bhatia (manjeet-s-bhatia)]

oslomerger looks like culprit

017-03-14 21:05:35.372834 | 2017-03-14 21:05:35.372 | sys.exit(main())
2017-03-14 21:05:35.373974 | 2017-03-14 21:05:35.373 | File "/tmp/os-log-merger/local/lib/python2.7/site-packages/oslogmerger/oslogmerger.py", line 648, in main
2017-03-14 21:05:35.375069 | 2017-03-14 21:05:35.374 | process_logs(cfg)
2017-03-14 21:05:35.376277 | 2017-03-14 21:05:35.375 | File "/tmp/os-log-merger/local/lib/python2.7/site-packages/oslogmerger/oslogmerger.py", line 382, in process_logs
2017-03-14 21:05:35.377377 | 2017-03-14 21:05:35.377 | for entry in method(logs):
2017-03-14 21:05:35.378502 | 2017-03-14 21:05:35.378 | File "/tmp/os-log-merger/local/lib/python2.7/site-packages/oslogmerger/oslogmerger.py", line 349, in process_logs_memory_hog
2017-03-14 21:05:35.379766 | 2017-03-14 21:05:35.379 | for entry in log:
2017-03-14 21:05:35.380930 | 2017-03-14 21:05:35.380 | File "/tmp/os-log-merger/local/lib/python2.7/site-packages/oslogmerger/oslogmerger.py", line 230, in next
2017-03-14 21:05:35.382000 | 2017-03-14 21:05:35.381 | self.entry, self.next_entry = self._readline(self.next_entry)
2017-03-14 21:05:35.383102 | 2017-03-14 21:05:35.382 | File "/tmp/os-log-merger/local/lib/python2.7/site-packages/oslogmerger/oslogmerger.py", line 213, in _readline
2017-03-14 21:05:35.384124 | 2017-03-14 21:05:35.383 | **self.entry_kwargs)
2017-03-14 21:05:35.385297 | 2017-03-14 21:05:35.385 | File "/tmp/os-log-merger/local/lib/python2.7/site-packages/oslogmerger/oslogmerger.py", line 129, in factory
2017-03-14 21:05:35.386391 | 2017-03-14 21:05:35.386 | self.date = self.parse_date(prepared_line)
2017-03-14 21:05:35.387540 | 2017-03-14 21:05:35.387 | File "/tmp/os-log-merger/local/lib/python2.7/site-packages/oslogmerger/oslogmerger.py", line 97, in parse_date
2017-03-14 21:05:35.388630 | 2017-03-14 21:05:35.388 | dt = datetime.strptime(line, self.date_format)
2017-03-14 21:05:35.389844 | 2017-03-14 21:05:35.389 | TypeError: strptime() argument 1 must be string without null bytes, not str

[Ihar Hrachyshka (ihar-hrachyshka)]

https://review.openstack.org/#/q/topic:fix-os-log-merger-crash

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/445884

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/445884
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=75612f200b984c9f42558ed951ae19e17c3633d9
Submitter: Jenkins
Branch: master

commit 75612f200b984c9f42558ed951ae19e17c3633d9
Author: Miguel Angel Ajo <email address hidden>
Date: Wed Mar 15 12:29:32 2017 +0100

    Bump os-log-merger version in post_test_hook

    The 1.0.6 version of os-log-merger exploded in cases where
    log files contained \0 characters. 1.1.0 fixes such issue.

    Change-Id: I470d38dfd5dd7e9cf6317d9f2f32875f33f3a996
    Closes-Bug: #1672921

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/445706
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9c7ca5cc020a93a65fc817ae960d221d43e0e924
Submitter: Jenkins
Branch: master

commit 9c7ca5cc020a93a65fc817ae960d221d43e0e924
Author: Ihar Hrachyshka <email address hidden>
Date: Wed Mar 8 05:30:04 2017 +0000

    Don't return null-byte separated string from ExternalProcess.cmdline()

    Instead construct the string using psutil.Process.cmdline() method that
    will parse the data from kernel for us and will return a proper
    space-separated command line (plus will provide some more safety
    guarantees, e.g. against zombies).

    This change also introduces a new psutil dependency for neutron. Before
    that, the library was used in functional tests only, and with a minimal
    version number that is not compatible with global-requirements.txt or
    .cmdline(). Moved the requirement into runtime requirements.

    Related-Bug: #1672921
    Change-Id: I780ea10bd1870c86c1a048071863843bc2993958

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.0.0b1 development milestone.