release-upgrade failed, report failed ImportError: cannot import name _coerceToFilesystemEncoding

[David Britton]

root@longs:/var/log/landscape# cat release-upgrader.log
2017-02-24 04:04:11,532 INFO [MainThread] Successfully fetched upgrade-tool files
2017-02-24 04:04:11,548 INFO [MainThread] Successfully verified upgrade-tool tarball
2017-02-24 04:29:41,479 INFO [MainThread] Queuing message with release upgrade results to exchange urgently.
2017-02-24 04:29:41,479 INFO [MainThread] Queuing message with release upgrade failure to exchange urgently.
2017-02-24 04:29:41,479 ERROR [MainThread] None
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
    current._debugInfo.failResult = None
  File "/usr/lib/python2.7/dist-packages/landscape/package/releaseupgrader.py", line 271, in abort
    return self._send_message(message)
  File "/usr/lib/python2.7/dist-packages/landscape/package/releaseupgrader.py", line 279, in _send_message
    deferred = self.get_session_id()
  File "/usr/lib/python2.7/dist-packages/landscape/package/taskhandler.py", line 248, in get_session_id
    result = self._broker.get_session_id()
  File "/usr/lib/python2.7/dist-packages/landscape/package/taskhandler.py", line 82, in wrapper
    result = self._connector.connect()
  File "/usr/lib/python2.7/dist-packages/landscape/amp.py", line 138, in connect
    self._connector = self._reactor.connect_unix(socket_path, factory)
  File "/usr/lib/python2.7/dist-packages/landscape/reactor.py", line 234, in connect_unix
    return self._reactor.connectUNIX(socket, factory)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 397, in connectUNIX
    p.startListening()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/unix.py", line 29, in <module>
    from twisted.python.filepath import _coerceToFilesystemEncoding
ImportError: cannot import name _coerceToFilesystemEncoding
2017-02-24 04:32:05,587 INFO [MainThread] Queuing message with release upgrade failure to exchange urgently.

Will attach all logs I have from the system.

[David Britton (dpb)]

client log files

[Francis Ginther (fginther)]

Reproduced this, logs attached.

[Francis Ginther (fginther)]

Reproduced this, logs attached.

[Francis Ginther (fginther)]

python-twisted-* packages get upgraded during the release upgrade from 13.2.0-1ubuntu1 to 16.0.0-1. The _coerceToFilesystemEncoding encoding is introduced in 16.0.0-1.

[Free Ekanayaka (free.ekanayaka)]

This happens because Twisted 13.2.0-1ubuntu1 in trusty has this transitional code:

    def listenUNIX(self, address, factory, backlog=50, mode=0o666, wantPID=0):
        assert unixEnabled, "UNIX support is not present"
        # Move this import back up to main level when twisted.internet.unix is
        # ported to Python 3:
        from twisted.internet import unix
        p = unix.Port(address, factory, backlog, mode, self, wantPID)
        p.startListening()
        return p

which means that the release-upgrader process will effectively import a mix of 13.2.0-1ubuntu1 and 16.0.0-1. Applying this diff to the current trusty landscape client should make the release upgrade succeed:

http://paste.ubuntu.com/24151525/

[Francis Ginther (fginther)]

Release upgrade works with the patch from Free. However, the landscape driven reboot activity after the upgrade failed with the message (in the UI):

Failed to set wall message, ignoring: Method "SetWallMessage" with signature "sb" on interface "org.freedesktop.login1.Manager" doesn't exist
Failed to call ScheduleShutdown in logind, proceeding with immediate shutdown: Method "ScheduleShutdown" with signature "st" on interface "org.freedesktop.login1.Manager" doesn't exist

[Free Ekanayaka (free.ekanayaka)]

logs?

[Francis Ginther (fginther)]

Someone else has now reported the same error with the reboot after upgrading to 16.04: lp:1670291. I'm grabbing the logs from my client and server.

[Francis Ginther (fginther)]

client logs

[Francis Ginther (fginther)]

server logs

[Free Ekanayaka (free.ekanayaka)]

This is arguably an Ubuntu bug (*maybe* uncovered by the specific way the client drives the Ubuntu release upgrade scripts, but unlikely given the but you linked and the reddit thread it points to).

I'd suggest to:

1) Perform the release upgrade activity from Landscape, which should succeed
2) Instead of triggering a reboot activity from Landscape, ssh to the client and run:

sudo /sbin/shutdown -r

which should reproduce the failure.

At this point we escalate this to Ubuntu and ask if it can be fixed or come up with a workaround if they can't fix it.

[Francis Ginther (fginther)]

@Free, you're correct, /sbin/shutdown also fails:

ubuntu@sekine:~$ sudo /sbin/shutdown -r
Failed to set wall message, ignoring: Method "SetWallMessage" with signature "sb" on interface "org.freedesktop.login1.Manager" doesn't exist
Failed to call ScheduleShutdown in logind, proceeding with immediate shutdown: Method "ScheduleShutdown" with signature "st" on interface "org.freedesktop.login1.Manager" doesn't exist

I'll use lp:1670291 to track this issue. It's not landscape-client related.