Client lost registration

[Andreas Hasenack]

I don't have enough yet to reproduce the problem, or even understand how it happened, so I will just describe what happened.

A computer was registered against production and exchanging just fine. Then we had a production incident (https://bugs.launchpad.net/landscape/+bug/1293129) and all computers became wedged.

This client was showing 503 errors in the logs while we fixed production. Once production was fixed, it was overloaded for a while and the client intermitently logged 503 and 502 errors. Then, all of a sudden, it lost the registration:

2014-03-17 19:34:12,735 ERROR [PoolThread-twisted.internet.reactor-2] Error contacting the server at https://landscape.canonical.com/message-system.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/landscape/broker/transport.py", line 71, in exchange
    message_api)
  File "/usr/lib/python2.7/dist-packages/landscape/broker/transport.py", line 45, in _curl
    headers=headers, cainfo=self._pubkey, curl=curl))
  File "/usr/lib/python2.7/dist-packages/landscape/lib/fetch.py", line 104, in fetch
    raise HTTPCodeError(http_code, body)
HTTPCodeError: Server returned HTTP code 503
2014-03-17 19:34:12,737 INFO [MainThread] Message exchange failed.
2014-03-17 19:34:12,738 INFO [MainThread] Message exchange completed in 60.76s.
2014-03-17 19:34:17,011 INFO [MainThread] Ping indicates message available. Scheduling an urgent exchange.
2014-03-17 19:34:47,281 INFO [MainThread] Ping indicates message available. Scheduling an urgent exchange.
2014-03-17 19:35:12,768 INFO [MainThread] Starting urgent message exchange with https://landscape.canonical.com/message-system.
2014-03-17 19:37:09,843 ERROR [PoolThread-twisted.internet.reactor-1] Error contacting the server at https://landscape.canonical.com/message-system.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/landscape/broker/transport.py", line 71, in exchange
    message_api)
  File "/usr/lib/python2.7/dist-packages/landscape/broker/transport.py", line 45, in _curl
    headers=headers, cainfo=self._pubkey, curl=curl))
  File "/usr/lib/python2.7/dist-packages/landscape/lib/fetch.py", line 104, in fetch
    raise HTTPCodeError(http_code, body)
HTTPCodeError: Server returned HTTP code 502
2014-03-17 19:37:09,844 INFO [MainThread] Message exchange failed.
2014-03-17 19:37:09,844 INFO [MainThread] Message exchange completed in 117.08s.
2014-03-17 19:38:09,877 INFO [MainThread] Starting urgent message exchange with https://landscape.canonical.com/message-system.
2014-03-17 19:39:37,234 INFO [PoolThread-twisted.internet.reactor-1] Sent 39214 bytes and received 217 bytes in 87.35s.
2014-03-17 19:39:37,239 INFO [MainThread] Switching to normal exchange mode.
2014-03-17 19:39:37,332 INFO [MainThread] Pending messages remain after the last exchange.
2014-03-17 19:39:37,333 INFO [MainThread] Message exchange completed in 87.46s.
2014-03-17 19:40:37,407 INFO [MainThread] Starting urgent message exchange with https://landscape.canonical.com/message-system.
2014-03-17 19:40:38,265 INFO [PoolThread-twisted.internet.reactor-2] Sent 37803 bytes and received 219 bytes in 0.85s.
2014-03-17 19:40:38,268 INFO [MainThread] Switching to normal exchange mode.
2014-03-17 19:40:39,660 INFO [MainThread] Accepted types changed: +register +register-provisioned-machine +register-cloud-vm test -computer-info -operation-result -c
eph-usage -memory-info -mount-info -text-message -network-activity -mount-activity -custom-graph -active-process-info -update-manager-info -reboot-required -change-packa
ges-result -temperature -apt-preferences -juju-info -keystone-token -users -reboot-required-info -cpu-usage -package-reporter-result -hardware-info -load-average -packag
es -unknown-package-hashes -add-packages -cloud-instance-metadata -free-space -resynchronize -processor-info -swift-device-info -network-device -distribution-info
2014-03-17 19:40:39,712 INFO [MainThread] Client is clone of computer bach
2014-03-17 19:40:39,791 INFO [MainThread] Queueing message to register with account 'andreas' with a password.
2014-03-17 19:40:39,795 INFO [MainThread] Starting message exchange with https://landscape.canonical.com/message-system.
2014-03-17 19:40:39,796 INFO [MainThread] Message exchange completed in 2.39s.

Full broker.log attached.

[🤖 Landscape Builder (landscape-builder)]

This bug has not seen any activity in the last 6 months, so it is being automatically closed.

If you are still experiencing this issue, please feel free to re-open.

Landscape Team

[Dhanesh (dyadav)]

Hi Team,

i am not bale to register client serve to landscape server i am getting below error every time while registering client please help me on this

PyCurlError: Error 56: Received HTTP code 407 from proxy after CONNECT
2016-03-21 15:15:48,382 INFO [MainThread] Message exchange failed.
2016-03-21 15:15:48,382 INFO [MainThread] Message exchange completed in 0.01s.
2016-03-21 15:15:58,694 INFO [MainThread] Broker stopped with config /etc/landscape/client.conf
2016-03-21 15:16:00,061 INFO [MainThread] Broker started with config /etc/landscape/client.conf
2016-03-21 15:16:00,569 INFO [MainThread] Starting urgent message exchange with https://abc.xyz.com/message-system
2016-03-21 15:16:00,585 ERROR [PoolThread-twisted.internet.reactor-1] Error contacting the server at https://abc.xyz.com/message-system.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/landscape/broker/transport.py", line 71, in exchange
    message_api)
  File "/usr/lib/python2.7/dist-packages/landscape/broker/transport.py", line 45, in _curl
    headers=headers, cainfo=self._pubkey, curl=curl))
  File "/usr/lib/python2.7/dist-packages/landscape/lib/fetch.py", line 109, in fetch
    raise PyCurlError(e.args[0], e.args[1])
PyCurlError: Error 56: Received HTTP code 407 from proxy after CONNECT
2016-03-21 15:16:00,586 INFO [MainThread] Message exchange failed.
2016-03-21 15:16:00,587 INFO [MainThread] Message exchange completed in 0.02s.
2016-03-21 15:17:00,588 INFO [MainThread] Starting urgent message exchange with https://abc.xyz.com/message-system
2016-03-21 15:17:00,599 ERROR [PoolThread-twisted.internet.reactor-1] Error contacting the server at https://abc.xyz.com/message-system
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/landscape/broker/transport.py", line 71, in exchange
    message_api)
  File "/usr/lib/python2.7/dist-packages/landscape/broker/transport.py", line 45, in _curl
    headers=headers, cainfo=self._pubkey, curl=curl))
  File "/usr/lib/python2.7/dist-packages/landscape/lib/fetch.py", line 109, in fetch
    raise PyCurlError(e.args[0], e.args[1])
PyCurlError: Error 56: Received HTTP code 407 from proxy after CONNECT
2016-03-21 15:17:00,600 INFO [MainThread] Message exchange failed.
2016-03-21 15:17:00,600 INFO [MainThread] Message exchange completed in 0.01s.

[Andreas Hasenack (ahasenack)]

HTTP 407 means "407 Proxy Authentication Required (RFC 7235)". Looks like your proxy configuration changed and it's now requiring authentication.