Activity log for bug #2054808

Date Who What changed Old value New value Message
2024-02-23 12:09:41 Adam Collard bug added bug
2024-02-23 12:09:54 Adam Collard description eb 23 12:05:35 maas-system-maas maas-rackd[9499]: provisioningserver.rpc.common: [critical] Unhandled failure dispatching AMP command. This is probably a bug. Please ensure that this error is handled within application code or declared in the signature of the b'EvaluateTag' command. [maas-system-maas:pid=9499:cmd=E valuateTag:ask=4] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: Traceback (most recent call last): Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py", line 271, in _onTimer Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self.runUntilCurrent() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/base.py", line 991, in runUntilCurrent Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: call.func(*call.args, **call.kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 700, in errback Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self._startRunCallbacks(fail) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self._runCallbacks() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: --- <exception caught here> --- Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 857, in _runCallbacks Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: current.result = callback( # type: ignore[misc] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 1138, in checkKnownErrors Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: key = error.trap(*command.allErrors) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/failure.py", line 451, in trap Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self.raiseException() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/failure.py", line 475, in raiseException Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: raise self.value.with_traceback(self.tb) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 244, in inContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = inContext.theWork() # type: ignore[attr-defined] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 260, in <lambda> Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: inContext.theWork = lambda: context.call( # type: ignore[attr-defined] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/context.py", line 117, in callWithContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.currentContext().callWithContext(ctx, func, *args, **kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/context.py", line 82, in callWithContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return func(*args, **kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 203, in wrapper Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args, **kwargs) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/rpc/tags.py", line 39, in evaluate_tag Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: process_node_tags( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 339, in process_node_tags Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: process_all( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 303, in process_all Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: nodes_matched, nodes_unmatched = classify( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/utils/__init__.py", line 75, in classify Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: for ident, subject in subjects: Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 279, in gen_node_details Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: for system_id, details in get_details(batch).items(): Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 78, in get_details_for_nodes Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: data = process_response(client.get(path, op="details")) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/apiclient/maas_client.py", line 270, in get Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.dispatcher.dispatch_query( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/apiclient/maas_client.py", line 122, in dispatch_query Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: res = opener.open(req) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 525, in open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = meth(req, response) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = self.parent.error( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 557, in error Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = self._call_chain(*args) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args) Feb 23 12:05:35 maas-system-maas maas-log[9784]: maas.tags: [error] Tag all (true()) could not be evaluated on rack controller maas-system-maas (nbmwp6): (b'UNHANDLED', 'Unknown Error [maas-system-maas:pid=9499:cmd=EvaluateTag:ask=4]') Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 749, in http_error_302 Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.parent.open(new, timeout=req.timeout) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 519, in open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = self._open(req, data) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 536, in _open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = self._call_chain(self.handle_open, protocol, protocol + Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 1391, in https_open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.do_open(http.client.HTTPSConnection, req, Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 1351, in do_open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: raise URLError(err) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)> Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: provisioningserver.rpc.common: [critical] Unhandled failure dispatching AMP command. This is probably a bug. Please ensure that this error is handled within application code or declared in the signature of the b'EvaluateTag' command. [maas-system-maas:pid=9499:cmd=E valuateTag:ask=4] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: Traceback (most recent call last): Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py", line 271, in _onTimer Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self.runUntilCurrent() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/base.py", line 991, in runUntilCurrent Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: call.func(*call.args, **call.kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 700, in errback Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self._startRunCallbacks(fail) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self._runCallbacks() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: --- <exception caught here> --- Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 857, in _runCallbacks Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: current.result = callback( # type: ignore[misc] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 1138, in checkKnownErrors Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: key = error.trap(*command.allErrors) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/failure.py", line 451, in trap Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self.raiseException() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/failure.py", line 475, in raiseException Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: raise self.value.with_traceback(self.tb) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 244, in inContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = inContext.theWork() # type: ignore[attr-defined] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 260, in <lambda> Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: inContext.theWork = lambda: context.call( # type: ignore[attr-defined] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/context.py", line 117, in callWithContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.currentContext().callWithContext(ctx, func, *args, **kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/context.py", line 82, in callWithContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return func(*args, **kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 203, in wrapper Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args, **kwargs) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/rpc/tags.py", line 39, in evaluate_tag Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: process_node_tags( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 339, in process_node_tags Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: process_all( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 303, in process_all Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: nodes_matched, nodes_unmatched = classify( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/utils/__init__.py", line 75, in classify Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: for ident, subject in subjects: Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 279, in gen_node_details Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: for system_id, details in get_details(batch).items(): Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 78, in get_details_for_nodes Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: data = process_response(client.get(path, op="details")) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/apiclient/maas_client.py", line 270, in get Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.dispatcher.dispatch_query( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/apiclient/maas_client.py", line 122, in dispatch_query Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: res = opener.open(req) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 525, in open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = meth(req, response) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = self.parent.error( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 557, in error Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = self._call_chain(*args) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args) Feb 23 12:05:35 maas-system-maas maas-log[9784]: maas.tags: [error] Tag all (true()) could not be evaluated on rack controller maas-system-maas (nbmwp6): (b'UNHANDLED', 'Unknown Error [maas-system-maas:pid=9499:cmd=EvaluateTag:ask=4]') Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 749, in http_error_302 Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.parent.open(new, timeout=req.timeout) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 519, in open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = self._open(req, data) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 536, in _open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = self._call_chain(self.handle_open, protocol, protocol + Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 1391, in https_open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.do_open(http.client.HTTPSConnection, req, Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 1351, in do_open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: raise URLError(err) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)>
2024-02-23 12:10:20 Adam Collard description Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: provisioningserver.rpc.common: [critical] Unhandled failure dispatching AMP command. This is probably a bug. Please ensure that this error is handled within application code or declared in the signature of the b'EvaluateTag' command. [maas-system-maas:pid=9499:cmd=E valuateTag:ask=4] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: Traceback (most recent call last): Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py", line 271, in _onTimer Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self.runUntilCurrent() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/base.py", line 991, in runUntilCurrent Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: call.func(*call.args, **call.kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 700, in errback Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self._startRunCallbacks(fail) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self._runCallbacks() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: --- <exception caught here> --- Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 857, in _runCallbacks Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: current.result = callback( # type: ignore[misc] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 1138, in checkKnownErrors Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: key = error.trap(*command.allErrors) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/failure.py", line 451, in trap Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self.raiseException() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/failure.py", line 475, in raiseException Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: raise self.value.with_traceback(self.tb) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 244, in inContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = inContext.theWork() # type: ignore[attr-defined] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 260, in <lambda> Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: inContext.theWork = lambda: context.call( # type: ignore[attr-defined] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/context.py", line 117, in callWithContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.currentContext().callWithContext(ctx, func, *args, **kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/context.py", line 82, in callWithContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return func(*args, **kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 203, in wrapper Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args, **kwargs) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/rpc/tags.py", line 39, in evaluate_tag Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: process_node_tags( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 339, in process_node_tags Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: process_all( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 303, in process_all Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: nodes_matched, nodes_unmatched = classify( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/utils/__init__.py", line 75, in classify Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: for ident, subject in subjects: Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 279, in gen_node_details Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: for system_id, details in get_details(batch).items(): Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 78, in get_details_for_nodes Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: data = process_response(client.get(path, op="details")) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/apiclient/maas_client.py", line 270, in get Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.dispatcher.dispatch_query( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/apiclient/maas_client.py", line 122, in dispatch_query Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: res = opener.open(req) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 525, in open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = meth(req, response) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = self.parent.error( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 557, in error Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = self._call_chain(*args) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args) Feb 23 12:05:35 maas-system-maas maas-log[9784]: maas.tags: [error] Tag all (true()) could not be evaluated on rack controller maas-system-maas (nbmwp6): (b'UNHANDLED', 'Unknown Error [maas-system-maas:pid=9499:cmd=EvaluateTag:ask=4]') Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 749, in http_error_302 Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.parent.open(new, timeout=req.timeout) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 519, in open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = self._open(req, data) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 536, in _open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = self._call_chain(self.handle_open, protocol, protocol + Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 1391, in https_open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.do_open(http.client.HTTPSConnection, req, Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 1351, in do_open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: raise URLError(err) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)> https://paste.ubuntu.com/p/JhfPdXHZGw/ Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: provisioningserver.rpc.common: [critical] Unhandled failure dispatching AMP command. This is probably a bug. Please ensure that this error is handled within application code or declared in the signature of the b'EvaluateTag' command. [maas-system-maas:pid=9499:cmd=E valuateTag:ask=4] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: Traceback (most recent call last): Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/asyncioreactor.py", line 271, in _onTimer Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self.runUntilCurrent() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/base.py", line 991, in runUntilCurrent Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: call.func(*call.args, **call.kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 700, in errback Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self._startRunCallbacks(fail) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self._runCallbacks() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: --- <exception caught here> --- Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 857, in _runCallbacks Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: current.result = callback( # type: ignore[misc] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/protocols/amp.py", line 1138, in checkKnownErrors Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: key = error.trap(*command.allErrors) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/failure.py", line 451, in trap Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: self.raiseException() Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/failure.py", line 475, in raiseException Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: raise self.value.with_traceback(self.tb) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 244, in inContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = inContext.theWork() # type: ignore[attr-defined] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 260, in <lambda> Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: inContext.theWork = lambda: context.call( # type: ignore[attr-defined] Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/context.py", line 117, in callWithContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.currentContext().callWithContext(ctx, func, *args, **kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/usr/lib/python3/dist-packages/twisted/python/context.py", line 82, in callWithContext Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return func(*args, **kw) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/utils/twisted.py", line 203, in wrapper Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args, **kwargs) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/rpc/tags.py", line 39, in evaluate_tag Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: process_node_tags( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 339, in process_node_tags Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: process_all( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 303, in process_all Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: nodes_matched, nodes_unmatched = classify( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/utils/__init__.py", line 75, in classify Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: for ident, subject in subjects: Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 279, in gen_node_details Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: for system_id, details in get_details(batch).items(): Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/provisioningserver/tags.py", line 78, in get_details_for_nodes Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: data = process_response(client.get(path, op="details")) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/apiclient/maas_client.py", line 270, in get Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.dispatcher.dispatch_query( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/snap/maas/33772/lib/python3.10/site-packages/apiclient/maas_client.py", line 122, in dispatch_query Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: res = opener.open(req) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 525, in open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = meth(req, response) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = self.parent.error( Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 557, in error Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = self._call_chain(*args) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args) Feb 23 12:05:35 maas-system-maas maas-log[9784]: maas.tags: [error] Tag all (true()) could not be evaluated on rack controller maas-system-maas (nbmwp6): (b'UNHANDLED', 'Unknown Error [maas-system-maas:pid=9499:cmd=EvaluateTag:ask=4]') Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 749, in http_error_302 Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.parent.open(new, timeout=req.timeout) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 519, in open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: response = self._open(req, data) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 536, in _open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = self._call_chain(self.handle_open, protocol, protocol + Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: result = func(*args) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 1391, in https_open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: return self.do_open(http.client.HTTPSConnection, req, Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: File "/usr/lib/python3.10/urllib/request.py", line 1351, in do_open Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: raise URLError(err) Feb 23 12:05:35 maas-system-maas maas-rackd[9499]: urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)>
2024-02-23 17:58:23 Alexsander de Souza maas: status New Triaged
2024-02-23 17:58:28 Alexsander de Souza maas: milestone 3.5.0
2024-02-23 17:58:31 Alexsander de Souza maas: importance Undecided High
2024-03-14 09:48:22 Thorsten Merten maas: assignee Anton Troyanov (troyanov)
2024-03-19 16:05:12 Anton Troyanov nominated for series maas/3.6
2024-03-19 16:05:12 Anton Troyanov bug task added maas/3.6
2024-03-19 16:05:18 Anton Troyanov bug task deleted maas/3.6
2024-03-19 16:05:23 Anton Troyanov nominated for series maas/3.5
2024-03-19 16:05:23 Anton Troyanov bug task added maas/3.5
2024-03-19 16:05:28 Anton Troyanov maas/3.5: status New Triaged
2024-03-19 16:05:31 Anton Troyanov maas/3.5: importance Undecided High
2024-03-19 16:05:33 Anton Troyanov maas/3.5: assignee Anton Troyanov (troyanov)
2024-03-19 16:05:36 Anton Troyanov maas: milestone 3.5.0 3.6.0
2024-03-19 16:05:37 Anton Troyanov maas/3.5: milestone 3.5.0
2024-03-20 14:37:18 Anton Troyanov maas/3.5: status Triaged In Progress
2024-03-20 14:37:22 Anton Troyanov maas: status Triaged In Progress
2024-03-25 21:59:10 Anton Troyanov merge proposal linked https://code.launchpad.net/~troyanov/maas/+git/maas/+merge/463060
2024-03-26 16:04:50 MAAS Lander maas: status In Progress Fix Committed
2024-03-28 09:42:39 Anton Troyanov merge proposal linked https://code.launchpad.net/~troyanov/maas/+git/maas/+merge/463148
2024-03-28 09:42:45 Anton Troyanov maas/3.5: status In Progress Fix Committed