buildd-retry-depwait crashes intermittently

Bug #1980703 reported by Paul Collins
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
High
Unassigned

Bug Description

We are alerted if this job hasn't run successfully in 6 hours. It seems to crash now and then as follows:

2022-07-04 23:54:28 ERROR Unhandled exception
Traceback (most recent call last):
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/services/scripts/base.py", line 108, in log_unhandled_exceptions_func
    return func(self, *args, **kw)
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/services/scripts/base.py", line 376, in lock_and_run
    self.run(use_web_security=use_web_security, isolation=isolation)
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/services/scripts/base.py", line 108, in log_unhandled_exceptions_func
    return func(self, *args, **kw)
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/services/scripts/base.py", line 330, in run
    self.main()
  File "/srv/launchpad.net/codelines/current/cronscripts/buildd-retry-depwait.py", line 22, in main
    updater.run()
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/services/looptuner.py", line 391, in run
    log=self.log).run()
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/services/looptuner.py", line 218, in run
    reraise(exc_info[0], exc_info[1], tb=exc_info[2])
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/env/lib/python3.5/site-packages/six.py", line 703, in reraise
    raise value
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/services/looptuner.py", line 169, in run
    self.operation(chunk_size)
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/soyuz/scripts/retrydepwait.py", line 70, in __call__
    build.updateDependencies()
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/soyuz/model/binarypackagebuild.py", line 596, in updateDependencies
    if not any(self._isDependencySatisfied(token) for token in or_dep):
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/soyuz/model/binarypackagebuild.py", line 596, in <genexpr>
    if not any(self._isDependencySatisfied(token) for token in or_dep):
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/lib/lp/soyuz/model/binarypackagebuild.py", line 568, in _isDependencySatisfied
    for dep_candidate in dep_candidates:
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/env/lib/python3.5/site-packages/storm/store.py", line 1024, in __iter__
    result = self._store._connection.execute(self._get_select())
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/env/lib/python3.5/site-packages/storm/databases/postgres.py", line 306, in execute
    return Connection.execute(self, statement, params, noresult)
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/env/lib/python3.5/site-packages/storm/database.py", line 323, in execute
    raw_cursor = self.raw_execute(statement, params)
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/env/lib/python3.5/site-packages/storm/database.py", line 457, in raw_execute
    self._run_execution(raw_cursor, args, params, statement)
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/env/lib/python3.5/site-packages/storm/database.py", line 471, in _run_execution
    self._check_disconnect(raw_cursor.execute, *args)
  File "/srv/launchpad.net/codelines/soyuz-production-rev-34bab3ec0b1fcebb63455ce9ce19de47fff774bb/env/lib/python3.5/site-packages/storm/database.py", line 542, in _check_disconnect
    raise DisconnectionError(str(exc))
storm.exceptions.DisconnectionError: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

2022-07-04 23:54:28 INFO OOPS-66fae80e66986fcd8195f23c1dd46286

Also:

2022-07-03 02:31:24 INFO OOPS-0e8d43d308dde84bc87bf2338add1078

Guruprasad (lgp171188)
Changed in launchpad:
importance: Undecided → High
Guruprasad (lgp171188)
Changed in launchpad:
status: New → Triaged
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.