parse-ppa-apache-logs failing (missing files)

Bug #951401 reported by Stuart Bishop on 2012-03-10
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
High
Stuart Bishop

Bug Description

parse-ppa-apache-access-logs.py on production is regularly failing with:

2012-03-10 07:06:49 ERROR Unhandled exception
 -> http://launchpadlibrarian.net/96185222/s9aeNuxZ3ghdgtbnm53cwfdUWYQ.txt ([Errno 2] No such file or directory: '/var/log/apache2/ppa.launchpad.net-access.log-20110912.gz')

The traceback is:

Traceback (most recent call last):
  File "/srv/launchpad.net/production/launchpad-rev-14914/lib/lp/services/scripts/base.py", line 103, in log_unhandled_exceptions_func
    return func(self, *args, **kw)
  File "/srv/launchpad.net/production/launchpad-rev-14914/lib/lp/services/scripts/base.py", line 387, in lock_and_run
    use_web_security=use_web_security, isolation=isolation)
  File "/srv/launchpad.net/production/launchpad-rev-14914/lib/lp/services/scripts/base.py", line 103, in log_unhandled_exceptions_func
    return func(self, *args, **kw)
  File "/srv/launchpad.net/production/launchpad-rev-14914/lib/lp/services/scripts/base.py", line 340, in run
    self.main()
  File "/srv/launchpad.net/production/launchpad-rev-14914/lib/lp/services/apachelogparser/script.py", line 79, in main
    for fd, position in files_to_parse:
  File "/srv/launchpad.net/production/launchpad-rev-14914/lib/lp/services/apachelogparser/base.py", line 39, in get_files_to_parse
    fd, file_size = get_fd_and_file_size(file_path)
  File "/srv/launchpad.net/production/launchpad-rev-14914/lib/lp/services/apachelogparser/base.py", line 71, in get_fd_and_file_size
    fd = gzip.open(file_path)
  File "/usr/lib/python2.6/gzip.py", line 33, in open
    return GzipFile(filename, mode, compresslevel)
  File "/usr/lib/python2.6/gzip.py", line 79, in __init__
    fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: '/var/log/apache2/ppa.launchpad.net-access.log-20110912.gz'

Hopefully, this error is spurious in which case the code needs to be updated to cope and not report errors (even if it does signal a production issue, such as files being removed before they can be processed, WARNING would be a better level).

Related branches

Stuart Bishop (stub) on 2012-03-10
Changed in launchpad:
status: New → Triaged
importance: Undecided → High
Julian Edwards (julian-edwards) wrote :

I wonder if it's a race condition with log rotation?

William Grant (wgrant) wrote :

That's been my assumption.

On Mon, Mar 12, 2012 at 8:08 AM, William Grant <email address hidden> wrote:

> That's been my assumption.

Given that the missing file is 6 months old, I think we can assume it
has already been processed and we are not losing data.

--
Stuart Bishop <email address hidden>

Stuart Bishop (stub) on 2012-03-12
Changed in launchpad:
status: Triaged → In Progress
assignee: nobody → Stuart Bishop (stub)
Launchpad QA Bot (lpqabot) wrote :
tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
Stuart Bishop (stub) on 2012-03-14
tags: added: qa-untestable
removed: qa-needstesting
William Grant (wgrant) on 2012-03-15
Changed in launchpad:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers