interrupted core file reception results in lack of resubmission

[Brian Murray]

I was looking at some worker timeout issues today and noticed the following in the daisy server app log.

2017-01-09 16:55:04 [15792] [INFO] (5e0e1c5e-d68c-11e6-abd2-fa163e171d9b) inserted into OOPS CF
2017-01-09 16:55:04 [15792] [INFO] (5e0e1c5e-d68c-11e6-abd2-fa163e171d9b) asked for CORE
2017-01-09 16:55:05 [15792] [INFO] swift_token: 84a5569f689949d8921292e275c9f8b3
2017-01-09 16:55:07 [15792] [WARNING] Worker graceful timeout (pid:15792)

The worker graceful shuts down due to reaching a timeout but we never finish receiving the core file e.g. we don't see 'has an 1234 byte core file' and 'CORE * written to bucket'.

I'd hope ask for the core again but grep'ing the log files we don't see the same OOPS ID.

for file in $(find . -name daisy-error.log); do zgrep -H 5e0e1c5e-d68c $file; done
./e-t-daisy-app-f1d7575e-36ff-4101-8a60-83de85e69f1d/production-logs/daisy-error.log:2017-01-09 16:55:04 [15792] [INFO] (5e0e1c5e-d68c-11e6-abd2-fa163e171d9b) inserted into OOPS CF
./e-t-daisy-app-f1d7575e-36ff-4101-8a60-83de85e69f1d/production-logs/daisy-error.log:2017-01-09 16:55:04 [15792] [INFO] (5e0e1c5e-d68c-11e6-abd2-fa163e171d9b) asked for CORE

And actually that's because it would have a new OOPS ID if it were submitted again, however we won't let the crash be submitted again because we check to see if it is a duplicate and then reject it.

[Brian Murray (brian-murray)]

I'd be better if whoopsie was able to send only the core dump again and not the whole report, but I think this behavior is okay for the time being.

Or possibly the duplicate check, in daisy/submit.py, should keep track of core files we wanted versus core files received.