full text searches include result pages that are broken links

[Abel Deuring]

A full text search may return result pages from deactivated projects; following these links leads to 404 errors and OOPSes.

Example: https://launchpad.net/+search?field.text=easy+peasy returns ca 2800 results; folloowing the link to, for example, https://answers.launchpad.net/ubuntu-eee , leads to a 404 errors, for anonymous users.

See also OOPS-1158G505.

[Curtis Hovey (sinzui)]

As I stated before this bug was reported, the problem is really upstream. Google's catalog can take a few weeks to update. The data will correct itself.

We can consider reparsing Google's data to attempt to locate the pillar or person and purge the items if the entity is deactivated. That can be costly to lookup, and break batching. Unless there is a better solution I'm not inclined to address this.

[Gavin Panella (allenap)]

Perhaps we can use the referrer header in 404 OOPSes to poke the search indexer?