/api should be added to robots.txt

Bug #582977 reported by Francis J. Lacoste on 2010-05-19
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Karl Fogel

Bug Description

As seen in OOPS-1597E304 and OOPS-1597C1300 we have googlebot making random requests to the API through the URL space used for AJAX requests.

We deny all requests to /api through the robots.txt file so that these spurious requests (making failures in many case) are solved.

Related branches

tags: added: api
Gary Poster (gary) on 2010-05-21
Changed in launchpad-foundations:
status: New → In Progress
importance: Undecided → High
assignee: nobody → Karl Fogel (kfogel)
Karl Fogel (kfogel) wrote :

Done as per RT 39388 (visited https://launchpad.net/robots.txt to confirm).

Removing the long-unused template file from the tree is a separate issue; I'm reusing the branch made for this bug to do that. See (internal link, sorry) the thread starting here for details: https://lists.ubuntu.com/mailman/private/canonical-launchpad/2010-May/056635.html .

Changed in launchpad-foundations:
status: In Progress → Fix Released
Changed in launchpad-foundations:
milestone: none → 10.05
tags: added: qa-needstesting
Gary Poster (gary) on 2010-05-28
tags: added: qa-ok
removed: qa-needstesting
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers