Comment 10 for bug 200572

I see two places we could put the sanity check, either as the first action in Navigation.publishTraverse or in Navigation._publishTraverse() in the setFirstLayer() block. I do not think NotFound error page will work without the layer being set first. The other issue is that the NotFound page errors trying to say that the url is invalid.

webapp.url.urlparse ensures the url is forces the encoding to ascii so calling
    urlparse(unquote('https://launchpad.net/CRACK'))
where CRACK is url encoded or raw non-ascii will raise UnicodeDecodeError. We can catch this error and raise a NotFoundError. The NotFound page will need to avoid working with the URL.

The are a few problems with this suggestion. It will cause 404s when param values contain non-ascii ('https://launchpad.net/+search?text=%E9'). We could parse what is before the query string, then parse the params and values individually as ascii and unicode. This might give us a comprehensive solution to bad encoding in URLs.