NoFile errors reported by staging for the same branch

[Curtis Hovey]

For 3 days, staging has been sending me hate mail about NoFile found for mark-mims/oneiric/node-app/trunk/hooks/* The specific file varies.

2013-07-17 22:21:22 [1415] [ERROR] http://staging.jujucharms.com/%7Emark-mims/oneiric/node-app/hooks/mongodb-relation-changed
Traceback (most recent call last):
  ...
    response = view(request)
  File "/home/webops_deploy/charmworld/charmworld/views/charms.py", line 317, in personal_hook
    return _hook_view(request, charm, request.matchdict["hook"])
  File "/home/webops_deploy/charmworld/charmworld/views/charms.py", line 162, in _hook_view
    cfile = CharmFileSet.get_by_id(fs, fileid)
  File "/home/webops_deploy/charmworld/charmworld/models.py", line 808, in get_by_id
    found = fs.get(fileid)
  File "/home/webops_deploy/charmworld/local/lib/python2.7/site-packages/gridfs/__init__.py", line 140, in get
    return GridOut(self.__collection, file_id)
  File "/home/webops_deploy/charmworld/local/lib/python2.7/site-packages/gridfs/grid_file.py", line 392, in __init__
    (files, file_id))
NoFile: no file in gridfs collection Collection(Database(Connection('10.55.63.146', 27017), u'juju'), u'fs.files') with _id u'mark-mims/oneiric/node-app/trunk/hooks/mongodb-relation-changed'

[Curtis Hovey (sinzui)]

attached is a summary of the charms that report NoFile errors. Many, maybe all are broken in manage.jujucharms.com.

The charms may have always had missing files, but recently this became a problem because browsers (and bots) are seeing the charms and the links to the hook files. If the charms are bad, we might want to stop making links to the hook files.

The processing order in ingest was changed a few weeks ago and we believe ingest is exiting during the bzr phase *before* recording that the charm has errors.

[Abel Deuring (adeuring)]

I looked a bit closer at the cs:~charmers/oneiric/rabbitmq

The related branch is lp:~charmers/charms/oneiric/rabbitmq-server/trunk

So, we start with the charm name "rabbitmq-server"

This name is used to build the IDs for the files.

But: The ingest function scan_charm() parses the file metadata.yaml -- and the file has this content:

ensemble: formula
name: rabbitmq
summary: "An AMQP server written in Erlang"
description: [not very interesting -- not copied]

So, _here_ the name is "rabbitmq".

And the last lines of scan_charm():

    charm_data.update(metadata)
    complete_charm_data = process_charm(charm_data)
    # Modify charm_data itself.
    charm_data.clear()
    charm_data.update(complete_charm_data)

The first line above changes the charm's name to "rabbitmq", so we can access the charm as http://localhost:2464/charms/oneiric/rabbitmq -- but the file IDs still contain the branch name "rabbitmq-server"...

[Abel Deuring (adeuring)]

[sorry, forgot to describe the other half of the problem:]

...and views.charms._hook:_view(request, charm, hook) calls

     fileid = CharmFileSet.gen_fileid(charm._representation, 'hooks/' + hook)

and gen_fileid() uses the charm's name as specified in metadata.yaml, while the the stored file still contains the LP package name.