make build can't explode if geoip-data is not present

[Michael Hudson-Doyle]

From a mail on the list (> > is me, the rest is Tom):

> > So this landing broke the staging codehosting systems as they don't have
> > the geoip database installed and make build now dies if the geoip
> > database is not installed. I've filed an RT to get geoip-data installed
> > on the codehosting systems, but it does raise a sort of interesting
> > dichotomy:
> >
> > (a) All launchpad servers should have launchpad-dependencies installed,
> > so it's fine for make build to hard-depend on geoip-data
> > (b) The codehosting systems are never going to _use_ the geoip-data, so
> > it's insane for make build to hard-depend on geoip-data.

b) is the right answer.

We don't want to install any package on a production server that isn't
necessary.

Thanks, Tom

> >
> > I don't know what the right answer is.
> >
> > Cheers,
> > mwh
> >

[Michael Hudson-Doyle (mwhudson)]

I don't know who's going to do this (probably not me) but it has to be done before the rollout!

[Steve McInerney (spm)]

FWIW - this just blew up on staging restore: @ chokecherry:

....
Thu Aug 28 02:00:02 BST 2008 About to restore production database dump to launchpad_staging_new
Thu Aug 28 05:20:46 BST 2008 Restored production database dump to launchpad_staging_new
rm -f bzr-version-info.py bzr-version-info.pyc
if which bzr > /dev/null && test -x `which bzr`; \
                then PYTHONPATH= bzr version-info --format=python > bzr-version-info.py 2>/dev/null; \
        fi
********************************************************
No GeoIP db found. Plese install launchpad-dependencies
********************************************************
make: *** [check_geoip_db] Error 1
Thu Aug 28 05:21:03 BST 2008 Applying database updates and permissions from test to new DB
Thu Aug 28 05:21:03 BST 2008 Changed staging config so upgrade scripts connect to the right DB
Traceback (most recent call last):
  File "./upgrade.py", line 20, in ?
    from canonical.launchpad.scripts import db_options, logger_options, logger
  File "/srv/staging.launchpad.net/test/database/schema/../../lib/canonical/launchpad/scripts/__init__.py", line 22, in ?
    import zope.app.appsetup
  File "/srv/staging.launchpad.net/test/database/schema/../../lib/zope/app/appsetup/__init__.py", line 22, in ?
    from zope.app.appsetup.appsetup import config, database
  File "/srv/staging.launchpad.net/test/database/schema/../../lib/zope/app/appsetup/appsetup.py", line 20, in ?
    import ZODB.interfaces
  File "/srv/staging.launchpad.net/test/database/schema/../../lib/ZODB/__init__.py", line 20, in ?
    from persistent import TimeStamp
  File "/srv/staging.launchpad.net/test/database/schema/../../lib/persistent/__init__.py", line 19, in ?
    from cPersistence import Persistent, GHOST, UPTODATE, CHANGED, STICKY
ImportError: No module named cPersistence
Thu Aug 28 05:21:03 BST 2008 ERROR: Failed to run upgrade.py

[Guilherme Salgado (salgado)]

It's easy to change the 'build' target to not check for the GeoIP database as it doesn't really need that. The problem is that we have another check for the GeoIP database which will fail if you execute_zcml_for_scripts() or similar as the check is inside the constructor of a zope3 utility.

Does the codehosting execute_zcml_for_scripts()?

[Michael Hudson-Doyle (mwhudson)]

No, or at least not at present.

Plenty of things do on forster though, which by Tom's criteria shouldn't have geoip-data installed on it either...

[Tom Haddon (mthaddon)]

This has just broken staging importd servers, as they use the "staging" config

I'm not sure whether this means we need to separate out the importd servers to use a separate config. Please advise.

[Tom Haddon (mthaddon)]

I should mention that I've had to disable the staging importd jobs until this is resolved.

[Guilherme Salgado (salgado)]

I don't know why I asked that question, it's obvious that we need to defer loading the GeoIP DB until it's actually needed. We can do that either by loading it in a cachedproperty of the utility (instead of doing so on its constructor) or by not registering our GeoIP class as a zope utility and import it directly.

I think I prefer the first, but I'm not 100% sure a cachedproperty would play well with a zope utility.

[Guilherme Salgado (salgado)]

landed on mainline r6934

[Steve McInerney (spm)]

importd on marambio and russkaya now at 6942. crontab's re-enabled and checked to be running ok.