Twisted Matrix process pool

ampoule should provide a facility to work with frozen or py2exe'd files

Reported by Glyph Lefkowitz on 2008-08-19
2
Affects Status Importance Assigned to Milestone
Ampoule
High
dialtone

Bug Description

One of the typical problems with process pools (as opposed to thread pools) is that they're sensitive to changes in the environment which make it impossible to start up a process.

One way that this can be mitigated is to provide an explicit integration mechanism for weird deployment environments, where tools like py2exe might be in use.

This may require some explicit cooperation from the deployment mechanism (a command-line flag saying "this should be a worker process", for example) but it should be explicitly accounted for.

dialtone (dialtone) wrote :

So basically what you are proposing is that subprocesses are started with a command line flag and the bootstrap mechanism of the children should pick that up or is it that the API should have a 'weird_deployment' flag that basically does the bootstrapping of children processes in a different way?

Glyph Lefkowitz (glyph) wrote :

Not a "flag", so much as an object that can be user-supplied, responsible for providing some or all of the spawnProcess arguments. Normally this would be sys.executable, os.environ, etc, but it may be different in unusual situations.

Plus, the other side of this equation: an API call you can put in your wacky frozen main-point which will detect whether the supplied arguments mean "run the app as normal" or "run a subprocess". (Maybe that's what you meant by a "flag"?)

Nothing here is difficult, it's just that there needs to be a single, clearly-documented way of handling unusual deployments so that users do not attempt to work around it themselves with half-working solutions that may not correctly deal with every contingency, or may not be compatible with future ampoule versions.

Glyph Lefkowitz (glyph) wrote :

Also, I believe it should be possible to provide *implementations* of this provide-me-some-spawnProcess-arguments interface for specific deployment technologies: one for py2exe, one for freeze, one for pyinstall, one for py2app (or whatever the flavor of the week is for single-file executables).

dialtone (dialtone) wrote :

gotcha, so basically some kind of Configuration object that supplies basically almost every argument that is not directly useful to the process pool but rather to the code that is below it. It's actually useful and I was thinking about doing the same thing too. So the next release will implement this.

dialtone (dialtone) wrote :

fixed in r21

Changed in ampoule:
assignee: nobody → dialtone
importance: Undecided → High
milestone: none → 0.0.5
status: New → Fix Committed
dialtone (dialtone) on 2008-11-14
Changed in ampoule:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers