database creation issue when running with gunicorn

[Cuong]

Hi,

I face an database creation issue while launched with gunicorn. Please look at my attached image.

Server Info:
openerp-server: 6.1 rev 4086
openerp-web: 6.1 rev 2244, operate in embedded mode
gunicorn: 0.13.4

Thanks,

[Vo Minh Thu (thu)]

Gunicorn support (with more than 1 worker process) is unfortunately limited to completely 'installed' databases. It is not possible to use it to create a new database, or to update any data for which the server has a local cache (e.g. record rules or translations).

We have a patch already available to coordinate the multiple processes so that their state (model pool and caches) is kept in sync at https://code.launchpad.net/~openerp-dev/openobject-server/trunk-gunicorn-signaling-vmt

This also requires a little change change client-side to use a blocking operation when creating a new database instead of a polling one (which would hit different (not in sync) workers.

[Cuong (bhcuong2008)]

Hi Thu,

When running just 1 worker, it can create new database. However, I face another issue with uwsgi (1 worker), it run forever. uwsgi log displays as below, (these messages repeat forever). Note that these messages are different from with normal openerp-server log (db creation) and gunicorn log. I also attach screen of web browser. Repeating time between messages is equal to 'loading' message on the web screen.

uwsgi log:
===
[pid: 2726|app: 0|req: 309/309] 192.168.1.131 () {46 vars in 932 bytes} [Thu Mar 1 17:25:44 2012] POST /web/database/progress => generated 51 bytes in 99 msecs (HTTP/1.1 200) 3 headers in 143 bytes (1 switches on core 0)
[pid: 2726|app: 0|req: 310/310] 192.168.1.131 () {46 vars in 932 bytes} [Thu Mar 1 17:25:45 2012] POST /web/database/progress => generated 51 bytes in 299 msecs (HTTP/1.1 200) 3 headers in 143 bytes (1 switches on core 0)
[pid: 2726|app: 0|req: 311/311] 192.168.1.131 () {46 vars in 932 bytes} [Thu Mar 1 17:25:46 2012] POST /web/database/progress => generated 51 bytes in 108 msecs (HTTP/1.1 200) 3 headers in 143 bytes (1 switches on core 0)
[pid: 2726|app: 0|req: 312/312] 192.168.1.131 () {46 vars in 932 bytes} [Thu Mar 1 17:25:46 2012] POST /web/database/progress => generated 51 bytes in 115 msecs (HTTP/1.1 200) 3 headers in 143 bytes (1 switches on core 0)
....
....
===

gunicorn log, the same as openerp-server log.
===
2012-03-01 17:12:51,156 INFO Starting gunicorn 0.13.4
2012-03-01 17:12:51,340 2515 INFO ? openerp.addons.web: embedded mode
2012-03-01 17:12:51 [2515] [INFO] Listening at: http://0.0.0.0:10001 (2515)
2012-03-01 17:12:51,966 2515 INFO ? gunicorn.error: Listening at: http://0.0.0.0:10001 (2515)
2012-03-01 17:12:51 [2515] [INFO] Using worker: sync
2012-03-01 17:12:51,967 2515 INFO ? gunicorn.error: Using worker: sync
2012-03-01 17:12:51 [2522] [INFO] Booting worker with pid: 2522
2012-03-01 17:12:51,972 2522 INFO ? gunicorn.error: Booting worker with pid: 2522

2012-03-01 17:13:20,083 2522 INFO template1 openerp.service.web_services: CREATE DATABASE test_05
2012-03-01 17:13:22,780 2522 INFO test_05 openerp.modules.loading: module base: loading objects
2012-03-01 17:13:22,878 2522 INFO test_05 openerp.modules.module: module base: creating or updating database tables
2012-03-01 17:13:31,773 2522 INFO test_05 openerp.osv.orm: Computing parent left and right for table res_partner_category...
...
...
===

Thanks,

[Cuong (bhcuong2008)]

Hi,

I found an interesting thing. When I enable multi-threads in uwsgi, it can operate normal, meaning create database successfully.

I dont understand the scene behind.

Thanks,

[Vo Minh Thu (thu)]

Cuong,

I have never tried with uwsgi.

But what you show looks like access logs shown by uswgi, while the client is polling to see if the database is eventually created. Are sure it 'loops forever', and not just for a while, eventually stopping when the database is ready?

[Cuong (bhcuong2008)]

Hi Thu,

Please look at my attached video. This lasts about 1.5 minutes. It runs forever, not stop. DB is created but not enough tables. I just capture about 1.5 minutes due to too big file size I can't capture more. In reality, I wait for 5 minutes but it does not stop.

I think there is some thing to the web client polling that blocks db creation process.

Thanks,

[Cuong (bhcuong2008)]

Hi Thu,

During my test with multi-processes (workers), I face another issue with installation module. In the attached picture, I'm installing POS module, I configure uwsgi running 4 workers. It's at the end of installation process, then error occurs. Video size is too big so I can't attach the video of installation process here.

With only 1 worker, It's fine.

After many tests, I think there's some issue with framework when working with multi concurrent processes.

Thanks,

[Dimitri John Ledkov (ex-credativ) (dle-credativ)]

Confirming the original bug (can't complete database install) with uwsgi multiple workers.

[Dimitri John Ledkov (ex-credativ) (dle-credativ)]

Was this fixed in: http://bazaar.launchpad.net/~openerp/openerp-web/6.1/revision/2267

BTW it doesn't seem to be merged in trunk.