document best practice with 'manage.py install' and packaging

[Daniel Manrique]

ubuntu@ubuntu-desktop:~/2014.com.tomechangosubanana:tester$ COLUMNS=200 dpkg --list '*plainbox*' "*checkbox*"
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-============================================-===========================-===========================-=============================================================================================
un checkbox <none> <none> (no description available)
ii checkbox-ng 0.4~dev+bzr3069+pkg7~ubuntu all PlainBox based test runner
ii checkbox-ng-service 0.4~dev+bzr3069+pkg7~ubuntu all CheckBox D-Bus service
ii plainbox 0.6~dev+bzr3069+pkg4~ubuntu all toolkit for software and hardware integration testing
un plainbox-insecure-policy <none> <none> (no description available)
ii plainbox-provider-checkbox 0.5~dev+bzr3068+pkg5~ubuntu amd64 CheckBox provider for PlainBox
ii plainbox-provider-resource-generic 0.4~dev+bzr3069+pkg5~ubuntu amd64 CheckBox generic resource jobs provider
ii plainbox-secure-policy 0.6~dev+bzr3069+pkg4~ubuntu all policykit policy required to use plainbox (secure version)
ii python3-checkbox-ng 0.4~dev+bzr3069+pkg7~ubuntu all PlainBox based test runner (Python 3 library)
ii python3-checkbox-support 0.3~dev+bzr3069+pkg3~ubuntu all collection of Python modules used by PlainBox providers
ii python3-plainbox 0.6~dev+bzr3069+pkg4~ubuntu all toolkit for software and hardware testing (python3 module)

What I did:

ubuntu@ubuntu-desktop:~$ plainbox startprovider 2014.com.roadmr:goop
ubuntu@ubuntu-desktop:~$ cd 2014.com.roadmr\:goop/
ubuntu@ubuntu-desktop:~/2014.com.roadmr:goop$ ./manage.py install
CRITICAL plainbox.crashes: Executable 'manage.py' invoked with Namespace(command=<plainbox.provider_manager.InstallCommand object at 0x7f61f22f2668>, debug_console=False, debug_interrupt=False, layout='flat', log_level=None, pdb=False, prefix='/usr/local', root='', trace=[]) has crashed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 513, in dispatch_and_catch_exceptions
    return self.dispatch_command(ns)
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 509, in dispatch_command
    return ns.command.invoked(ns)
  File "/usr/lib/python3/dist-packages/plainbox/provider_manager.py", line 264, in invoked
    self._write_provider_file(ns.root, ns.prefix, ns.layout, provider)
  File "/usr/lib/python3/dist-packages/plainbox/provider_manager.py", line 272, in _write_provider_file
    lambda stream: self._get_provider_config_obj(
  File "/usr/lib/python3/dist-packages/plainbox/provider_manager.py", line 353, in _write_to_file
    with open(root + pathname, 'wt', encoding='UTF-8') as stream:
PermissionError: [Errno 13] Permission denied: '/usr/local/share/plainbox-providers-1/2014.com.roadmr.goop.provider'
Traceback (most recent call last):
  File "./manage.py", line 20, in <module>
    gettext_domain="2014_com_roadmr_goop",
  File "/usr/lib/python3/dist-packages/plainbox/provider_manager.py", line 1267, in setup
    raise SystemExit(ProviderManagerTool(definition, kwargs).main())
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 291, in main
    return self.dispatch_and_catch_exceptions(ns)
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 513, in dispatch_and_catch_exceptions
    return self.dispatch_command(ns)
  File "/usr/lib/python3/dist-packages/plainbox/impl/clitools.py", line 509, in dispatch_command
    return ns.command.invoked(ns)
  File "/usr/lib/python3/dist-packages/plainbox/provider_manager.py", line 264, in invoked
    self._write_provider_file(ns.root, ns.prefix, ns.layout, provider)
  File "/usr/lib/python3/dist-packages/plainbox/provider_manager.py", line 272, in _write_provider_file
    lambda stream: self._get_provider_config_obj(
  File "/usr/lib/python3/dist-packages/plainbox/provider_manager.py", line 353, in _write_to_file
    with open(root + pathname, 'wt', encoding='UTF-8') as stream:
PermissionError: [Errno 13] Permission denied: '/usr/local/share/plainbox-providers-1/2014.com.roadmr.goop.provider'

It clearly will be unable to write /usr/local/share/whatever if I'm not root. This means that if we do dpkg-buildpackage to try to build the binary directly (i.e. no chroot or sbuild), it will also fail as it won't be able to run as root in the local system.

Strangely, we get the same error if first we try to build a source package (which works) and then we try to sbuild that. Is it because the directory doesn't exist? (/usr/local/share/plainbox-providers-1/).

[Daniel Manrique (roadmr)]

OK, this is by design, what's hitting us is using the recommended debian/rules from the documentation:

#!/usr/bin/make -f

%:
 dh "$@"

override_dh_auto_build:
 $(CURDIR)/manage.py build
 $(CURDIR)/manage.py install

manage.py install tries to install to /usr/share, causing this problem. The plainbox providers have this instead :

override_dh_auto_build:
 cd $(PPA_DISPLACEMENT) && python3 manage.py build
 cd $(PPA_DISPLACEMENT) && python3 manage.py i18n

override_dh_auto_install:
 cd $(PPA_DISPLACEMENT) && python3 manage.py install \
  --prefix=/usr --layout=unix --root=$(CURDIR)/debian/plainbox-provider-resource-generic/

Adjusting the root appropriately allows the provider to build, but it does need this updated manage.py.

[Jeff Lane  (bladernr)]

Setting this to high mainly because I need to get the packaging worked out for the OCP Plainbox provider and this is becoming a huge hurdle to test that my packaging works. (it doesn't, but this is blocking me from easily and reliably testing it)

Feel free to bump it back if you think that's too high.

[Daniel Manrique (roadmr)]

Also, the --root=$(CURDIR)/debian/$BLAH, $BLAH needs to match the name of the binary package we're trying to build, for this to pick up all the files. Notice there's no .install file in this provider.

I'd like to make this bug about updating the documentation to reflect these new needs. It took Jeff and I a few hours to figure this out :)