make plugin: can't copy directories or nested artifacts

[James Lucas]

I'm trying to create a snap of camlistore -- the easiest way I've found is using the 'make' plugin. However, given the way that camlistore builds its artifacts -- all artifacts go into a /bin directory -- I haven't been able to get the artifacts in the snap without some modifications to snapcraft.

I've tried two different scenarios, both of which fail. The first was just to set 'bin' as the artifact and have it copy the whole directory:

```
parts:
  camlistore:
    plugin: make
    artifacts:
    - bin
```

This fails because make uses file_utils.link_or_copy, not file_utils.link_or_copy_tree.

The other thing I tried was passing each artifact individually -- like this:

```
parts:
  camlistore:
    plugin: make
    artifacts:
    - bin/cammount
    - bin/camget
    - bin/camput
<snip>
```

This doesn't work either, because of file_utils.link_or_copy doesn't support creating nested directories prior to copying the files.

I've thrown together a PR to adddress both scenarios, which I'll submit once I file this. I'm not sure if I took the best approach, but the changes are this:

. For the first scenario, have the make plugin check if the artifacts specified represent a directory, and, if so, use file_utils.link_or_copy_tree instead
. For the second issue, modify file_utils.link_or_copy to create subdirectories if the directory of the destination object doesn't exist.

[Mark Shuttleworth (sabdfl)]

What I've found works best is to write a small snapcraft plugin which
drives the specific build I am making.

I think we should document that approach, and add a 'snapcraft
init-plugin <foo>' which gives you a skeletal plugin with pointers to
the documentation.

Essentially, I subclass BasePlugin and call its methods before doing
whatevers needed. It sets up environment variables and common functions,
I just drive the build process.

Here's an example:

class BuildopensslPlugin(snapcraft.BasePlugin):
    """Build Nginx

This plugin is designed to wrap the OpenSSL build process only. It takes no
parameters.

"""

    def _build_environment(self):
        env = os.environ.copy()

        include_paths = []
        for root in [self.installdir, self.project.stage_dir]:
            include_paths.extend(
                snapcraft.common.get_library_paths(root,
self.project.arch_triplet))

        return env

    def build(self):
        super().build()
        print('Configuring OpenSSL...')
        self.run(['./config',
          '--prefix='+self.installdir,
          '--openssldir='+self.installdir,
          'zlib-dynamic', '--strict-warnings', 'no-ssl',
          'no-deprecated', 'enable-heartbeats' ],
          cwd=self.builddir, env=self._build_environment())
        print('Building OpenSSL...')
        self.run(['make', ],
          cwd=self.builddir, env=self._build_environment())
        self.run(['make', 'test'],
          cwd=self.builddir, env=self._build_environment())
        print('Installing OpenSSL...')
        self.run(['make', 'install'],
          cwd=self.builddir, env=self._build_environment())
        #self.run(['strip', os.path.join(self.installdir, 'bin/openssl')],
        # cwd=self.builddir), env=self._build_environment())

Mark

[James Lucas (jaymell)]

Awesome. Thanks for the example, Mark, this definitely gives me some context for a better approach to my issue.

Is there any utility in something like a 'shell' or 'cmd' plugin, that alternatively allows you to specify an array of commands that could be run within the plugin's build method? I guess I'm not sure whether that stretches the original intent of plugins enough to be worthwhile.

Thanks again for the feedback -- if it seems worthwhile, I'd be glad to take a stab at the 'init-plugin' option and/or a separate plugin for passing arbitrary commands.

[Leo Arias (elopio)]

Hey James. Thanks for your contributions.
About the shell plugin, we are discussing it in https://github.com/snapcore/snapcraft/pull/727. Feel free to leave your comments and suggestions there.

[James Lucas (jaymell)]

Thank you for the link!

[Sergio Schvezov (sergiusens)]

btw, thanks for getting one of the first "sort of" production snaps back into action (http://blog.sergiusens.org/posts/Ubuntu%20Core/) :-)

[James Lucas (jaymell)]

Hah. Awesome, glad the fix is wokring, and my apologies for obviously not having previously undertaken the arduous task of googling 'camlistore snap', else I would have seen this. Cheers!