Distribution tarball has licensing problems that prevent redistribution

[Robie Basak]

I have failed to update the debian/copyright for an upload of 1.20 to
Utopic for the following reasons:

src/code.google.com/p/go.net/html/charset/testdata/ has no included
license permitting redistribution. It appears to have come from the
referenced W3C URL, which appears licensed under the W3C documentation
license that prohibits modification, contrary to DFSG.

src/github.com/binary132/gojsonschema/json_schema_test_suite has unknown
copyright holder and no license for redistribution.

src/launchpad.net/gnuflag: no LICENSE file as referenced from source
code.

src/launchpad.net/godeps: no license for redistribution (and no
copyright statement).

src/launchpad.net/gomaasapi: not all files have a license statement that
apply to them (and no general licensing statement that clearly applies
to all files).

src/launchpad.net/goyaml: not clear on this. LICENSE.libyaml appears to
have no corresponding source files. Is this redundant now? What
copyright statement, licensing and mandatory statements apply to each
file?

src/github.com/juju/errgo: no license for redistribution.

src/github.com/juju/errors: LICENSE says LGPL-3; some files have a
copyright statement and state GPL-3, others have no statement at all.

src/github.com/juju/names: LICENSE says LGPL-3 with exception, files say
AGPL-3

src/github.com/juju/schema: LICENSE says LGPL-3 with exception, files
say AGPL-3

src/github.com/juju/testing: LICENSE says LGPL-3 with exception, some
files say AGPL-3 but refer to LICENSE (others say LGPL-3)

src/github.com/juju/utils LICENSE says LGPL-3 with exception, some files
say AGPL-3 but refer to LICENSE (others say LGPL-3)

src/github.com/juju/juju: very big; not yet checked.

To be clear: every file must have clear copyright ownership and a
license that permits redistribution under the terms of the DFSG, whether
express, by reference to some other file, implied, etc.

It would be easiest if, for each package: 0) a single license is used,
or failing this, there is an separate explanation of what license
applies to what files, what dual-licensing is in place, etc; 1) there
is a general copyright and licensing statement that clearly applies to
the whole package, to disambiguate files which end up with no individual
statement; 2) files that can contain comments (and where practical)
contain a copyright notice and licensing statement for the elimination
of any doubt; 3) copyright notices and licensing statements are in a
form detectable by the "licensecheck" tool.

Note that changing licensing requires agreement from all existing
copyright holders.

[Curtis Hovey (sinzui)]

The core devs can fix the licenses for all the github.com/juju/* There might be some adventures updating 1.20 to use them as 1.20 tends to freeze its versions of those libraries.

Maybe we can delete code.google.com/p/go.net/html/charset/testdata/ from the tarball, or possibly more. Does the test suite look at that data.

[Ian Booth (wallyworld)]

I have addressed almost all of the issues as highlighted in this bug. There's one more repo update needed but I don't have permission (I'll organise to get it done).

Changes:
- the correct LICENSE file is in place for each juju sub repository mentioned in the bug
- all the source file headers have been updated (or added) to refer to the LGPLv3 licence for each repo mentioned

Since trunk has diverged from 1.20 for several of the sub repos, I branched and updated both the 1.20 and trunk branches of each.

Todo (by me) - update the juju-core dependencies.tsv file to ensure the latest source code revision containing the updated licence information is reflected in the dependencies list. This is trivial and can be done at the end once everything else is fixed.

>
> src/code.google.com/p/go.net/html/charset/testdata/
> has no included license permitting redistribution. It appears to have
> come from the
> referenced W3C URL, which appears licensed under the W3C documentation
> license that prohibits modification, contrary to DFSG.
>
> src/github.com/binary132/gojsonschema/json_schema_test_suite
> has unknown copyright holder and no license for redistribution.
>

I have not done these; excluding these files will not affect the passing of the juju-core test suite.

> src/launchpad.net/gnuflag:
> no LICENSE file as referenced from source code.
>

I don't have commit access to update this branch. I'll poke Roger to do it.

> src/launchpad.net/godeps:
> no license for redistribution (and no copyright statement).
>

We don't distribute godeps so nothing to do AFAIK.

> src/launchpad.net/gomaasapi:
> not all files have a license statement that
> apply to them (and no general licensing statement that clearly applies
> to all files).
>

Done. Although the only file I saw without a copyright header was the Makefile. Do we need one for that?

> src/launchpad.net/goyaml:
> not clear on this. LICENSE.libyaml appears to
> have no corresponding source files. Is this redundant now? What
> copyright statement, licensing and mandatory statements apply to each
> file?
>

No changes made by me. I think Gustavo has clarified the issue?

> src/github.com/juju/errgo:
> no license for redistribution.
>

Done.

> src/github.com/juju/errors:
> LICENSE says LGPL-3; some files have a copyright statement and state
> GPL-3, others have no statement at all.
>

Done.

> src/github.com/juju/names:
> LICENSE says LGPL-3 with exception, files say AGPL-3
>

Done.

> src/github.com/juju/schema:
> LICENSE says LGPL-3 with exception, files say AGPL-3

Done.

>
> src/github.com/juju/testing:
> LICENSE says LGPL-3 with exception, some files say AGPL-3 but refer to
> LICENSE (others say LGPL-3)
>

Done.

> src/github.com/juju/utils:
> ICENSE says LGPL-3 with exception, some files say AGPL-3 but refer to
> LICENSE (others say LGPL-3)
>

Done.

> src/github.com/juju/juju: very big; not yet checked.
>
>

From what I can see, only 2 source files have an incorrect header. I'll fix at the same time when updating the dependencies file.

[Robie Basak (racb)]

Ian,

Thank you for looking at this on such short notice.

On Wed, Jul 16, 2014 at 05:50:08AM -0000, Ian Booth wrote:
> > src/code.google.com/p/go.net/html/charset/testdata/
> > has no included license permitting redistribution. It appears to have
> > come from the
> > referenced W3C URL, which appears licensed under the W3C documentation
> > license that prohibits modification, contrary to DFSG.
> >
> > src/github.com/binary132/gojsonschema/json_schema_test_suite
> > has unknown copyright holder and no license for redistribution.

> I have not done these; excluding these files will not affect the
> passing of the juju-core test suite.

Do we need a juju-release-tools task on this bug then, so that these are
stripped from the juju-core tarball releases?

> > src/launchpad.net/godeps:
> > no license for redistribution (and no copyright statement).
>
> We don't distribute godeps so nothing to do AFAIK.

These files are present in the 1.20.0 release tarball. I was working
against this since before the 1.20.1 release. If it's gone now, then
that's fine.

> > src/launchpad.net/gomaasapi:
> > not all files have a license statement that
> > apply to them (and no general licensing statement that clearly applies
> > to all files).
> >
>
> Done. Although the only file I saw without a copyright header was the
> Makefile. Do we need one for that?

Also README, which is copyrightable.

There must be a license which applies to everything that is
copyrightable, since copyright applies by default. Without a license,
copyrightable files without a license are not redistributable.

A LICENSE file in the directory would be sufficient. For minor
copyrightable files that don't have an explicit statement, it could
reasonably be implied that a general license specified in the top-level
directory applies to them. I don't think COPYING and COPYING.LESSER are
sufficient here, as both are present, it isn't clear which applies, and
there isn't a statement that actually applies the license (as opposed to
the license text itself).

> > src/launchpad.net/goyaml:
> > not clear on this. LICENSE.libyaml appears to
> > have no corresponding source files. Is this redundant now? What
> > copyright statement, licensing and mandatory statements apply to each
> > file?
>
> No changes made by me. I think Gustavo has clarified the issue?

I've not seen this. Gustavo, please could you clarify here?

I don't have easy visibility of the entire licensing delta against the
tree I checked, so I guess I'll wait before I verify everything.

Thanks,

Robie

[Ian Booth (wallyworld)]

> > src/code.google.com/p/go.net/html/charset/testdata/
> > has no included license permitting redistribution. It appears to have
> > come from the
> > referenced W3C URL, which appears licensed under the W3C documentation
> > license that prohibits modification, contrary to DFSG.
> >
> > src/github.com/binary132/gojsonschema/json_schema_test_suite
> > has unknown copyright holder and no license for redistribution.

> I have not done these; excluding these files will not affect the
> passing of the juju-core test suite.

Q. Do we need a juju-release-tools task on this bug then, so that these are
stripped from the juju-core tarball releases?

A. I believe Curtis will exclude the files from the release tarball.
Also, we are working to get the gojsonschema files properly licensed as it turns out the code was devloped on Canonical time so should be under the juju repo.

> > src/launchpad.net/godeps:
> > no license for redistribution (and no copyright statement).
>
> We don't distribute godeps so nothing to do AFAIK.

Q. These files are present in the 1.20.0 release tarball. I was working
against this since before the 1.20.1 release. If it's gone now, then
that's fine.

A. Yes, Curtis confirmed some files we mistakenly included. Will be fixed for the next release.

> > src/launchpad.net/gomaasapi:
> > not all files have a license statement that
> > apply to them (and no general licensing statement that clearly applies
> > to all files).
> >
>
> Done. Although the only file I saw without a copyright header was the
> Makefile. Do we need one for that?

[robie] Also README, which is copyrightable.

[ian] Many if not most of our sub projects on launchpad - goose, gomaasapi, gwacl etc - either:
1. have a README without a copyright statement
2. have no README at all
Do we need to add a copyright header to the existing README files and add README files where they don't exist?

[robie] A LICENSE file in the directory would be sufficient. For minor
copyrightable files that don't have an explicit statement, it could
reasonably be implied that a general license specified in the top-level
directory applies to them. I don't think COPYING and COPYING.LESSER are
sufficient here, as both are present, it isn't clear which applies, and
there isn't a statement that actually applies the license (as opposed to
the license text itself).

[ian] for those projects with a COPYING files, there is also a LICENSE file with content that refers back to the COPYING files
eg
<snip>
See both COPYING and COPYING.LESSER for the full terms of the GNU Lesser
General Public License.
<snip>

[Robie Basak (racb)]

On Thu, Jul 17, 2014 at 12:53:02AM -0000, Ian Booth wrote:
> > > src/launchpad.net/gomaasapi:
> > > not all files have a license statement that
> > > apply to them (and no general licensing statement that clearly applies
> > > to all files).
> > >
> >
> > Done. Although the only file I saw without a copyright header was the
> > Makefile. Do we need one for that?
>
> [robie] Also README, which is copyrightable.
>
> [ian] Many if not most of our sub projects on launchpad - goose,
> gomaasapi, gwacl etc - either:
> 1. have a README without a copyright statement
> 2. have no README at all
> Do we need to add a copyright header to the existing README files and
> add README files where they don't exist?

No - a LICENSE file (as the others have) is sufficient.

> [robie] A LICENSE file in the directory would be sufficient. For minor
> copyrightable files that don't have an explicit statement, it could
> reasonably be implied that a general license specified in the top-level
> directory applies to them. I don't think COPYING and COPYING.LESSER are
> sufficient here, as both are present, it isn't clear which applies, and
> there isn't a statement that actually applies the license (as opposed to
> the license text itself).
>
> [ian] for those projects with a COPYING files, there is also a LICENSE
> file with content that refers back to the COPYING files
> eg
> <snip>
> See both COPYING and COPYING.LESSER for the full terms of the GNU
> Lesser General Public License.
> <snip>

Right, but src/launchpad.net/gomaasapi has no LICENSE file. Not in
1.20.0, anyway. That was my point - you need one since not all other
files have a clear license statement within them. Just stick that in the
same as the others, and you're fine. Or put a statement in every README,
Makefile, etc, but that's just tedious and LICENSE is easier. I was just
trying to make the actual requirement clear and not dictate anything not
strictly required - sorry for the confusion.

I would like to get this cleared up all in one go, but don't have easy
visibility of your changes to make sure. I'm also worried I missed
something in my initial review and would like to double check. Is there
an easy way I can see all packages in a single tree, like I'm looking at
the juju-core 1.20.0 release tarball, but with all your updates?

Robie

[Ian Booth (wallyworld)]

Outstanding questions/clarifications:

- gomaasapi has a LICENSE file - the next build of juju will have it.

RE: goyaml, here's Gustavo's answer from the mailing list

--
> src/launchpad.net/goyaml:
> not clear on this. LICENSE.libyaml appears to
> have no corresponding source files. Is this redundant now? What
> copyright statement, licensing and mandatory statements apply to each
> file?

Both license files still apply. The code that was ported from libyaml is
covered by the respective license.
--

Current status is that Juju 1.20 and master branches now both have had their dependency files updated to pull in source code from repositories where all the issues highlighted in this bug have been resolved (apart from the go.net and gojsonschema issues). The offending files from go.net will be excluded from the tarball. Work is under way to fix gojsonschema. The next Juju builds from the CI server will produce tarballs with all the fixes (apart from the above) included.

[Ian Booth (wallyworld)]

gojsonschema repo (and also gojsonreference and gojsonpointer) have been moved under the juju repo and dependency file updated.
AFAIK, the licensing information should all now be correct, so long as the go.net test files are excluded.

[Robie Basak (racb)]

I just uploaded 1.20.7 to Utopic, which has an updated debian/copyright file and everything is now policy compliant and redistributable (as far as I can determine, anyway). Thanks, all, for working on this and getting it all fixed up.