Please remove the non-free 3rdparty folder

[Thomas Goirand]

Since I've done the work in Debian, python-antlr3 contains the necessary Python bindings for Congress, which doesn't need to contain antlr3 itself. This is also available in Ubuntu Xenial. So there shouldn't be any problem anymore for the Ubuntu gate if that's what you're using.

The 3rdparty folder is considered non-free, as it contains already pre-built files (ie: antlr-3.5-complete.jar). This makes the whole package non-free from the point of view of all downstream distributions (Debian, Ubuntu, RDO, SuSE and Gentoo).

[Tim Hinrichs (thinrichs)]

Let's look into this soon, as it impacts downstream.

[Eric K (ekcs)]

Thank you Thomas! We've been wanting to do this for a while. Having python-antlr3 in xenial is a big help. Thanks for the alert!

The main issues we still have are:
1. Not available in 14.04 (including backports). I need to confirm if it's okay to ditch 14.04 now in a new release.
2. For python3, we rely on upstream antlr3/python3 runtime not available in packages.

Long term we're still looking at replacing antlr3 completely because upstream development has moved on, but that's a complex job and in the mean time we still want to see what we can do.

@Tim, the antlr-3.5-complete.jar file is only used because the Makefile calls on antlr3 to re-generate the parser and lexer from the grammar file on each setup. Do you know why this was needed instead of just using the pre-generated parser and lexer in the repo?
A developer who changes the grammar will still need to use antlr3, but I don't think the end-users need it.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Eric K (<email address hidden>) on branch: master
Review: https://review.openstack.org/420273
Reason: wait for full fix.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/420273
Committed: https://git.openstack.org/cgit/openstack/congress/commit/?id=cdfb121cf4eef96b917b0e3d7e86616b965e3506
Submitter: Jenkins
Branch: master

commit cdfb121cf4eef96b917b0e3d7e86616b965e3506
Author: Eric K <email address hidden>
Date: Fri Jan 13 19:10:28 2017 -0800

    Remove antlr3 files except Python runtime

    The removed files are not needed by users, and some files
    are incompatible with free software packaging (e.g. jar)

    Drawback: if a developer changes congress grammar, s/he
    needs to compile or download the antlr3 file to generate
    the Parser/Lexer.

    Instructions added to download antlr3 and generate files.

    Partial-Bug: 1591763

    Change-Id: I716ffe9f651dc48eb513efbb653aadf8f33f604a

[Thomas Goirand (thomas-goirand)]

Hi,

Thanks Eric for this patch. Now, it'd be nice to also remove the Python runtime files: since they are also available in Debian and Ubuntu, there's no reason to keep them embedded.

Also, CongressLexer.py CongressParser.py should be removed from the Git, as they are generated. In the Debian package, we had to remove them from the original sources.

[Eric K (ekcs)]

Thank you Thomas for the guidance.

Would it help packaging if we made the runtime and generated files unnecessary for python2 usage on newer distros (hence deletable for packaging), but still kept them in the repo for python3 tests (python3 runtime not available in distros) and to support older distros?

I plan to rewrite the parsing bits to remove antlr completely in the Pike cycle to properly support python3.

Also what are some critical packaging deadlines to be aware of?

Thanks again!

[Thomas Goirand (thomas-goirand)]

Eric, could you please reopen this bug? The 3rd party folder is still present in master, there's absolutely no reason to close this bug, especially that antlr3 is in both Debian and Ubuntu. Please do remove the 3rdparty folder.

[Thomas Goirand (thomas-goirand)]

Would it help if python3 runtime for antlr3 was packaged?

[Eric K (ekcs)]

Hi Thomas! Not closing the bug, but slipping to Queens.

If hitting Pike is really important, what I can do is this:
Keep the generated files and the remaining antlr runtime in the repo, but clearly document that they can be removed for python2 packaging. For python3, the antlr runtime is still required in repo because like you said it's not been packaged.

Thanks so much for helping us along!

[Corey Bryant (corey.bryant)]

Will this be fixed in Rocky? It's causing packaging issues in Ubuntu.

[Thomas Goirand (thomas-goirand)]

Same over here. There are packages for both py2 and py3 so there is no reason to keep this folder. Please make this a priority.