[MIR] zookeeper, netty, log4cxx

[Clint Byrum]

== Availability ==

Zookeeper is written mostly in java, with a C client library as well, and has been available in Ubuntu since at least 10.04.

== Rationale ==

Zookeeper is used for large distributed systems and provides several primitives necessary to coordinate network activities. Zookeeper is a key component of juju, which we have identified as a key tool needed to help users transition into the cloud and deploy large scale services as well.

== Quality assurance ==

Zookeeper is fairly complicated and a low level service, so documentation is a must, though it is fairly simple to 'apt-get install zookeeperd' and have a working, powerful service immediately available.

In Ubuntu, ZK has no bugs above Medium priority (and that one is not severe at all).

The package is maintained in Debian currently by our own core developer, James Page.

Work is ongoing to enable the various test suites shipped in the package. Unfortunately there are a few non-deterministic tests which need to be patched out and fixed upstream (they pass sometimes). Thus, the package doesn't run the unit tests now in build, but that should be a simple patch once the bad tests are identified, which james-page is doing now. That process will be tracked in this bug report:

https://bugs.launchpad.net/ubuntu/+source/zookeeper/+bug/913881 (Closed)

Package has an up to date watch file.

== UI standards ==

N/A

== Dependencies ==

All of zookeeper's build-deps and dependencies/recommends are already in main, except these three:

daemon - Change is already staged to drop this dependency in the next upload
log4cxx - Very simple stable C++ library, used in zktreeutil, a useful cli tool in zookeeper. It has a test suite, but it is somewhat unmaintained and so is not suitable to be run on upload.
netty - Simple java library which is useful in many contexts. Has a test suite but it is not run currently. This bug was opened to enable it:

https://bugs.launchpad.net/ubuntu/+source/netty/+bug/913878

None of these have any important bugs open in Debian or Ubuntu at this time.

== Maintenance ==

The package is reasonably straight forward, and ZK itself is a stable upstream project which should not require much maintenance.

== Background information ==

This MIR is part of the blueprint servercloud-p-juju-mir. Zookeeper is a key component of juju.

[James Page (james-page)]

I've enabled the C and Java test suites for zookeeper; these now build/test reliably on x86 architectures.

However I get a build failure for the test suite on armel/armhf - see bug 920871.

I've also enable what I can of the unit testing suite for netty; the package only builds a partial feature set to reduce main dependencies (think it was a dep of Eucalyptus).

[James Page (james-page)]

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=650882 was closed in the last Debian release of the packaging.

[James Page (james-page)]

daemon dependency of zookeeper has also now been dropped in preference for start-stop-daemon

[James Page (james-page)]

All non-x86 architectures now build OK; good for final MIR review IMHO.

[Michael Terry (mterry)]

log4cxx needs either a symbols file or a "-V" argument with no parameter to dh_makeshlibs to tighten reverse dependencies. Since this is a C++ library, I'd recommend the "-V" approach.

It also has a test suite that isn't enabled. Adding "dh_auto_test" to it's build target runs it, but it has 3 test failures that should be examined.

I'd also like to see a bug subscriber for it, but that's not a blocker. Debian maintenance is very light, but the bug list isn't bad.

[Michael Terry (mterry)]

Netty seems fine. Simple packaging, no bugs, almost in sync with Debian. Even has a bug subscriber. :)

When I built it in a pbuilder though, it pulled in libslf4j-java from universe. Seems like that needs to be added to this MIR?

[Michael Terry (mterry)]

Just noticed your previous comments about log4cxx's test suite being unmaintained and not suitable for automated running. A bummer. Is it difficult to fix? If those 3 failures can be easily patched to success, that would be a big win. Else, don't worry about it.

And of course the -V issue remains.

[Michael Terry (mterry)]

Zookeeper I'm passing to Jamie, since it seems likely to want a security review.

[Clint Byrum (clint-fewbar)]

mterry, thanks for the review. I fixed the test suite, it now runs on build, and added a -V to dh_makeshlibs.

[Michael Terry (mterry)]

Thanks, Clint! log4cxx approved then.

[Jamie Strandboge (jdstrand)]

Report for shallow review of zookeeper:
 * /usr/bin/zooinspector does not have a man page
 * upstart job looks fine, and zookeeperd runs as non-root
 * There are no dbus services, sudoers fragments or setuid binaries
 * test suite is enabled in the build
 * build logs do have some warnings. See with "egrep -i '(error|warning):' <buildlog>". Notably:
  * [javac] PerChannelBookieClient.java:44: warning: [deprecation] org.jboss.netty.channel.ChannelPipelineCoverage in org.jboss.netty.channel has been deprecated
  * [javac] PerChannelBookieClient.java:62: warning: [deprecation] org.jboss.netty.channel.ChannelPipelineCoverage in org.jboss.netty.channel has been deprecated
 * looking at the code and spot checking here and there, memory, string operations, et al seem fine
 * zkfuse.cc had some interesting stuff and could be problematic, but we don't ship it
 * ./src/contrib/bookkeeper/src/java/org/apache/bookkeeper/bookie/Bookie.java: predictable file name in a world writable directory. This should be checked out. Shouldn't be an issue with yama

While ACLs are supported and various forms of authentication come built in (eg ip and digest authentication, with kerberos authentication is in 3.4 (not in precise yet)), one thing this is a bit disappointing (which is all too common) is the lack of native SSL support. Upstream says network security is the answer to this (ie, firewalls, security groups, etc), and also suggested stunnel: http://<email address hidden>/msg00588.html. It would be good if the documentation or prominent man page made this clear.

Based on my review and an active upstream, ACK with suggested documentation change.

[James Page (james-page)]

Marking Won't Fix as Python Juju won't go to main.