Ejabberd strips custom XML attributes

[Bill Erickson]

Ubuntu 18.04 -- Breaking off from bug #1703411.

The version of Ejabberd that ships with Ubuntu 18.04 no longer allows custom XML attributes to live in XMPP elements. This prevents OpenSRF from passing commands to the router (router_command, router_class, etc.) Custom values must be passed in a custom element, which may contain its own attributes.

https://github.com/processone/ejabberd/issues/1692

Mike pushed code here pending further testing:

http://git.evergreen-ils.org/?p=working/OpenSRF.git;a=shortlog;h=refs/heads/user/miker/lp-1703411-xmpp-opensrf-message-subelement

[Bill Erickson (berick)]

Follow-up branch pushed:

http://git.evergreen-ils.org/?p=working/OpenSRF.git;a=shortlog;h=refs/heads/user/berick/lp1793356-xmpp-opensrf-message-subelement

1. I squashed and signed-off on Mike's 2 commits. The two commits can easily be untangled from the original branch, but given this makes opensrf XMPP compliant, I'm inclined not to support both attribute-passing mechanisms going forward.

This would mean sites with multiple servers that communicate via XMPP internally will have to be upgraded at the same time or they won't be able to communicate with one another.

2. Pushed a second commit with a number of small fixes, including a tiny ancient memory leak.

After applying fixes, I was able to use the system normally. I tested on Ubuntu 16.04 to confirm at was backwards-compatible with older Ejabberd versions.

Testing on Ubuntu 18.04 still needed.

I did not test the Java change and the Python libs remain unchanged. I believe both of these libs are in disrepair as it stands after the chunking/bundling changes in recent versions of OpenSRF.

[Jason Stephenson (jstephenson)]

I can test on Ubuntu 18.04.

[Jason Stephenson (jstephenson)]

I get the following on Ubuntu 18.04:

opensrf@egdev:~/OpenSRF$ osrf_control -l --diagnostic
* ERR opensrf.dbmath Has PID file entry [7687], which matches no running opensrf.dbmath processes
* ERR opensrf.math Has PID file entry [7678], which matches no running opensrf.math processes
* opensrf.persist [7645] uptime=00:13 cputime=00:00:00 #drones=5/25 20%
* opensrf.settings [7637] uptime=00:13 cputime=00:00:00 #drones=5/15 33%
* opensrf.validator [7658] uptime=00:13 cputime=00:00:00 #drones=5/15 33%
* router [7629] uptime=00:15 cputime=00:00:00
* router [7630] uptime=00:15 cputime=00:00:00

Logs are attached.

[Bill Erickson (berick)]

Thanks, Jason. I'm seeing errors like:

opensrf.math 2018-09-19 16:17:33 [ERR :7678:osrf_application.c:151:] Failed to dlopen library file libosrf_math.so: libosrf_math.so: cannot open shared object file: No such file or directory

Ubuntu 18.04 issues unrelated to Ejabberd, perhaps?

[Jason Stephenson (jstephenson)]

Thanks, Bill. I'll take another look tomorrow. Maybe I missed ldconfig or need to run it again or something. I noticed some weirdness with ldconfig on Debian 9 when installing multiple times.

[Ben Shum (bshum)]

Ubuntu 18.04 freshly installed VM worked for me with this branch, and I was able to get a successful test conducted with opensrf.math service.

[Jason Stephenson (jstephenson)]

Nope. Still not working for me. Same errors after running ldconfig, again, rebuilding, relinking, etc.

Must be something wrong with my vm.

[Bill Erickson (berick)]

Thanks, Ben. Given your successful test, I'm going to add a pullrequest for full visibility.

Are there any concerns about not maintaining compatibility with older opensrf XMPP clients? Note this won't affect clients talking via the gateways, only via XMPP/Ejabberd, which is typically contained inside the firewall, so to speak.

[Mike Rylander (mrylander)]

I'd prefer a version number bump because of backward compatibility breaking, personally, but a big, loud release note addition may be enough.

Thanks, Bill and Ben (and Jason, preemptively)!

[Jason Stephenson (jstephenson)]

Mike, the bug is currently targeted at OpenSRF 3.1-beta. Is 3.1 not a big enough version bump? Do you think it should go to 4.0?

[Jason Stephenson (jstephenson)]

I tested this with a fresh Ubuntu 18.04 virtual machine, and it works for me. I pushed a signoff branch to collab/dyrcona/lp1793356-xmpp-opensrf-message-subelement-signoff

http://git.evergreen-ils.org/?p=working/OpenSRF.git;a=shortlog;h=refs/heads/collab/dyrcona/lp1793356-xmpp-opensrf-message-subelement-signoff

[Mike Rylander (mrylander)]

Regarding versioning, if I were the Dictator of OpenSRF (tm) I'd probably go with 4.0 because of the client breaking. However, since Evergreen is the only (known) project that has OpenSRF as a dep, we have more lee-way. At the other end, one feature-y thing that may tip the balance toward 4.0 is websocketd support. Is that baked in yet? I'll have to go look...

[Jason Stephenson (jstephenson)]

Even though I have already signed off on this branch, I have noticed that the check_transport_message test fails with this branch applied.

It looks like the hard coded messages on lines 114 and 202 need to be updated.

[Bill Erickson (berick)]

I pushed a commit back to Jason's branch with fixes for 'make check':

http://git.evergreen-ils.org/?p=working/OpenSRF.git;a=shortlog;h=refs/heads/collab/dyrcona/lp1793356-xmpp-opensrf-message-subelement-signoff

[Jason Stephenson (jstephenson)]

I have pulled Bill's fix and run the tests. They pass again. I added my sign-off and force pushed back to the collab branch (an implicit no-no, I know). I think this is ready to go, but there is a question of the version number to be hashed out. On that front, I think 3.1 is fine, but I also think 4.0 is fine.

[Mike Rylander (mrylander)]

I'd rather have the code out there than argue over the color of the shed it lives in... I'll merge to master now and we can call it 3.1.

Thanks, Bill, Ben and Jason!

[Job Diógenes Ribeiro Borges (jobdrb-gmail)]

Hi :)

I would like to test this new code, but, Mike, I did not found 3.1 in GitHub.

github master has this changes?

[Ben Shum (bshum)]

Hi! Github's OpenSRF mirror does have these changes in the master branch: https://github.com/evergreen-library-system/OpenSRF

The Evergreen community git server also has the changes in the master branch: http://git.evergreen-ils.org/?p=OpenSRF.git;a=summary

Looks like OpenSRF 3.1 beta is still only a target in Launchpad, it has not been made into a rel_3_1 branch quite yet or tagged for release.