Irregular HTML entities in fm_IDL.xml

[Blake GH]

EG Master

fm_IDL.xml contains bare queries with the greater than and less than symbols sometimes escaped and sometimes not.

example from the
<class id="ocirccount"
section

((fine_interval >= '1 day' AND due_date >= 'today') OR (fine_interval &lt; '1 day' AND due_date > 'now'))

this example shows us that there is a mixture of html entities and non html entities.

It's not clear if this is causing widespread issues, however I have an example where this caused an error for postgres which is how I found this issue.

[Mike Rylander (mrylander)]

I suspect some IDL views just need to be wrapped in a CDATA section. Note, only bare <s /must/ be escaped to avoid breaking xml, so what you're seeing is intentional, not "mixed".

[Jeff Godin (jgodin)]

Blake, can you provide the "example where this caused an error for postgres"?

[Jason Stephenson (jstephenson)]

As discussed in IRC: http://irc.evergreen-ils.org/evergreen/2018-03-06#i_349015

It is my opinion that all SQL queries in fm_IDL.xml should be wrapped in CDATA sections. It is far better that we do this for all of them rather than just some of them.

I'll throw up a branch later, unless someone else beats me to it.

[Blake GH (bmagic)]

2018-03-06 13:08:59 EST ERROR: syntax error at or near ";" at character 164
2018-03-06 13:08:59 EST STATEMENT: SELECT usr,
                SUM(
                    CASE
                        WHEN (
                            ((fine_interval >= '1 day' AND due_date >= 'today') OR (fine_interval &lt; '1 day' AND due_date > 'now'))
                            AND (stop_fines IS NULL OR stop_fines NOT IN ('LOST','CLAIMSRETURNED','LONGOVERDUE'))
                        ) THEN 1
                        ELSE 0
                    END
                ) AS out,

                SUM(
                    CASE
                        WHEN (
                            ((fine_interval >= '1 day' AND due_date &lt; 'today') OR (fine_interval &lt; '1 day' AND due_date &lt; 'now'))
                            AND (stop_fines IS NULL OR stop_fines NOT IN ('LOST','CLAIMSRETURNED','LONGOVERDUE'))
                        ) THEN 1
                        ELSE 0
                    END
                ) AS overdue,

                SUM( CASE WHEN (xact_finish IS NULL AND stop_fines = 'LOST') THEN 1 ELSE 0 END) AS lost,
                SUM( CASE WHEN stop_fines = 'CLAIMSRETURNED' THEN 1 ELSE 0 END) AS claims_returned,
                SUM( CASE WHEN (xact_finish IS NULL AND stop_fines = 'LONGOVERDUE') THEN 1 ELSE 0 END) AS long_overdue
          FROM action.circulation
          WHERE checkin_time IS NULL
          GROUP BY 1
          limit 10

[Jason Stephenson (jstephenson)]

Well, it is later!

I made 3 branches just in case there were patch-affecting IDL changes between master, rel_3_1, and rel_3_0. There weren't.

Master: http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dyrcona/lp1753813-idl-source-def-cdata

Rel_3_1: http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dyrcona/lp1753813-idl-source-def-cdata_rel_3_1

Rel_3_0: http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dyrcona/lp1753813-idl-source-def-cdata_rel_3_0

This is a pretty easy one to test: Apply the patch and verify that the IDL is still valid XML.

I have targeted the bug at 3.0, 3.1, and 3.2-rc assuming that this is actually a bug. If anyone disagrees, I am not opposed to not backporting, but I think it should at least make it into 3.next as we really should wrap all SQL in the IDL with CDATA sections. Perhaps we should also document that recommendation somewhere?

[Ben Shum (bshum)]

Picked to master and backported to rel_3_2 and rel_3_1. I think having a consistent approach sounds like a good idea for the future.