Escaping double quotes in database fields does not work

[Ralph Lange]

Reported by Rod Nussbaumer, Electronics Development Group, TRIUMF:

I am generating EPICS .db files from a perl script, and would like to be able to generate records which have DESC fields containing the '"' 'double-quote' character. Is this possible? What is the syntax? I have tried editing an otherwise working database file, and have not been able to create one that passes the database loader. Have tried using combinations of multiple d-quotes and escaping them with various numbers of '' 'backslash' characters.

I am using R3.14.8.2 on Scientific Linux 4.

Additional information:
Staring at the code in .../src/dbStatic/dbLex.l and the other source files there, I would say that escaping a double quote is supported in a half-assed way:
\\" doesn't end the string and hence is copied into the field, but no one ever takes out the backslash. Uh-oh.

OS: any

Platform: any

Version: R3.14.8.2

Original Mantis Bug: mantis-250
    http://www.aps.anl.gov/epics/mantis/view_bug_page.php?f_id=250

[Andrew Johnson (anj)]

ISTR that Marty didn't want to do all the character translations for every quoted string - the dbTranslateEscape() routine in libCom/misc/epicsString.c will do it, but it will to slow down database loading. KEK probably have a patch that inserts this at the right place since they worked on dbTranslateEscape().

[Ralph Lange (ralph-lange)]

Hmm ...
I can see the point, but I don't like the idea of having to patch base to make character escaping work.
What about a switch (global variable or - more OSIish - a function) to switch translation on or off? That way you could do it in the start-up file - and decide yourself if you do it for all IOCs, only for the IOCs that use it, or even only for the database files where you need it.
The default would be OFF (compatibility), maybe configurable in CONFIG_SITE.

[Andrew Johnson (anj)]

I added a variable dbTranslateFields to dbLexRoutines which controls whether field values are unescaped. The variable can be set using the iocsh 'var' command or using the regular vxWorks shell syntax.

Unfortunately while this does permit the use of escaped double quotes, other escaped constructs in a field value are causing a segfault. I hope I'm not going to have to redesign dbLex.l to get this working safely...

[Andrew Johnson (anj)]

The segfault was a buffer overflow from the "invalid character" message in dbLex.l but not the source of the original problem, which was incorrect patterns in the lexer. Fixed and simplified those, and decided that a switch to enable the escape translation shouldn't really be necessary so I took it out again. Also added a pattern and error message to capture strings that includes a newline, implying they're missing a closing quote. If you want a real newline in a field value, use backslash-n to put it there.

Note that the escaping is only done on record field values, not on anything else.

This fix will appear in R3.14.9

[Andrew Johnson (anj)]

R3.14.9 Released.