NPP Put to a pp(TRUE) VAL field doesn't trigger monitors

[Andrew Johnson]

There is code in dbAccess.c::dbPut() that calls db_post_events() after writing to a field. This code is conditional, and explicitly avoids posting a monitor if the put was to the record's VAL field and that field is marked pp(TRUE); if the put came from a CA client the record is about to be processed anyway, so this conditional prevents 2 monitor events from being generated by the same put.

However if the put was actually from a DB link marked NPP, the above logic is wrong because the record is not about to be processed. This behavior is somewhat obscure, and does catch out database designers (it came up again at APS just today).

It is possible for the DB link to call db_post_events() itself in this case, but it isn't obvious whether it should or not. When I add the necessary code the result looks a bit strange because the record's VAL changes but the alarm state and timestamp do not. This could also conflict with the monitor deadband processing as this monitor will happen every time the put occurs and will not update the MLST field.

I will attach the necessary code changes to this bug report for the 3.16 branch; earlier branches would need it moving since the modifications are to the DB link type, and I don't think it should go in any earlier than 3.16 anyhow.

Opinions please, should this be applied or not?

[Ralph Lange (ralph-lange)]

The side effect is even worse for an archiver:
The archiving engine will either reject the value (because the timestamp is too old) or take the value, which creates wrong archived data (showing the new value as if it was set at the old timestamp).

[Andrew Johnson (anj)]

Would it make any significant difference if the link only posted a DBE_VALUE event and not DBE_LOG (or would that be even more confusing)? Does the Archive Appliance even use DBE_LOG for its subscriptions? I guess that wouldn't affect any channels archived using periodic ca_gets instead of monitors, but those channels are already seeing a new value with the old timestamp from a DB-link put with NPP, so I don't see a huge change here.

To be clear, I'm not necessarily pushing to get this in, but it would clean up one of our (many) corner cases.

[mdavidsaver (mdavidsaver)]

I'm not sure I follow the situation which triggers mis-behavior. Can you give an example? I'd like to understand if this has any implications for my new PVA link code.

> ... would that be even more confusing)?

Yes

[Andrew Johnson (anj)]

Load this database:

record(ai, target) {
  field(TPRO, 1)
}
record(ao, put-val) {
  field(OUT, "target.VAL NPP")
}
record(ao, put-rval) {
  field(OUT, "target.RVAL NPP")
}

Start "camonitor target.VAL target.RVAL".

Compare the difference in camonitor output between doing "dbpf put-val 1" (no monitor triggers) and "dbpf put-rval 1" (monitor is triggered on the RVAL field). In neither case is the target record actually processed (no output from having set TPRO) so the timestamp is still <undefined> as it should be, but clients monitoring both fields only get notified when the RVAL field changes, not for VAL.

I don't see any implications for PVA links, although there might be for QSRV if it allows a client to put to a field without triggering record processing. I assume it's using the dbChannel API to talk to the database; if it always calls dbChannelPutField() then QSRV is going to behave like RSRV and the pp() attribute of the field determines whether the record gets processed or not.

If QSRV calls dbChannelPut() then it gets to decide whether to process the record or not (and it has to do that itself). In both cases though the underlying dbPut() routine that writes to the field will call db_post_events() on the field, unless the field is VAL and the pp() attribute is true. For external CA clients that logic is appropriate because the record's process() routine will call db_post_events() soon afterwards anyway, and this avoids sending two monitor events.

[Andrew Johnson (anj)]

Core Group review at ESS: Post more information about the use-case or DB change that triggered this.

[Andrew Johnson (anj)]

I had another report of this behavior from an APS developer.

He has an aSub record which is writing a scalar from VALA via a DB link in OUTA to a passive ai record. He has to set the PP flag on the OUTA link for his OPI screens to see the change in the ai.VAL field, even though a caget of that field demonstrates that the put has succeeded. If he was writing to any field other than VAL there would be no need to set PP for monitors to fire automatically and the OPI screens to update.

I take Ralph's point that not processing the record results in its timestamp never getting set and alarms never being checked, which are good reasons to tell users they should be setting PP. However this is evidently a cause for user confusion. Before I withdraw this bug is there any way we can help users to understand the current behavior better? The confusion is caused by the isValueField conditional towards the end of dbPut():

    if (precord->mlis.count &&
        !(isValueField && pfldDes->process_passive))
        db_post_events(precord, pfieldsave, DBE_VALUE | DBE_LOG);

I note that DB links don't have any way to say "follow the target field's pp() setting" although that could be implemented relatively easily with a new (RPP or TPP?) link flag. Would it be worth adding something like that? I might even be inclined to make that the default in the absence of either PP/NPP flag, but doing so might need a warning at load-time to avoid breaking too many existing databases.

[Ralph Lange (ralph-lange)]

IMHO changing the default is a no-no. That would break way too many databases.

The only way toward that would be deprecating any default for a few years, forcing everyone to explicitly set NPP/PP/TPP. Then, in a second step (if at all), a new default could be established.

[mdavidsaver (mdavidsaver)]

Looking back...

wrt. QSRV

Before I was thinking of a different special condition from dbPutField() which I do replicate in QSRV to decide whether a Put should dbProcess(). (since I can't use dbPutField() with groups) In the case of QSRV, this is a default which the client can override with 'record._options.process' in a pvRequest blob.

> if (paddr->pfield == &precord->proc ||
> (pfldDes->process_passive &&
> precord->scan == 0 &&
> dbrType < DBR_PUT_ACKT)) {

wrt. PP/NPP while I don't like these (seemingly) odd conditions in dbPut() or dbPutField() or dbNotify. Working around one by adding another to dbDbPutValue() does not seem like a sustainable way forward.

[Andrew Johnson (anj)]

I hope we all agree that database designers need to be able to specify whether a put through a DB link should trigger processing of the target record or not, hence the PP/NPP flags supported by the DBF_OUTLINK. I don't see any real alternatives to that, a new TPP flag isn't that compelling.

The per-field pp(TRUE) mechanism used for CA put operations does decouple the IOCs from their clients. I'm now particularly possessive of that design but it has worked well enough up to now, and it means that EPICS GUIs never needed to know anything about the database they're connecting to, which I see as a good thing. It might be nice if a database designer could configure that behavior on a per-record-field basis, but implementing that could be tricky and/or slow.

@Michael Does QSRV default to using the pp(TRUE) value? Do any GUIs that support PVA provide the ability to override the process flag?

The isValueField conditional at the end of dbPut() was added so a CA put to a record's VAL field doesn't trigger 2 monitor events, one from dbPut() and the second from the record's monitor() routine called from process(). The first monitor event is suppressed when the target field is VAL and it is marked pp(TRUE) which means a CA put would process the record. My original patch for this undid that suppression if necessary when the put actually comes from a DB link which is NPP.

[Andrew Johnson (anj)]

Tim Mooney wrote:

> I thought the reason the VAL field was only posted by the record was to give
> the record the ability to refuse a value written to it and not have anyone
> else notified that the value had been written.

That could very well have been the original reasoning, it looks like puts to the VAL field have behaved like that since the very earliest commits I have access to (1990).

I now accept that the existing behavior is correct, but it is confusing to users that DB puts to the VAL field behave differently than to every other field.

[Ralph Lange (ralph-lange)]

The only reason why I was considering a "TPP" option: it fills a gap in the available functionality for DB links. You can make them PP or NPP, but you can't make them work like CA links do. Which is why you cannot split a DB on two IOCs without thoroughly checking for unexpected side effects.

Other than that, I have no use case, so my point is admittedly somewhat academic.