dbExpand.pl outputs records in non-deterministic order

Bug #1858467 reported by Ben Franksen on 2020-01-06
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
EPICS Base
Status tracked in 7.0
7.0
Medium
Andrew Johnson

Bug Description

This concerns only the 7.0 branch since in 3.15 there is no dbExpand.pl.

The observed non-determinism makes it hard to compare (diff) generated db files if DB_OPT has been set in a Makefile.

The solution is simple, at the cost of making the procedure slightly less efficient:

diff --git a/modules/database/src/tools/DBD/Output.pm b/modules/database/src/tools/DBD/Output.pm
index 869b485..55051df 100644
--- a/modules/database/src/tools/DBD/Output.pm
+++ b/modules/database/src/tools/DBD/Output.pm
@@ -118,8 +118,7 @@ sub OutputBreaktables {

 sub OutputRecords {
     my ($out, $records) = @_;
- foreach my $name (sort(keys %$records)) {
- my $rec = $records->{$name};
+ while (my ($name, $rec) = each %{$records}) {
         next if $name ne $rec->name; # Alias
         printf $out "record(%s, \"%s\") {\n", $rec->recordtype->name, $name;
         printf $out " alias(\"%s\")\n", $_

Note: other output routines inside this module are similarly non-deterministic in their output, but they are not used by dbExpand.pl.

Andrew Johnson (anj) on 2020-01-06
Changed in epics-base:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Andrew Johnson (anj)
Andrew Johnson (anj) wrote :

The suggested solution makes the result deterministic but still changes the record order. Would it be preferable to keep a list of the records in the order in which they were parsed? That wouldn't be hard to do if desirable.

NB: The sense of the diff above is reversed, '-' lines normally show the existing code and '+' lines the new.

Am 06.01.20 um 20:45 schrieb Andrew Johnson:
> The suggested solution makes the result deterministic but still changes
> the record order. Would it be preferable to keep a list of the records
> in the order in which they were parsed? That wouldn't be hard to do if
> desirable.

An option that tells dbExpand.pl to keep the records in their original
order may be useful for some applications.

However, given that the standard purpose of dbExpand.pl is to create a
"normalized" version of the input db file via the DB_OPT flag (including
re-formatting and removing comments), additionally imposing an order
seems the more useful default behavior.BTW, this is also what the
predecessor dbst did (though with a different order: first by record
type, then by name, IIRC).

There also remains the question of whether to make the other output
commands deterministic, too.

> NB: The sense of the diff above is reversed, '-' lines normally show the
> existing code and '+' lines the new.

Sorry, I mixed up the order of commits in the git diff command.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers