dbExpand.pl outputs records in non-deterministic order

[Ben Franksen]

This concerns only the 7.0 branch since in 3.15 there is no dbExpand.pl.

The observed non-determinism makes it hard to compare (diff) generated db files if DB_OPT has been set in a Makefile.

The solution is simple, at the cost of making the procedure slightly less efficient:

diff --git a/modules/database/src/tools/DBD/Output.pm b/modules/database/src/tools/DBD/Output.pm
index 869b485..55051df 100644
--- a/modules/database/src/tools/DBD/Output.pm
+++ b/modules/database/src/tools/DBD/Output.pm
@@ -118,8 +118,7 @@ sub OutputBreaktables {

 sub OutputRecords {
     my ($out, $records) = @_;
- foreach my $name (sort(keys %$records)) {
- my $rec = $records->{$name};
+ while (my ($name, $rec) = each %{$records}) {
         next if $name ne $rec->name; # Alias
         printf $out "record(%s, \"%s\") {\n", $rec->recordtype->name, $name;
         printf $out " alias(\"%s\")\n", $_

Note: other output routines inside this module are similarly non-deterministic in their output, but they are not used by dbExpand.pl.

[Andrew Johnson (anj)]

The suggested solution makes the result deterministic but still changes the record order. Would it be preferable to keep a list of the records in the order in which they were parsed? That wouldn't be hard to do if desirable.

NB: The sense of the diff above is reversed, '-' lines normally show the existing code and '+' lines the new.

[Ben Franksen (bfrk)]

Am 06.01.20 um 20:45 schrieb Andrew Johnson:
> The suggested solution makes the result deterministic but still changes
> the record order. Would it be preferable to keep a list of the records
> in the order in which they were parsed? That wouldn't be hard to do if
> desirable.

An option that tells dbExpand.pl to keep the records in their original
order may be useful for some applications.

However, given that the standard purpose of dbExpand.pl is to create a
"normalized" version of the input db file via the DB_OPT flag (including
re-formatting and removing comments), additionally imposing an order
seems the more useful default behavior.BTW, this is also what the
predecessor dbst did (though with a different order: first by record
type, then by name, IIRC).

There also remains the question of whether to make the other output
commands deterministic, too.

> NB: The sense of the diff above is reversed, '-' lines normally show the
> existing code and '+' lines the new.

Sorry, I mixed up the order of commits in the git diff command.

[Andrew Johnson (anj)]

See https://github.com/epics-base/epics-base/pull/267 for a solution to this.