iterate tests regression on CLISP: finding.minimizing.1, minimize.1, minimize.2

[Anton Vodonosov]

There seems to be a regression on clisp-2.49-unix between quicklisp 2012-07-03 and quicklisp 2012-05-20.

In quicklisp 2012-05-20 CLISP had only one test failure: bug/walk.2
In quicklisp 2012-07-03 it additionally has 3 new: finding.minimizing.1 minimize.1 minimize.2

Logs:
https://cl-test-grid.appspot.com/blob?key=197101

Strange is that tests on clisp-2.49-win do not have this regression,
it still has only one failure - bug/walk.2
https://cl-test-grid.appspot.com/blob?key=226017

[Patrick Stein (nklein)]

I suspect this is the same as bug# 1009821. The problem is that a random lists of 100 integers (with the kth list's elements chosen randomly from 0 to k-1) is used for the minimize and maximize tests.

The test package goes to some efforts to try to get certain portions evaluated at read-time, but I suspect that some portions get re-evaluated later and the list is different enough to make a difference. I suspect if you re-ran the test on the same machine (maybe deleting the FASLs in between), you'd succeed.

Here is the relevant snippet from this test-grid blob:

Test ITERATE.TEST::FINDING.MINIMIZING.1 failed
Form:
(ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:FINDING ELT ITERATE:MINIMIZING #'LENGTH ITERATE.TEST::INTO (ITERATE.TEST::E ITERATE.TEST::M))
 (ITERATE:FINALLY (RETURN ITERATE.TEST::M)))
Expected value: 2
Actual value: 1.
Test ITERATE.TEST::MINIMIZE.1 failed
Form:
(ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:MINIMIZING (LENGTH ELT) ITERATE.TEST::INTO ITERATE.TEST::M)
 (ITERATE:FINALLY (RETURN ITERATE.TEST::M)))
Expected value: 2
Actual value: 1.
Test ITERATE.TEST::MINIMIZE.2 failed
Form: (ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:MINIMIZE (LENGTH ELT)))
Expected value: 2
Actual value: 1.

[Patrick Stein (nklein)]

Err... sorry, misspoke about bug# 1009821. I still think this is the same, but the relevant code creates a list of 100 lists of random lengths 1 to 20. Then, it goes through and finds the longest list in the maximize cases and the shortest list in the minimize cases. If that list of lists gets regenerated, there is no guarantee that it will come up with the same answer.

We should eliminate the randomness or seed it in these or at least fix the eval oddness.