Problem with test strategy involving *list-of-lists*

[Patrick Stein]

cl-test-grid shows failures for lw-6.0.0-linux-x86 ( http://cl-test-grid.appspot.com/blob?key=121029 )

In addition to the bug/walk.2 from all lisps Bug #1009817 ( https://bugs.launchpad.net/iterate/+bug/1009817 ), it also failed tests: "find-longest-list.1" "find-longest-list.2" "find-longest-list.3" "find-longest-list.4" "maximize.1" "maximize.2" "subblocks.maximize.1".

The common theme is that these are all of the tests that use maximize or maximizing. Oddly, the minimizing tests seem to work. But, from the looks of the error output, it may have just been unlucky. The test package makes *list-of-list* be 100 lists of randomly chosen lengths. But, it does this in an eval-when for compile-time load-toplevel and execute. The expected results are determined at read-time for the tests.
    (eval-when (:compile-toplevel :load-toplevel :execute)
      (defparameter *an-alist* '((a . 2) (b . 3) (zero . 10) (c . 4) (one . 20) (d . 5) (e . 99)))
      (defparameter *list-of-lists* (loop for i from 0 to 100
                                          collect (loop for len from 0 to (random 20)
                                                        collect len)))
      (defun longest-list (list1 list2)
        (if (< (length list2) (length list1))
            list1
            list2)))

And, for example:
    (deftest find-longest-list.1
        (iterate (for elt in *list-of-lists*)
                 (finding elt maximizing (length elt)))
      #.(reduce #'longest-list *list-of-lists*))

So, I'm thinking this is a dodgy test strategy.

Test ITERATE.TEST::FIND-LONGEST-LIST.1 failed
Form: (ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:FINDING ELT ITERATE:MAXIMIZING (LENGTH ELT)))
Expected value: (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19)
Actual value: (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18).

Test ITERATE.TEST::FIND-LONGEST-LIST.2 failed
Form: (ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:FINDING ELT ITERATE:MAXIMIZING (LENGTH ELT) ITERATE.TEST::INTO (ITERATE.TEST::E ITERATE.TEST::M)) (ITERATE:FINALLY (RETURN ITERATE.TEST::M)))
Expected value: 20
Actual value: 19.

Test ITERATE.TEST::FIND-LONGEST-LIST.3 failed
Form: (ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:FINDING ELT ITERATE:MAXIMIZING (FUNCTION LENGTH)))
Expected value: (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19)
Actual value: (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18).

Test ITERATE.TEST::FIND-LONGEST-LIST.4 failed
Form: (ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:FINDING ELT ITERATE:MAXIMIZING (FUNCTION LENGTH) ITERATE.TEST::INTO (ITERATE.TEST::E ITERATE.TEST::M)) (ITERATE:FINALLY (RETURN ITERATE.TEST::M)))
Expected value: 20
Actual value: 19.

Test ITERATE.TEST::MAXIMIZE.1 failed
Form: (ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:MAXIMIZING (LENGTH ELT) ITERATE.TEST::INTO ITERATE.TEST::M) (ITERATE:FINALLY (RETURN ITERATE.TEST::M)))
Expected value: 20
Actual value: 19.

Test ITERATE.TEST::MAXIMIZE.2 failed
Form: (ITERATE:ITERATE (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:MAXIMIZE (LENGTH ELT)))
Expected value: 20
Actual value: 19.
 ITERATE.TEST::FINDING.MINIMIZING.1 ITERATE.TEST::MINIMIZE.1
 ITERATE.TEST::MINIMIZE.2

Test ITERATE.TEST::SUBBLOCKS.MAXIMIZE.1 failed
Form: (ITERATE:ITER ITERATE.TEST::OUTER (ITERATE:FOR ELT ITERATE:IN ITERATE.TEST::*LIST-OF-LISTS*) (ITERATE:ITERATE ITERATE.TEST::RUNNING (ITERATE:FOR ITERATE.TEST::E ITERATE:IN ELT) (ITERATE:IN ITERATE.TEST::OUTER (ITERATE:MAXIMIZE ITERATE.TEST::E))) (ITERATE:MAXIMIZING (LENGTH ELT)))
Expected value: 20
Actual value: 19.