RFE: :initially-do and :finally-do

[sds]

mk:defsystem in clocc has options :initially-do and :finally-do which allow executing code before and after compiling.
it also offers mk:system-source-size (which prints the system size nicely - files, bytes &c), mk:files-in-system, which tells which files needs to be recompiled (inter alia).
this allows me to do regression testing after each compilation on the modified files only.
this also allows me to regenerate autoload files only after compilation.

see clocc/src/cllib/cllib.system:

(let (tbc) ; To Be Compiled

(mk:defsystem cllib
    :depends-on (port metering)
    :components
    ((:file "animals" :depends-on
            ("base" "string" "miscprint" "fileio" "closio" "symb"))
     ............................................................
     (:file "xml" :depends-on
            ("base" "string" "withtype" "closio" "log" "fileio" "url")))
    :initially-do
    (progn (mk:system-source-size :cllib :all) ; nice output
           (mk:system-source-size :cllib :new-source-and-dependents) ; nice output
           (setq tbc (mk:files-in-system :cllib :new-source-and-dependents
                                         :source))) ; IMPORTANT: save what is to be compiled for regression testing
    :finally-do
    (when tbc
      (in-package :cllib)
      (when (member "tests" tbc :test #'string= :key #'pathname-name)
        (funcall (intern "TEST-ALL" :cllib) :what
                 (mapcar #'pathname-name tbc)))
      (let ((auto (translate-logical-pathname "clocc:src;cllib;auto.lisp")))
        (funcall (intern "AUTOLOAD-GENERATE" :cllib)
                 (mk:files-in-system :cllib) auto)
        (compile-file auto))))

) ; tbc

[Robert P. Goldman (rpgoldman)]

Request: Please break this unitary ticket into two "bite-sized" tickets that can be individually checked off (or possibly three). Otherwise, if we were to add FINALLY-DO, e.g., but not add FILES-IN-SYSTEM, this would hang around forever.

The things you want (IIUC) are:

1. INITIALLY-DO -- ASDF already has DO-FIRST for this purpose, but this may not work well (it's somewhat poorly understood), and may not be fully supported going forward. I'd regard it as "not exported" for now.

2. FINALLY-DO

Both of the above need refinement. I don't see how they work in MK-DEFSYSTEM because I don't know what they are relative to in ASDF terms. LOAD-OP? COMPILE-OP? TEST-OP? DOC-OP?

Looks like the FINALLY-DO example might be partially replaceable simply by defining TEST-OP. I think autoload generation could probably be handled by a PERFORM method, as well.

3. Addition of FILES-IN-SYSTEM. This should /definitely/ be pushed to another ticket. Also we need more semantics for it. Are the FASLs part of FILES-IN-SYSTEM? Only the source-files? What about STATIC-FILES? C-SOURCE-FILES?

[sds (sds-gnu)]

see bug#620515

I see initially-do and finally-do as :before and :after methods
they should be able to share information, so I suggest the following semantics:
:initially-do should be a function of 2 arguments: system & operation
:finally-do should be a function of 3 arguments: system, operation, & the return value of the initially-do invocation (if any)

[Faré (fahree)]

Sounds good indeed. Now for someone to do it... not high on my personal priority list, I fear.

Also, I don't think forcing a test after every build is necessarily a good idea (though registering what needs to be tested might be). Sometimes you do want to build in an environment less suitable for tests, and test somewhere in an environment more suitable.

[sds (sds-gnu)]

On 8/19/10, Faré <email address hidden> wrote:
> Sounds good indeed. Now for someone to do it... not high on my personal
> priority list, I fear.

is this "irresponsibility" or "territoriality"? :-)

> Also, I don't think forcing a test after every build is necessarily a
> good idea (though registering what needs to be tested might be).
> Sometimes you do want to build in an environment less suitable for
> tests, and test somewhere in an environment more suitable.

when I call asdf (as opposed to compiling the individual function or a
file) it means that I have reached a certain milestone which warrants
a regression test run, even if it is relatively expensive.

--
Sam Steingold <http://sds.podval.org>

[Faré (fahree)]

I'm not territorial on that - I'll gladly let other people hack on that code, and I'll even merge any patch someone sends me that implements these features.

Responsibility - I care about the interface. Now, we may agree to disagree on matters of taste.

I understand that when YOU recompile, you want to test. On some systems, I can say the same. That's why people have (asdf:test-system :foo) or (asdf:load-system :foo-test) instead of (asdf:load-system :foo).

[sds (sds-gnu)]

On 8/20/10, Faré <email address hidden> wrote:
>
> I understand that when YOU recompile, you want to test. On some systems,
> I can say the same. That's why people have (asdf:test-system :foo) or
> (asdf:load-system :foo-test) instead of (asdf:load-system :foo).

the absolutely critical part here is the introspection.
I want to test ONLY those parts which were recompiled.
that's the whole point.
I can easily run a full test whenever I compile-system,
but I only want to check the parts which have been changed!

--
Sam Steingold <http://sds.podval.org>

[Robert P. Goldman (rpgoldman)]

Suggestion: consider adding a new subtype of test-op like incremental-test-op or add special methods on TEST-OP for your system to give the incremental testing behavior you want. I don't see why INITIALLY-DO (or, in ASDF lingo "DO-FIRST") and FINALLY-DO are necessary for this.

Indeed, a PERFORM on a SYSTEM (or a MODULE) *already* has the semantics of FINALLY-DO, because of the odd semantics of PERFORMing an OPERATION on a MODULE....

I'm inclined to think that ASDF already has the features you need to do this (although I agree that it is difficult to discover that fact because of limitations in the manual).

[Robert P. Goldman (rpgoldman)]

[Sorry, reading a little out of order...]

If INITIALLY-DO (DO-FIRST) and FINALLY-DO are just before and after methods, why can't they be implemented in that way? Why is it that we need to add these?

[sds (sds-gnu)]

On 8/20/10, Robert P. Goldman <email address hidden> wrote:
>
> If INITIALLY-DO (DO-FIRST) and FINALLY-DO are just before and after
> methods, why can't they be implemented in that way? Why is it that we
> need to add these?

ease of passing information collected in do-first (list of modules to
be compiled) to finally-do (testing those modules).

--
Sam Steingold <http://sds.podval.org>

[Faré (fahree)]

In this particular case, I would
(defmethod operate ((o test-op) (c (eql (find-system :foo))))
   (let* ((modified (files-in-system :foo ...))
           (tests (tests-for-modified-files modified)))
      (load-system :foo)
      (run-tests tests))))

or something.