Re: Common Lisp and RegressionTesting

Thank you all for your responses. I appreciate the time and thought
you've put into them.

Though I feel like, with your help, I am closer to the answer I'm
looking for, as is often the case with USENET posts, I think I need to
clarify my original question.

I know that I am *supposed to* make sure that I modify all of the
references to foo when I change it to hoo. But unit tests are
supposed to guard me against all of those things I'm supposed to do
but forget to do. The example is perhaps simple, but it makes sense
if you see that it's possible that bar is in another file (which I
apologize for not making clear in my original post), possibly in a
different package, and that I just plain forgot about it in the
hypothetical situation. I make mistakes all the time.

I am not making this example up. I realize that it's a silly mistake
to make, and I apologize for what you may consider a silly question,
but I thought people with more experience than me would have more
insight than I have had into how to resolve the issue, and that I
might learn something from them. And I did: from these posts, I
realize that most people deal with it by keeping track of it all in
their heads, deleting the package in general or unbinding the symbol-
function, or else they reload the lisp image.

I want to make it clear that I do not blame Lisp for how it works.
Many people much smarter than me developed it the way it is for
reasons I perhaps will never fathom. I just want to fix a problem
with my code that I inadvertently created, and prevent myself from
making further silly mistakes.

It may come as a surprise to you, but I already do keep track of what
functions have been loaded, and how it differs from the text files I'm
editing. I realize that posts to newsgroups can sometimes lead to
arguments, misunderstandings, etc. However, I keep reviewing my post
to see what indications I gave that would make anyone speak in defense
of Common Lisp (which I truly love -- I go to bed thinking about how
Common Lisp is doing in the world). I don't see any insult or blame,
or any implications of such, in my original post, though I take all
responsibility for any perceived transgressions.

I was asking about the issue of one of the fundamental assumptions of
Regression Testing: that they do a relatively thorough job of allowing
changes to the code which do not alter the correctness of the code,
while alerting you to changes that do alter the correctness of the
code. Lisp is different from most other languages, in that it
maintains a dynamic environment, which exists apart from the files of
code inputted to it. There is therefore a breach of the Regression
Testing assumption. I want to rectify that, in some way. Being
human, I want as automated and as foolproof a way to do that as I can
get, while maintaining the joyous power that Common Lisp affords. I
was wondering if there was some mechanism *besides* the obvious and
error-prone you-just-have-to-deal-with-it approach. I am very much
for incremental improvement -- which means that I look for
opportunities and possibilities for improvement *despite* there
existing a de facto solution, and despite opposition from those who
have invested in their own ways.

Like I said before, I love Common Lisp. Without going into details, I
bliss out thinking about the potential for such a great language. I
just want to develop what Common Lisp already does really well, so
that it does more even better.

I hope I've given a proper introduction, which will allow me to
continue:

Some constraints:

* Unit tests are meant to test the *code*, and so must test a clean
lisp environment with only the files loaded (or some equivalent
approach)
* Unit tests are meant to be automated -- to prevent error and to
encourage running them often, so they must be easy to run (one
command, button, etc). Automated unit testing has improved many a
developer's life

I have, with all of your help, come up with a few solutions to the
above constraints:

* Create a shell script which loads all of the files of the
application into a new lisp image, then runs the tests, prints out the
results, then exits
Pros:
+ simple to implement
+ automated
Cons:
- loading some code/tests may take a long time
* In the file containing the tests, add code to delete and reload the
package containing the code to be tested as well code to run the
tests. A simple recompile of that file will flush the symbols from
that package, reload the symbols of that package from its code file,
and run the tests.
Example:
Say you have a file unit-tests.lisp which tests code in
package :package-to-test
unit-tests.lisp would contain
;; flush package
(eval-when (:compile-toplevel)
(delete-package :package-to-test)
(load "package-to-test")) ;; or some other
mechanism
;; unit tests here
.......

;; run tests
(run-tests)

Pros:
+ totally automated, pushbutton testing
Cons:
- Any? I don't know Lisp well enough to say
* Be sure to fmakunbound and/or uninter symbols when you no longer
need them
Pros:
+ will catch problems when you remember it
Cons:
- requires a lot of discipline
- if you forget it, you've broken the constraints of the
problem

Do you have any more solutions, or comments on the ones I've provided?

Thanks for you time.
Eric Normand
http://www.lispcast.com

.