Subj : Line-by-line testing, not Gries! (was: Not debugging?)
To   : comp.programming,comp.lang.java.programmer,comp.lang.lisp
From : rem642b
Date : Fri Sep 02 2005 10:34 pm

> From: George Neuner <gneune...@comcast.net>
> I support testing and assertions wherever possible, including
> intra-procedure if there is some meaningful test that can be done on a
> partial result - but testing line by line is ridiculous.

I disagree 99%. What I do most of the time is write one line of code,
then execute it to see if it did what I wanted it to. If it did, I
write the next line of code, which depends on the (already seen to be
correct) result from the first line of code, then I execute it to see
if it also did what I wanted it to. Repeat as many lines of code as
needed to finish the function these line will go into. By the time I am
ready to wrap a function declaration form around them, every line of
code works for that first test case, so I know for sure they are
connected properly, and testing additional boundary cases would then
make sure any decisions within my code, or any boundary cases of the
individual statements, are correct.

So what you do is write one line of code, then without checking whether
it works at all you go ahead to write another line that depends on it,
then another, then another, then another, etc., until you have enough a
complete function definition, at which point you wrap a function
definition form around the code and perform the very first test of all
that code as a whole unit, and if you have a bug you have to guess
which of those several lines of code, or which dependency between two
adjacent lines of code, might be the problem?? That sounds stupid
compared to how I do it.

I say 99% disagree, because once in a while I need to write a function
where every line of code is so trivial that I can write the whole bunch
of code faster than I can test each part individually. This hardly ever
happens with inline code. Usually it happens only with mapping
functions, whereby I treat the whole mapping operation as if it were a
single line of code. If the entire function has just that one "line" of
code, I might write the function wrapper immediately after the inner
code, and test the whole one-line-of-code function as a unit. But more
often the one "line" of code including the mapping call is not the
whole function, so I indeed test that "line" of code individually, and
other lines of code before/after it likewise, and only write the
function definition wrapper after all the lines of code are tested.

Here's an example of a mapping operation which is sufficiently complex
that I probably tested the innerds individually before putting the
pieces together into the mapc form:
    (mapc
      (function (lambda (ip)
        (let ((ix (position ip iptable :test (function ipinrange))))
          (cond (ix (cond (verbose (format t " ~D " ix))) (push ip revmatch))
                (t (cond (verbose (format t " x "))) (push ip revmiss))
                ))
        ))
      ip3nums)
This was written several years ago so I can't remember exactly how I
did it at the time, I'm just expressing what I probably would have done.

By comparison, here's an example where I probably wrote and tested the
entire function at one time (after the key function it calls had
already been fully unit-tested):
(defun one-nicid-find-eadrs (nicid res2)
  (mapcan (function (lambda (rec2)
      (one-nicid-one-rec2-maybe-eadr nicid rec2)))
    res2))

> It may be extremely difficult or quite impossible to figure out what
> would happen if a particular line of code is wrong or missing.

That's a "red herring". If there is a setup followed by three lines of code:
;TestRig: (setq x0 '(sometestdata))
  (setq x1 (dosomething x0))           ;first loc
  (setq x2 (dosomethingelse x1))       ;middle "
  (setq x3 (dosomethingdifferent x2))  ;last   "
then it's obvious what happens if the middle line of code is missing:
The last line of code will encounter an unbound variable and signal
that error. How is that difficult or impossible to figure out?? If the
middle line of code is wrong because it can't accept the kind of data
in x1, then it'll signal an error when it's tested. If the middle line
of code is wrong because it produces wrong output, and you don't notice
it's wrong when testing it, surely when you run the last line of code
it'll signal an error or produce obviously wrong output. If no errors
are signalled, then you pay special attention to the final output from
the very last line of code intended to be in your function, and you'll
notice any bugs up to that point, and then looking back at the
intermediate results you can figure out where you went wrong. There's
no need to figure out, in advance, what would happen if a particular
line of code is wrong. You just try it and it's blatantly obvious
either immediately or very shortly that it was wrong.

> It's also unwieldy because such tests frequently can't be batched but
> must be executed in-line due to following code altering the test
> conditions.

Line-by-line unit testing is done only during initial coding of each
new function, or when making a drastic refactoring that requires you to
check the function all over again from scratch. There's nothing
unwieldy about executing each line of code individually, assuming
you're using any decent live-interactive programming environment such
as Lisp read-eval-print or Java BeanShell.

On the other hand, assertions can be left in the code permanently, and
they aren't unwieldy in any case, regardless of programming
environment.

> I didn't agree with this "line by line proof" approach in 1981 when
> Gries proposed it in "The Science of Programming" and I don't agree
> with it now.  YMMV.

I haven't seen the writing by Gries. Is a summary online?
Google search gets:
  http://www.springeronline.com/sgw/cda/frontpage/0,10735,4-40109-22-1332860-0,00.html
   This is the very first book to discuss the theory and principles of
   computer programming on the basis of the idea that a proof of
   correctness and a program should be developed hand in hand. ...
   Propositions
   and predicate calculus are presented as a took [sic] for the programmer,
   rather than simply an object of study.
This is something very different from line by line testing. Constructing
a formal proof of correctness it nothing at all like simply testing a
piece of code under a variety of typical situations, both valid and
invalid, to get a pretty good idea that indeed it's all working
correctly, producing good results with good data, and signalling a
clearly worded explanatory error if given bad data.

> I use [sic] to do image processing in which the processing was a dependent
> chain of fuzzy logic.  All the possible answers were wrong in some
> absolute sense - and the object was to find the answer that was least
> wrong.  If I missed a minor step or had a bug in a calculation
> somewhere, the chances are I wouldn't be able to tell by looking at
> the intermediate results.

It might be interesting to revisit that code, refactoring it to clearly
use a bottom-up tool-building design, whereby you can map the tiny
tools across contrived synthesized images, starting with clean
geometric images, then noisy geometric images, then clean fractally
generated landscapes, and finally noisy fractally generated landscapes,
to test whether they produce the expected outputs as inherent in the
synthesizer input. Also, natural images can be fed through the image
processing to create an abstract model, then that abstract model can be
fed back into the synthesizer to see if that synthetic image when
re-processed produces the same model gotten the first time around.

.