Newsgroups: comp.software-eng
Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!uupsi!ficc!rogers
From: rogers@ficc.ferranti.com (keith rogers)
Subject: Re: Personal growth and software engineering!
Message-ID: <+CTADP6@xds6.ferranti.com>
Summary: metrics, et al (long response!)
Organization: Ferranti International Controls Corporation
References: <JGAUTIER.91Apr8184934@vangogh.ads.com> <35017@athertn.Atherton.COM> <1991Apr15.200458.11331@dg-rtp.dg.com>
Date: Thu, 18 Apr 91 13:42:40 GMT

Some comments on various postings (all pasted together - sorry!).... 

> If the metrics are bogus, then fix them by including the workers
> in the process.

"Bogus" may be in the eye of the beholder.  Failure to use the
numbers (and this includes being too concerned with the collection
of data, to the expense of analysis) has probably doomed more 
metrics than "bogusness." 

>          ....  Most processes are too informal to measure effectively.  

If the process is too informal to be measured, then either the
process should be formalized, and made measurable, or you should
decide, a priori, to not care too much what the result/output of 
the process will be.  Without some definition of what is going 
on in a process, you cannot expect to control it, or its output.

> Measurement is a useful approach in some circumstances.  However, if
> measurements don't "tell you what to do" about your process they can
> be misleading at best.

Measurements are unlikely to "tell you what to do."  Most "misleading"
is really misinterpretation; i.e., the failure to do the proper analysis, 
and then use the data correctly, perhaps to improve the measurements.

>           ... software engineering is a pseudo-science, right up there
> with, and in the grand style of, say, phrenology.

You have to admit that for all the failed sciences, a few did pan out.
I mean, who was to know whether phrenology was right or wrong at the
time, ludicrous as it seems now.  Invasive surgery probably seemed
a little radical, too, along with others.  And, maybe metrics will
be relegated to the dustbin when it's all said and done.  But, if your
alternative is to toss a specification "over the wall" and say a prayer,
I'd put that in the same class as 19th century rainmaking :-).

> [ ... lots of good stuff deleted ... ]
> We want answers, not more questions.  Since metrics like these raise more
> questions, we don't want them.  In fact, we don't want them so strongly,
> that we ignore useful information in them that would have to be derived.
> [ ... more good stuff deleted ... ]

Yes!

> There are metrics that tell you about your process right now: but in
> most cases, they are very crude approximations, or they are based on
> assumptions that are little more than guesswork.  If you change your
> process based on them, you may be as likely to worsen as to improve it.

Right.  But if you are changing your process without understanding it,
or knowing whether the variation you have measured is due to special
(i.e., non-process related) or common causes, then you deserve what you get.

> 1.            .... The defect-is-a-defect-is-a-defect mentality doesn't
> reflect the way we view products -- software or not.

Who's "we?"  (I believe defects are usually classified by level of severity.) 

> 2. How do we measure productivity in rev n+1 software?

Productivity is difficult, but it's very important.  If you bid a project 
that involves modifications to your base product, you need to be able to 
estimate how long the work will take.  Then you need to measure, somehow, 
the amount of work actually done, so that when you ultimately find out 
that your estimate wasn't [very good], you have some likelihood of doing 
a better job on your next estimate, before you go out of business.  

> 3. Who cares what the lines-per-timeframe is? or the defects-per-line
> if the software is delivered per the agreed-upon schedule and with the
> agreed-upon level of quality ...

Who cares?  The banks, for one.  If you have to hire twice as many people
to make that agreed upon schedule, and your bottom line starts showing
up in red, you won't be around to sell another high-quality product.

> We can measure and 'quantify' all day, but the bottom line is customer
> satisfaction.  And the perception of quality.

The bottom line is measured in $.  Customer satisfaction is paramount,
of course, but you can't do it for long on a non-positive cash-flow.
-----
Keith Rogers (rogers@ficc.ferranti.com)
(standard disclaimer)
