Subj : Re: Not debugging? To : comp.programming,comp.lang.java.programmer,comp.lang.lisp From : Greg Menke Date : Thu Aug 25 2005 10:09 am "Phlip" writes: > Greg Menke wrote: > > > Sure... that'll help debug drivers on bizzaro hardware with > > incomplete/inaccurate datasheets & interrupt/context switch routines > > suffering from various race conditions. Sometimes you just have to bite > > the bullet and debug right on the metal because of heisenbugs if nothing > > else. > > > > Care to guarantee that the emulator duplicates the bugs & timing > > characteristics of the real thing? > > My admonition answers the common complaint, "I don't have time to write > tests; I'm too busy debugging". Never heard that one myself. We routinely work out detailed end-to-end test plans as well as unit tests, but as much as possible on the real hardware. Automated where possible certainly. Often, detailed testing on an emulator is a waste of time because the emulator isn't high fidelity enough. > You have time to write tests. Writing an emulator gives you a framework to > hang all your discoveries about the real environmentthings' bugs, and this in turn > frees up your schedule by automating as much low-value labor as possible. Sure thing, thats why we have scripts and analysis programs to suck data out of logic & bus analyzers to profile interrupt performance, packet latencies, etc.. but an emulator only provides the roughest sort of functional test. > And you find those bugs --> by debugging <--. > > You describe a legacy situation - someone else invented this bizarro > hardware, and legacy situations require debugging to learn their > characteristics. No I don't. I describe a realtime system- or even a simple device driver. There are lots of those, new and old. Say you wrote some subsystem on this embedded system, and you implemented some kind of test framework to help you get it as functionally debugged as possible. What you've tested is your inputs, outputs and algorithms work in an fairly abstract and simplistic situation. Thats helpful for first order easy debugging, but you're going to be doing it the hard way along with everybody else once the software is on the hardware and some of the unknown idiosyncracies of the system start showing up. More of the unknown idiosyncracies will appear over time. And at this stage, your emulator is pretty much useless because nobody cares about what it says when the real thing is sitting in the lab & thats were the bugs are. > As you learn them, add tests about them to your emulator, to _approach_ a > state where a code failure causes a red flag in the tests _before_ it causes > an error situation near the hardware. Sounds great. You going to step up and write the emulator for the embedded system, from the fpga glue, busses and cpu- prove that it is of reasonable fidelity- and keep it in sync with the vhdl as it evolves too? Be advised that hardware specs will continue to change (you'll have to emulate all its bugs, or at least its most important ones- maybe they're documented), you'll have to also emulate as much as possible of the expected and unexpected interface characteristics the system will operate in. AND you have deadlines on your part of the delivered software whatever your testing methodology is- nobody is going to wait for you to write an emulator before you start delivering code according to project schedule. The last cpu emulator I worked with had writable registers that were read-only on the variation of the cpu we're actually using... which is not to say the emulator is useless, I use it for rough "does it crash on boot" tests so I don't waste time on the real hardware. If you want to call that a "test case", feel free- but its not really testing much. > The point is not to never debug. The point is to always seek ways to replace > any necessary debugging with test cases, so the remaining debugging is > manual labor of the highest value. I'm not talking about web-apps, I'm talking about realtime systems on dedicated hardware- or maybe just a simple device driver- and useful "test cases" on that sort of thing essentially involve the real hardware in situations as close as possible to what the system will experience in the field. Gregm .