Taming the Embedded Tiger – Agile Test …

1 Taming the Embedded Tiger Agile Test Techniques for EmbeddedSoftwareNancy Van Schooenderwoert, Ron MorsicatoAgile Rules, Lexington, unit testing is the foundation of agilesoftware development but Embedded systemspresent special problems. Test of embeddedsoftware is bound up with test of hardware,crossing professional and organizationalboundaries. Even with evolving hardware in thepicture, Agile methods work well provided you usemultiple test strategies. This has powerfulimplications for improving the quality of high-reliability systems, which commonly haveembedded software at their Agile methodologies, ExtremeProgramming, Embedded Development, EmbeddedAgile, High Reliability software , Real-timeSoftware, Agile IntroductionThis paper focuses on the testing techniquesthe authors used on a recent Embedded real-timeproject.

2 Our team and management went througha transition to Extreme Programming andexperienced many of the same insights anddifficulties described by other teams, but thispaper will concentrate primarily on the testtechniques we used. The reasons for that choiceare Extreme Programming is almostunheard-of in Embedded work, possiblybecause there is little guidance on how todo Agile testing there We assembled a set of powerful testtechniques and a good description ofthem is enough material for one paper Our goal with this paper is to help otherreal-time/ Embedded teams get on boardwith Agile techniques strong testing iswhere it starts!

3 We offer this technical paper with one caveat:We, and many other XP practitioners, have cometo understand that Agile software development isfirst a people-centered activity and onlysecondarily a technical discipline. XP cannot beimposed on developers who don t buy into itsphilosophy. It is a perfect example of what Deming [1] called an intrinsicmotivator in that it depends on people s pride intheir work, wanting to be part of a team, andwillingness to pitch in. The opposite is what hecalled extrinsic motivators like arbitrarydeadlines, threats, and bureaucratic rules. Thosethings can never generate real teamwork theydestroy it.

4 You need real teamwork to get themost from the test techniques described in Our SituationThe authors worked together on a recent real-time Embedded project, Nancy as team technicalleader and Ron as a senior member of the project s goal was to produce a mobilespectrometer, using a newly developedproprietary technology. It was a new product,being designed from scratch. The system wouldreceive a spectrum of data, apply a complexmathematical algorithm, and send results out overa communication link to a separately supplieddisplay design called for a 32-bit microprocessorwith floating point. We used C with someassembler, an RTOS, and had to support twodifferent communications protocols.

5 The final sizeof the Embedded code was just under 30,000 lines,excluding project is typical of many real-timeembedded projects a data acquisition stage,application of an algorithm, followed by output ofa test techniques themselves are not new,but we used them to an extent we have not seenbefore, and in a combination we have not following sections on each technique willpresent the problem it addresses and then discusshow well that technique worked for us. Theconcluding section will present our reflections onthe use of these practices in Evolution of Hardware PlatformIn Embedded development, the hardware isalways changing.

6 It evolves in steps that thesoftware must support. This dovetails nicely withiterative software development. Our project wasusing a new microprocessor, which could only beobtained as part of an evaluation board from thechip vendor. This was because the chip was notyet fully working the CPU worked but not all ofthe on-board 1 gives an idea of the hardwareevolution we experienced, which is not at allunusual in Embedded work. Early on, anevaluation board (often several of them fromdifferent vendors) is obtained so engineers canselect a CPU and check out its performance. Thefigure shows configuration 1 as having a rev-Amicroprocessor.

7 Later configurations add aprototype board. In our case this board was anearly version of the sensor hardware for bringingin the spectrum that would be measured. The finalconfiguration would dispense with the evaluationboard and combine everything on one board foreconomical production costs. Ideally one wouldgo straight from Configuration 4 to Configuration7 if possible. It s not uncommon that the boardintended to be for final production becomesinstead one more pre-production board, asproblems or new requirements are 1 Config 2 BEvalConfig 3 CEvalProto-AConfig 4 DEvalProto-BConfig 5 GPre-Production-AConfig 6 HPre-Production-BConfig 7 JProduction-AFigure 1.

8 Evolution of the hardwareAny Embedded software development strategymust deal with changing hardware. A commonapproach (and the one we used) is to let #definestatements control parts of the code that arespecific to a given hardware configuration. Thisstrategy was used in parallel with the remainingtest practices described Embedded Testing Trouble Log That s Always OnAn ancient defense against bugs is to code amechanism for displaying brief messages toindicate that some portion of the code wasreached, or a variable has an invalid value, methods are awkward, and when enabledthey can make the code execute differently,frustrating attempts to troubleshoot a logging is normally disabled because itconsumes too many system resources.

9 Havingseen things like this on past projects, we wantedto improve on in our project, Nancy established adifferent kind of trouble log to avoid thosedisadvantages. It s an extension of the idea putforth by Steve Maguire [2] that makes use of theC library assert macro. The ordinary assertmacro can be used to state an assumption, assert( bufcount > 0);so that execution will halt if the expression inthe parentheses is not true. Such a halt lets youexamine the call stack to see what went s ok in the development lab, but the codeshouldn t halt when it s in customer use! Awrapper function provided the smarts to either usethe plain assert macro or quietly write a diagnosticmessage to a buffer, depending on a flag thatindicated development or field overhead of a printf call is high for anembedded system, and there is typically nodisplay device anyway.

10 So we used a circularbuffer in RAM to hold brief fixed-length textmessages. Each message reported the source file,line number, severity, and diagnostic text. Here isan example of a trouble log message beingwritten: Log( I, __FILE__, __LINE__, "BeginningAlgo run.\n" );and how the resulting log file looks (below)Rec # Sev Filename Line # Description Timer Ticks 4 I meas_prd 654 Beginning Algo run. 17384 5 W stat_tbl 2536 2147565568:BIT check failed 17383 6 I can_comm 2421 17112 7 I meas_prd 654 Beginning Algo run.