DEBUGGING TECHNIQUES

Introduction Rule of Thumb: Write good, bug-free code from start if you could Testing/Debugging embedded software is more difficult than application software Post-shipment application problems are more tolerable than embedded (realtime or life-critical) software

4/24/2012

EMBEDDED SYSTEM DESIGN - II

DEBUGGING TECHNIQUES Testing on Host Machine

Some reasons why you cant test (much, if any) on target machine:
Test early (target may not ready or completely stable) Exercise all code, including exceptions (real situations may be difficult to exercise) Develop reusable, repeatable test (difficult to do in target environment, and likelihood of hitting the same bug is low) Store test results (target may not even have disk drive to store results)

4/24/2012

EMBEDDED SYSTEM DESIGN - II

DEBUGGING TECHNIQUES Testing on Host Machine 1

Basic Techniques Fig 10.1

Target system on the left: (hardware-indep code, hardware-dep code, hw) Test system (on host) on the right: (hardware-indep code same, scaffold rest) Scaffold provides (in software) all functionalities and calls to hardware as in the hardware-dep and hardware components of the target system more like a simulator for them!

4/24/2012

EMBEDDED SYSTEM DESIGN - II

4/24/2012

EMBEDDED SYSTEM DESIGN - II

DEBUGGING TECHNIQUES Testing on Host Machine 2

Basic Techniques

Fig 10.2 Radio.c -- hardware independent code Radiohw.c hardware dependent code (only interface to hw: inp() and outp() supporting vTurnOnTransmitter() and vTurnOffTransmitter() functions Inp() and outp() must have real hardware code to read/write byte data correctly makes testing harder!! Fig 10.3 Replace radiohw.c with scaffold, eliminating the need for inp() and outp() both are simulated in software a program stub!!

4/24/2012

EMBEDDED SYSTEM DESIGN - II

Figure 10.2 A Poor Plan for Testing /* File: radio.c */ void vRadioTask (void) { . . . !! Complicated code to determine if turning on the radio now !! is within FCC regulations. . . . !! More complicated code to decide if turning on the radio now !! makes sense in our protocol. If (!! Time to send something on the radio) { vTurnOnTransmitter (FREQ_NORMAL); !! Send data out vTurnOffRadio (); } } -----------------------------------------------

4/24/2012

(continued) EMBEDDED SYSTEM DESIGN - II

Figure 10.2 (continued) /* File: radiohw.c */ void vTurnOnTransmitter (int iFrequencyValue) { BYTE byPower; /* Byte read from device controlling power. */ int i; /* Turn on main power for the radio. */ disable_interrupts (); byPower = inp (PWR_CONTROL_ADDR); byPower |= PWR_CONTROL_RADIO_MAIN; outp (PWR_CONTROL_ADDR, byPower); enable_interrupts (); /* Shift the frequency value out to hardware. */ for (i = 0; i < 16; ++i) { /* Send out the lowest bit of iFrequencyValue */ if (iFrequencyValue & 0x0001) { /* The data is a binary 1 */ /* Put a '1' on the data line; pulse the clock line. */ outp (FRQ_CONROL, DATA_1 & CLOCK_LOW) outp (FRQ_CONROL, DATA_1 & CLOCK_HIGH); } EMBEDDED SYSTEM DESIGN - II (continued)

4/24/2012

Figure 10.2 (continued) else { /* The data is a binary 0 */ /* put a '0' on the data line; pulse the clock line. */ outp (FRQ_CONROL, DATA_0 & CLOCK_LOW) outp (FRQ_CONROL, DATA_0 & CLOCK_HIGH); } /* Shift iFrequencyValue to get the next lowest bit. */ iFrequencyValue >>= 1; } /* Turn on the receiver. */ byPower = inp (PWR_CONTROL_ADDR); byPower |= PWR_CONTROL_RADIO_RECEIVER; outp (PWR_CONTROL_ADDR, byPower); enable_interrupts (); } void vTurnOffRadio (void) { BYTE byPower; /* Byte read from device controlling power. */ /* Turn off main power for the radio. */ disable_interrupts (); byPower = inp (PWR_CONTROL_ADDR); byPower &= ~PWR_CONTROL_RADIO_MAIN; outp (PWR_CONTROL_ADDR, byPower); enable_interrupts ();

4/24/2012

-------------------------------------------

EMBEDDED SYSTEM DESIGN - II

(continued)

Figure 10.2 (continued) /* File: test.c */ void outp (int Address, BYTE byData) { #ifdef LET_USER_SIMULATE_HARDWARE PRINTF ("program wrote %02x to %04x.", byData, iAddress); #endif #ifdef SIMULATE_HARDWARE !! Remember that software wrote byData to iAddress !! Update state of simulated hardware. #endif } BYTE inp (int iAddress) { int iData; #ifdef LET_USER_SIMULATE_HARDWARE PRINTF ("program wrote %02x to %04x. Enter value.", iAddress); scanf ("%x", &iData); #endif #ifdef SIMULATE_HARDWARE !! Figure out what the real hardware would return #endif return ((BYTE) iData); } 4/24/2012

EMBEDDED SYSTEM DESIGN - II

Figure 10.3 Better Plan for Testing /* File: radio.c */ void vRadioTask (void) { . . . !! Complicated code to determine if turning on the radio now !! is within FCC regulations. . . . !! More complicated code to decide if turning on the radio now !! makes sense in our protocol. If (!! Time to send something on the radio) { vTurnOnTransmitter (FREQ_NORMAL); !! Send data out vTurnOffRadio (); } } ----------------------------------------------(continued)

4/24/2012

EMBEDDED SYSTEM DESIGN - II

10

Figure 10.3 (continued) /* File: test.c */ static BOOL fRadionOn; static int iRadioFrequencyValue; void vTurnOnTransmitter (int iFrequencyValue) { /* Record the state of the radio. */ fRadionOn = TRUE; iRadioFrequencyValue = iFrequencyValue; /* Tell the user */ printf ("Radio turned on with frequency %04x", iFrequencyValue); } void vTurnOffRadio (void) { /* Record the state of the radio. */ fRadioOn = FALSE; /* Tell the user */ printf ("Radio now off");

4/24/2012

EMBEDDED SYSTEM DESIGN - II

11

10.0 DEBUGGING TECHNIQUES Testing on Host Machine 3 Calling Interrupt Routines

Embedded systems are interrupt-driven, so to test based on interrupts

1) Divide interrupt routines into two components

A) a component that deals with the hardware B) a component of the routine which deals with the rest of the system

2) To test, structure the routine such that the hardware-dependent component (A) calls the hardware-independent part (B). 3) Write component B in C-language, so that the test scaffold can call it

E.g., Fig 10.4

Hw component (A) is vHandleRxHardware(), which reads characters from the hw Sw component (B) is vHandleByte, called by A to buffer characters, among others The test scaffold, vTestMain(), then calls vHandleByte(), to test if the system works [where vTestMain() pretends to be the hardware sending the chars to vHandleByte()] EMBEDDED SYSTEM DESIGN - II 12

4/24/2012

Figure 10.4 Dividing Interrupt Routines into Two Parts /* File: serial.c */ #define CR 0x0d #define SIZEOF_CMD_BUFFER 200 BYTE a_byCommandBuffer[SIZEOF_CMD_BUFFER]; /* Queue to send message to command-handling task. */ extern unsigned long qidCommands; void interrupt vHandleRxHardware (void) { BYTE byChar; /* The character we received */ int iHwError; /* Hardware error, if any */ iHwError = !! Get status from hardware; if (iHwError == CHARACTER_RXD_OK) { /* We received a character; deal with it. */ byChar = !! Read byte from hardware; vHandleRxByte (byChar); } else !! Deal with hardware error !! Reset the hardware as necessary. !! Reset interrupt controller as necessary. } 4/24/2012 (continued) EMBEDDED SYSTEM DESIGN - II

13

Figure 10.4 (continued) void vHandleRxByte (BYTE byReceived) { static BYTE *p_byCommandBufferTail = a_ byCommandBuffer; extern BYTE *p_byCommandBufferHead; unsigned long a_ulMessage[4]; /* Message buffer. */ /* Advance the tail pointer and wrap if necessary */ ++ p_byCommandBufferTail; if (p_byCommandBufferTail == &a_ byCommandBuffer [SIZEOF_CMD_BUFFER]) p_byCommandBufferTail = a_ byCommandBuffer; /* If the buffer is not full . . . . */ if (p_byCommandBufferTail != p_byCommandBufferHead) { /* Store the character in the buffer. */ *p_byCommandBufferTail = byReceived; /* If we got a carriage return, wake up the command-handling task. */ if (*p_byCommandBufferTail == CR) { /* Build the message . . . */ a_ulMessage[0] = MSG_COMMAND_ARRIVED; a_ulMessage[1] = 0L; a_ulMessage[2] = 0L; a_ulMessage[3] = 0L; EMBEDDED SYSTEM DESIGN - II (continued)

4/24/2012

14

Figure 10.4 (continued) /* . . . and send it. */ q_send (qidCommands, a_ulMessage); } } else { /* Discard the character; move the pointer back. */ if (p_byCommandBufferTail == a_ byCommandBuffer) p_byCommandBufferTail == &a_ byCommandBuffer[SIZEOF_CMD_BUFFER]; -- p_byCommandBufferTail; } } -------------------------------------------/* File: test.c */ void vTestMain (void) {
BYTE a_byTestCommand[] = "THUMBS UP\x0dSIMON SAYS THUMBS UP\x0d";

4/24/2012

BYTE *p_by; . . /* Send each of the characters in a_byTestCommand */ p_by = a_byTestCommand; while (*p_by) { /* Send a single character as though received by the interrupt */ vHandleRxByte (*p_by); /* Go to the next character */ ++p_by; } . . }

EMBEDDED SYSTEM DESIGN - II

15

DEBUGGING TECHNIQUES Testing on Host Machine 4

Calling the Timer Interrupt Routine

Design the test scaffold routine to directly call the timer interrupt routine, rather than other part of the host environment, to avoid interruptions in the scaffolds timing of events This way, the scaffold has control over sequences of events in the test which must occur within intervals of timer interrupts

Script Files and Output Files

To let the scaffold test the system in some sequence or repeated times, write a script file (of commands and parameters) to control the test Parse the script file, test system based on commands/parameters, and direct output intermixture of the input-script and output lines into an output file The commands in the script cause the scaffold to call routines in the B (sw-indp) component -- See Fig 10.5 and Fig 10.6 for the cordless bar-code scanner

4/24/2012

EMBEDDED SYSTEM DESIGN - II

16

4/24/2012

EMBEDDED SYSTEM DESIGN - II

17

4/24/2012

EMBEDDED SYSTEM DESIGN - II

18

DEBUGGING TECHNIQUES Testing on Host Machine 5

More Advanced Techniques

Making the scaffold automatically control sequence of events e.g., calling the printer interrupt many times but in a controlled order to avoid swamping Making the scaffold automatically queue up requests-to-send output lines, by automatically controlling the button interrupt routine, which will cause successive pressing of a button to let the next output line be received from the hardware (the printer interrupt routine). In this way, the hardware-independent software is controlled by the scaffold, where the button interrupts serve as a switch The scaffold may contain multiple instances of the software-independent code, and the scaffold serves as a controller of the communication between the instances where each instance is called by the scaffold when the hardware interrupt occurs (e.g., the scanner or the cash register). In this way, the scaffold simulates the hardware (scanner or register) and provides communication services to the softwareindependent code instances it calls. See Fig 10.7

4/24/2012

EMBEDDED SYSTEM DESIGN - II

19

4/24/2012

EMBEDDED SYSTEM DESIGN - II

20

DEBUGGING TECHNIQUES Testing on Host Machine 6

Objections, Limitations, and Shortcomings 1) Hard to test parts which are truly hardware dependent, until the target system is operational. Yet, good to test most sw-independent parts on host (see Fig 10.8) 2) Time and effort in writing scaffold even if huge, it is worthwhile 3) Having the scaffold run on the host and its RTOS scaffold can run as low priority task within the RTOS and have nicely integrated testing environment 4) The hard to justify limitations cant tell in scaffold until the actual test

Writing to the wrong hardware address software/hardware interactions Realistic interrupt latency due to differences in processor speeds (host v. target) Real interrupts that cause shared-data problems, where real enable/disable is the key Differences in network addressing, size of data types, data packing schemes portability issues

4/24/2012

EMBEDDED SYSTEM DESIGN - II

21

4/24/2012

EMBEDDED SYSTEM DESIGN - II

22

DEBUGGING TECHNIQUES Instruction Set Simulators

Using software to simulate:

The target microprocessor instruction set The target memory (types - RAM) The target microprocessor architecture (interconnections and components) Simulator must understand the linker/locator Map format, parse and interpret it Simulator takes the Map as input, reads the instructions from simulated ROM, reads/writes from/to simulated registers Provide a user interface to simulator for I/O, debugging (using, e.g., a macro language)

4/24/2012

EMBEDDED SYSTEM DESIGN - II

23

DEBUGGING TECHNIQUES Instruction Set Simulators 1

Capabilities of Simulators:

Collect statistics on # instructions executed, bus cycles for estimating actual times Easier to test assembly code (for startup software and interrupt routines) in simulator Easier to test for portability since simulator takes same Map as the target Other parts, e.g., timers and built-in peripherals, can be tested in the corresponding simulated versions in the simulated microprocessor architecture

What simulators cant help:

Simulating and testing ASICs, sensors, actuators, specialized radios (perhaps, in future systems!!) Lacking I/O interfaces in simulator to support testing techniques discussed (unless additional provision is made for I/O to support the scaffold; and scripts to format and reformat files between the simulator, simulated memory, and the scaffold)

4/24/2012

EMBEDDED SYSTEM DESIGN - II

24

DEBUGGING TECHNIQUES The assert Macro

The assert is used (with a boolean-expression parameter) to check assumptions If the expression is TRUE nothing happens, if FALSE, a message is printed and the program crashes Assert works well in finding bugs early, when testing in the host environment On failure, assert causes a return to the host operating systems (cant do on target, and cant print such message on target may not have the display unit) Assert macro that runs on the target are useful for spotting problems:

1) disabling interrupts and spin in infinite loop effectively stopping the system 2) turn on some pattern of LEDs or blinking device 3) write special code memory for logic analyzer to read 4) write location of the instruction that cause problem to specific memory for logic analyzer to read (the Map can help isolate which source code is the culprit!) 5) execute an illegal op or other to stop the system e.g., using in-circuit emulators

4/24/2012

EMBEDDED SYSTEM DESIGN - II

25

4/24/2012

EMBEDDED SYSTEM DESIGN - II

26

DEBUGGING TECHNIQUES Using Laboratory Tools Hardware-focused

Lab tools help reveal hard-to-find, very infrequently occurring bugs Types useful to software engineers:

Voltmeters (measure voltage diff); Ohmmeters (measure resistance/connectedness) Oscilloscopes (scopes) test events that repeat periodically monitoring one or two signals (graph of time v. voltage), triggering mechanism to indicate start of monitoring, adjust vertical to know ground-signal, used as voltmeter (flat graph at some vertical relative to ground signal), test if a device/part is working is graph flat? Is the digital signal coming through expecting a quick rising/falling edge (from 0 VCC or VCC 0) if not, scope will show slow rising/falling indicating loading, bus fight, or other hardware problem (See Fig 10.10, Fig 10.11, Fig 10.12, Fig 10.13, Fig 10.14)

4/24/2012

EMBEDDED SYSTEM DESIGN - II

27

4/24/2012

EMBEDDED SYSTEM DESIGN - II

28

4/24/2012

EMBEDDED SYSTEM DESIGN - II

29

4/24/2012

EMBEDDED SYSTEM DESIGN - II

30

4/24/2012

EMBEDDED SYSTEM DESIGN - II

31

4/24/2012

EMBEDDED SYSTEM DESIGN - II

32