Books

CO531: Software Engineering Practice: Testing
or some of the things you should know about testing but never knew you had to ask! • Amazon returns 400+ hits for “software testing” in the title • Recommended: How to Break Software by James A. Whittaker, Addison Wesley, 2003. Price: £25.19 (from Amazon) • Google returns about 175,000,000 hits to a search on “software testing” • Truth vs Anecdotes?

Testing
• Why do we do it?
– Because we ain’t perfect

Simple Example
• CO522 Example class • p(x) = 4096x6 – 36864x5 + 129280x4 – 222720x3 + 194224x2 – 78096x + 10395 • Did anyone test their calculation method? • How did/do you test this? • Balance?

• When do we do it?
– All the time

• How do we do it?
– Systematically and repeatedly

• When do we stop doing it?
– When the software isn’t used any longer

• … is generally not an option. • How long will this take? General Introduction • More next term • What is the point of testing? – XP is very hot on testing Dijkstra says… • Program testing can be used to show the presence of bugs. • What does testing actually tell you? • What can’t testing tell you? • Confidence building … but don’t forget – that “Oh fcuk” moment . • Object-oriented programming is an exceptionally bad idea which could only have originated in California.Exhaustive testing • Exhaustively test a method that takes two integer parameters. • We must not put mistakes into programs because of sloppyness. we have to do it systematically and with care. but never to show their absence! • You must not give the world what it asks for. but what it needs. Assume – 32-bit integers – runs a test every 50 nanoseconds (10-9 secs) Exhaustive testing ….

black box Iterative testing vs Big bang testing . any of the above (Microsoft) • Debugging – fixing it – safely! • Removal cost increases with time Why/How Do Errors/Bugs Occur • Designed/written/tested/used by fallible/malicious people.Errors --Nomenclature • looking for bugs. • You made a silly error – a simple oversight • You’re being pushed to complete a project • “Unforeseeable” circumstances occur • Difficult/impossible to test some circumstances for real • • • • Types of Testing Just talking about testing code Unit testing vs System testing White/Glass box. grey box. wrong answers. blue screens. faults. defects. crashed aircraft. failures. no passports. showstoppers … • lost marks. errors. features. dead people.

provides steady and measurable progress • Quality: built-in and maintained via testing • Low-level design: context for decisions (classes/methods. counter-productive • Quality: Interleaving tests with coding • Productivity: Not so obvious • Studies: inconclusive • Other factors: team dynamics. self-esteem. Kent Beck Why Test-First • Feedback: instant on code changes • Task orientation: focuses on problem decomposition. …) Advocates and Sceptics • Advocates: increased productivity and better quality • Sceptics: Hard to learn. courage What else is it? • Test first coding is not a testing technique Ward Cunningham (being provocative!) • Analysis technique • Design technique • and. it’s also a testing technique! . names. I believe.The Test First (TDD) Scheme Story Understand Add one failing test Add production code for that test Run ALL tests Rework OK Story Complete Next Story Never write a line of functional code without a broken test. maintains focus. interfaces.

• Typical excuses – – – – It takes too long to write tests It takes too long to run the tests It’s not my job to test my code I don’t know what the code is supposed to do so I can’t test it – I feel guilty putting testers and QA staff out of work – I’m not allowed to run unit tests on a live system . – Don’t have to write drivers – You have to know the answer to each test – You can build the tests up – You can run all the tests easily (and batches if you want to do a bit more work. – Invest up-front to save time later JUnit • Nice to have help with unit tests • BlueJ provides a neat interface to the JUnit testing framework.Unit Testing • Usually tests just one particular method • Need confidence that low-level routines work before adding high level code • Not concerned with formal validation of code – Still need other forms of testing Unit Testing contd • Answers the question – Is the code doing exactly what I think it should? • Note that this isn’t the question – Does the code do what the requirements demand? • You need to ensure your code does what you want ALL THE TIME – Regression tests – all tests must pass NOT JUST THE NEW ONES • Wish to avoid the ripple effect • Good unit testing – Better code design – Less time in the debugger • Unit testing helps with documentation – Tests are visible – Test are up-to-date and correct (because you keep running them!) – Code documentation (on the other hand) tends to drift Unit Testing • Testing isn’t free but ...

but … – … they do pay back time invested • Other tools might be available – Compiler flags to do additional checks – Specific checks e. security. performance.Unit Tests vs Function Tests • Unit tests are local – Include integration tests – Written by (and for) programmers Generating Test Cases • Problem dependent -. memory management • Maintenance phase (more?) . fix code.g. run regression) – Learn (discover why and record) – Tools Debugging • Methodical process of finding and reducing the number of defects in a system.. • Use tools – you’re not a sissy because you use a debugger – You are a plonker if you don’t • Debuggers are often quite sophisticated tools – Steep learning curve. installability On Finding an Error • Same for all reported errors – Communicate – Reproduce (the error not literally!) least code/data = failing test – Debug (why. where) – Repair (add new test.experience • Some general rules – equivalence classes – one example covers many – edge cases – what happens at the limits (< | <=) – odd cases • Function testing = higher order testing – – – – – Does code do what the user expects? XP has continual customer involvement Iterative function testing = minimal surprises Done by end user (as well as programmers) Communication • Illegal cases – comment (assume programmers read them) – trap (defensive programming) • Others: stress.

3. • • • Back out of changes that don’t fix the problem Problem not where you think it is Modified code never called Use a source code control system (Subversion/CVS) . Recognise that there is a defect in your code • • • • • • Check input data (program should already do this?) Reproducibility Identify the symptoms What part of the system is causing the problem Check inputs and outputs over smaller parts of the system Experience counts here • • • • • • Basic Guide (contd.. TEST and TEST • • • • Apply the fix Check it does correct the problem (TEST) Check nothing else breaks (TEST) Add new test (which one???) to regression (and unit ??) tests 1. Fix.) 5. • Useful things to do Have the correct mind-set Your code probably is broken 2. incorrect value generated but problem not detected until it is used much later Are there similar sections of code elsewhere? Sometimes simple – one line Sometimes catastrophic – major rewrite Quick fix may be necessary 2. Why has the problem occurred? Incorrect code may be a long way from the problem’s appearance (code distance/time) E.) 3.Basic Guide to Debugging 1. TEST.g. Determine the fix Basic Guide (contd. 4. Isolate the source of the problem 4. • Add assert/debug statements into code so they can be activated when necessary Use log files Only ever make one substantial source change at a time Helps prevent adding new bugs 5.

. • Doesn’t say I can’t add tests that don’t fail! • Add test if – they show a failure (need to write code) – there is something special about the test Testing GUIs • Is a form of function testing • XP approach is not as successful (Kent Beck says so – so it must be true ☺) • It’s hard to do • It’s hard to automate • Screen shots are worth a thousand words.Do I Only Add Failing Tests • Never write a line of functional code without a broken test. • Document all tests Whittaker’s Book • Read his book for a really useful list of things to try when you perform both unit and function testing. • Try some of these attacks out on your project code. Kent Beck.

Sign up to vote on this title
UsefulNot useful