You are on page 1of 8

CSE 4313 Software Engineering Testing Report 1 Specification Due: Wednesday, January 27 Where: In class Format: Individual If the

class has begun your report is late 1 Main Points
Be sure to read and follow all the guidelines from the links on reports and academic honesty from the WWW home page for the course. The specification is the union of this document plus the program text you are given.

1.1 Learning objectives
The purpose of the assignment is the following. 1. To gain experience reading and evaluating bug reports written by other people. This will give you practice thinking about what a professional bug report should contain, as well as gaining experience in analyzing bugs. 2. Designing and implementing functional test cases based on boundary analysis, equivalence classes and decision tables. 3. Using Junit to create tests for Java programs.

1.2 What to submit
Hand in, in class, packages 1 and 2. If a package is too big to staple as one unit, then please staple in multiple parts. Cover page as part 1 of N, the remaining as part x of N. Include your name(s) on each part. Package 1 • Cover page (bug report version) – attached to announcement. Fill in your name, Prism username and your Issue Tracker user name, as well as the bug report numbers in the order they appear in your report. Your bug report document

Package 2 • • Cover page (Standard version) – attached to announcement. Fill in your name and Prism username Your functional testing report.

Package 3 Submit electronically before the deadline your Sreadhex project, where Sreadhex is the directory containing the project. submit 4313 r1 Sreadhex

2 To get started
Section 2.1 describes how to get started with the open office bug reports. Section 2.2 describes how to get started with the functional testing system.

It implements the pointer to an integer needed for some of the parameters of sreadhex. 7. Your user name must begin with your family name and you must use your CSE email account. 1.1. • • • ooffice.3. Fill in the project name – I used Sreadhex 3.html 2.2. There are three different versions of Open Office available on Prism.2 Functional testing system Download the file 4313_ report1.2 Bugs and issues explained Read about the issues and why bug tracker is used. 4. Select File -> New -> Java Project 2. you will get a directory called Sreadhex that contains the directory sreadhex that corresponds to the sreadhex Java package in which the system is implemented. http://qa.org/servlets/Join to join the project. Click the button Create separate folders for source and class files 4. http://qa. It is needed for the translation of the C version to Java. You can run each one with the following commands. You will modify this file to implement a subset of your equivalence class test cases. DecisionTest.1 Join the QA project of Open Office Go the web page http://www. EquivalenceTest.openoffice.java – Do not modify this class.java. Setable_Integer.1.org.3. You can also install Open Office on your computer by downloading it from http://www. sreadhex contains the following files.jar – It contains the class that contains the variations of the Java version of the sreadhex program that you are testing. 6.1.java – This file has some sample test cases as an illustration of how Junit testing may be done. 3. 2.2.1 Run Open Office Run Open Office by typing ooffice on Prism to get familiar with it.4.java – Initially it is the same a BoundaryTest. You should not have to change this file. sreadhex.c – The C language version of the program.java– the starting point for the system. In Eclipse do the following.1 ooffice. 2.html 2. which is the default download for both Mac OS and Windows. sreadhex.3 Issue writing guidelines The following web page explains how issue reports should be reported. When you untar with the command tar xzf 4313_report1.0 2.org/issue_handling/project_issues. Click the button Create project from existing source .1.openoffice. 1.1.org/issue_handling/bug_writing_guidelines. AllTests. BoundaryTest. You will modify this file to implement a subset of your boundary test cases.tar. as it invokes a test class for each of the three sets of test cases. You will modify this file to implement a subset of your decision table test cases.tar.gz to a local directory.java – Initially it is the same a BoundaryTest.gz. For this report you are to use version 3.openoffice.openoffice. 5.1.1 ooffice – also ooffice.3.java.

The Issue Type for these reports should be DEFECT. Click browse and select the folder Sreadhex that you downloaded (this is the highest level folder 6. Figure 1: Sreadhex project expanded in the Package Explorer Frame Sreadhex should be compiled and executable.1 Ooffice bug reports Find 2 bug reports in Issue Tracker about problems with Open Office that have not been resolved. expand the lowest directory until you can select sreadhex. Perform further analysis using various types of follow-up testing. select Run –> Run as –> Junit Test.openoffice. Click Add Jars.1. IMPORTANT: the bug report should require a few steps to replicate. Ooffice issues appear towards the end of the Version combo box. select Add Library 8. NEW. 3. then only that test file will be run. screen format or spelling error. the Status UNCONFIRMED. click OK 10. Click the Libraries tab. or describe a simpler set of replication steps (if you found one). Add comments as appropriate – these would be added to the comments box in Issue Tracker. Version 3. and the Resolution should be -None-. for example. when you select Run –> Run as –> Junit Test.cgi For each bug report: • • • • Review the report for clarity and tone. Now you are back to the Libraries tab. Select Junit. Click Next at the bottom of the page 7.1. Attempt to replicate the bug. select Next. You can search for reports from the following web page. it should not be trivial.5. All platforms.jar. select Finish 9. Select the project in the package tab. You can also be in a listing of one of the test files. click Finish At this point you should have the project listed under the package tab in the left frame. These comments could mention that you successfully replicated the bug on XXX configuration in YYY build. Your comments . Figure 1 shows what you should see when you expand Sreadhex by clicking on the triangle to the left of the name. and REOPENED (all four are selected at once). http://qa. All OS. STARTED.org/issues/query. select Junit 4.

Making a good argument regarding the importance of the issue (pro or con). like “insert a file like blah. what you will see? • Does the bug report include unnecessary information. Do not include the online comments in your written report. but if it does. Demonstrating that you credibly tested for reproducibility of a bug that is not reproducible. • • . “Insert a file and you will see '”? Is there an example. Your report includes an evaluation of the issue reports you worked with. Following is a list of things that can be addressed. For your report. when useful.. The following items could be addressed in your report. effective or ineffective. which tell you exactly what to do and.g. Is the summary short (about 50-70 characters) and descriptive? Can you understand the bug report? As you read the description. The marker will find them through Issue Tracker. You are not changing his work. have helped? • Did you have to guess about what to do next? • Did you have to change your configuration or environment in any way that wasn't specified in the bug report? • Did some steps appear unnecessary? Were they unnecessary? • Did the description accurately describe the failure? • Did the summary accurate describe the failure? • What were the follow-up tests that you run on this bug report? • Since you will probably NOT have time or the resources to run many follow-up tests.g. • • • • • Reporting the configuration and build you used. You are not expected to go through every item in the list that could be addressed (not all these items apply to all bugs). you are adding comments to it at the end of the report. e. what was done poorly.fee”? • Can you replicate the bug? • Did you need additional information or steps? • Did you get lost or wonder whether you had done a step correctly? Would additional feedback.. The content of your comments can vary depending on the problem. e. The key thing is that they are useful to the reader. do you understand what the reporter did? Can you envision what the program did in response? Do you understand what the failure was? • Is it obvious where to start (what state to bring the program to) to replicate the bug? • Is it obvious what files to use (if any)? Is it obvious what you would type? • Is the replication sequence provided as a numbered set of steps. For each issue report. you might address different items.'”. If you edit the report in the database. accurate. comment on what was done well. and useful. personal opinions or anecdotes that seem out of place? • Is the bug report too long? Too short? Does it have a lot of unnecessary steps? • Is it overly general. it should be credible. Instead.should NEVER appear critical or disrespectful of the original report or of the person who wrote it. what other tests would you like to run? What would you hope to learn from these tests? • Does the description include non-factual information (such as the tester's guesses about the underlying fault) and if so. Reporting a simpler set of replication steps. never change what the reporter has actually written. “the program will respond like this. the marker will be looking for details that suggest you had insight into what makes issue reports good or bad. you should address a few items of interest.foo or blah2. does this information seem credible and useful or not? • Does the description include statements about why this bug would be important to the customer or to someone else? The bug report need not include such information. For each bug report. and what was missing that should have been there. Reporting the results of a good follow-up test.

1001 and 1111. Your report should discuss how you developed the test cases and give a rationale for tests you select and reject. For example. odd_digit. . equivalence class and decision table methods.3. In the above case. “34ABF”. Thus. • • sreadhex(str.1 Motivation sreadhex is a function that takes a string of ASCII characters where each character represents a hexadecimal digit. while a string of digits can be packed into a byte-array of half the length of the string. Do not include any comparison of methods. if 15 is returned by one call. what value should the ???? below have? Byte 1: 00110100 Byte 2: 10101011 Byte 3: 1111???? Answer: The solution is not to put the last digit in the result array. nread. rlen. the result string would be the first two bytes and the integer 15. 9. what would be done with the unused half of the last byte in the result string? Given.2. pp. That is. all of the following strings will yield the same result. 1010. i. It is complex.e. comes from the book The Craft of Software Testing.1 The tasks You are to develop test cases using each of boundary value. the string “A49F” represents the hexadecimal digits 10. A49F  byte 1: 10100100 and byte2: 10011111 Question: What should be done if there are an odd number of characters? Since two digits fit per byte. 4. but to return it as a separate return value. Prentice Hall PTR. 4 The SREADHEX problem sreadhex is a function from the GhostScript program. a pair of digits can be packed into a byte. I suggest you log your impressions as you proceed with this task.3 Specification Here is the specification for sreadhex. The problem. To make the specification easier to read. rlen is the maximum number of bytes to be filled in str. where it is used as the first 4 bits of the next result byte-array. Each digit can be represented in four bits. It’s the destination.2 Functional testing 3. Question: What should be done if there are characters in the source string that do not represent hexadecimal digits? Answer: sreadhex completely ignores them. s) str is a byte array to fill. Brian Marick. then the next call has 15 as input. “34AB” “3 4 A B” “3G4GAMB-“ 4. 1995. names of variables and definitions are in bold. Other styles of specification read more naturally but are harder to use for devising test cases. which processes the Postscript page-descriptionlanguage – the basis of pdf files. I recommend that you work up from the least sophisticated method to the most sophisticated method. 0100. thus the string corresponds to the bit sequences. Thus the result string always has an even number of digits. Limit your report to discussion each method independently. as described in this specification. 24-29. As a consequence. as a consequence. as that will be the first task for Report 2. you develop three sets of test cases based on the three methods. and 15. 4. The “extra” return value is typically used in the next call to sreadhex.

Notice that both uppercase and lowercase characters are used. Postconditions 1. is placed in digit 0. 2. For any hexadecimal character in s at index index. 0 1 2 3 4 5 6 7 8 9 A B C D E F a b c d e f 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 10 11 12 13 14 15 hex_char[index] – This is the indexth hexadecimal character in s. They are listed here. Numbering begins with 0. the return value is 1. then num_hex_chars is 3. the return value is 1. On failure: *nread is 0. Definitions There are 22 hexadecimal characters. Validated: rlen is not 0. then hex_char[0] is ‘A’ hex_char[1] is ‘a’ hex_char[2] is ‘0’ Notice that the dash is not counted. Digits 0 and 1 are in byte 0 of str. When the routine finishes. 7. If s is “Aa-0”. along with their corresponding integer values. It is the source of hexadecimal characters. it leaves the number of bytes it used in that integer. use them all) if start + num_hex_chars < rlen*2 and start + num_hex_chars is odd then A. digits 2 and 3 are in byte 1. Assumed: s is a non—null pointer to a null-terminated string. 3. *nread is (start + num_hex_chars)/2. D. 5. Assumed: str is a non-null pointer to an array that can hold rlen bytes (hence 2*rlen digits). s is a null-terminated string. . 9 is placed as the 0 digit of str. Assumed: rlen ≥ 0. digit[start + index] = hex_char[index] (For example. The return value is 0. 1 means sreadhex ran out of characters before filling str. 4. the routine leaves either –1 or the extra digit in it after return. Assumed: nread is a non-null pointer to an integer. *odd_digit is –1. B. C. For example. The caller uses it to pass in the first digit (if the last call used an odd number of digits). (An odd number of digits that don’t fill str. 6. Preconditions 1.• • • • nread is a pointer to an integer. (The “odd” in odd_digit refers to its being set when there are an odd number of hexadecimal characters to be placed in str. (An even number of digits that don’t fill str: use them all) if start + num_hex_chars < rlen*2 and start + num_hex_chars is even then A. Remember that *odd_digit. if (*odd_digit == -1) then start is 0 else start is 1 digit[index] – This is the indexth digit (4-bit chunk) of str. 15]. odd_digit is a pointer to an integer. num_hex_chars is the number of hexadecimal characters in s. sreadhex returns 0 or 1. Assumed: *odd_digit is in the range [–1. if s is “Aa-0”. if character 0 is ‘9’ and start th is 0. Assumed: odd_digit is a non-null pointer to an integer. and so on. 2. start – This is the location where the value of the first hexadecimal character in s is to be placed in str. if not –1.

3. /* Read a hex string from a stream. */ /* *odd_digit should be -1 initially. */ /* if an odd number of hex digits was read. memset(decoder-1. otherwise *odd_digit is set to -1. C. This version. v4 contains both bugs in v2 and v3. uint *nread. byte val2. register byte *s. { byte *ptr=str. byte val1 = (byte)*odd_digit. int i. its name is v1 (see BoundaryTest. digit[start + index] = hex_char[index]. D. */ static byte decode_hex[257]={0}. *odd_digit is unchanged. i<16+6. typedef unsigned int uint. if *odd_digit is initially ≥ 0 then digit[0] = the initial value of *odd_digit. *nread is (start + num_hex_chars – 1)/2. byte *limit = ptr+rlen. for (i=0. */ /* 0 if we filled the string first. or <0 on error. thereby containing that bug and no other). For any hexadecimal character in s at index index such that start + index < rlen*2 digit[start + index] = hex_char[index] D. The following is the program in C. odd_digit. register char *decoder = (char*)(decode_hex+1). 257). *odd_digit is set to the value of the last hexadecimal character in s. v3 contains a different seeded bug. rlen. nread. v2 contains one seeded bug. hex_none. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 typedef unsigned char byte. *nread is rlen. if (decoder[-1]==0) /* not initialized yet */ { static char hex_chars[]= "0123456789ABCDEFabcdef". while ((val1=decoder[sgetc(s)])>0xf) d1: . C. ignore any excess) if start + num_hex_chars ≥ rlen*2 and start + num_hex_chars is odd then A. } if (val1<=0xf) goto d2. uint rlen. 4. /* not initialized yet */ #define hex_none 0x10 #define hex_eofc 0x20 #define sgetc(s) (*((s)++)) int sreadhex(str. s) byte *str.java) has been translated into Java (I hope correctly. the return value is 0. For any hexadecimal character in s at index index except the final hexadecimal character. i++) decoder[hex_chars[i]] = (i>=16? i-6: i).B. (Enough digits to fill str. This version contains a fault that has long since been fixed in later versions. */ /* Answer 1 if we reached end-of-file before filling string. int *odd_digit. decoder[0] = hex_eofc. B. *odd_digit is set */ /* the odd digit value.

The loops skip non-hexadecimal characters. In line 26 decoder is set to point to the last 256 characters of the decode_hex array. *nread = ptr .38 39 40 41 42 43 44 45 46 47 48 49 d2: ended: } { if (val1==hex_eofc) {*odd_digit=-1. The case on line 24 converts the –1 to a large positive number. if (ptr<limit) goto d1. 26 – The program uses a lookup table. decoder[character] is the corresponding value. The character is used as an index in the table. The first character (decoder_hex[0] or decoder[-1] indicates whether the table has already been initialized. In the original program. it should be in the range [0 . to convert characters into values. so that val1 ≤ 0xf evaluates to false. return 1. sgetc got characters from a stream datatype. the second one finds the second. • • • • • Part 1: Overall presentation – 10% Part 2: Bug reports – 36% – 18% each Part 3: Boundary value testing – 18% – half for the test cases and half for the report Part 4: Equivalence class testing – 18% – half for the test cases and half for the report Part 5: Decision table testing – 18% – half for the test cases and half for the report .str. Lines 10. Lines 28-35 – decoder is initialized with memset.. Note that the characters “abcdef” are given the same values as “ABCDEF”. Program notes Line 13 – The program gets characters via sgetc. *nread = rlen. 15]. If the first digit is passed in. which causes sreadhex to return. 5 Grading scheme The grade for the report is partitioned into the following parts. goto ended.} } while ((val2=decoder[sgetc(s)])>0xf) { if (val2==hex_eofc) {*odd_digit=val1. return 0. Then the entries for hexadecimal characters are set by iterating through a string of all such characters. in this simpler version it picks characters from a string. Character ‘\0’ (which is found at the end of a C string) has value hex_eofc (value 0x20). decoder. and placing that in decoder. Lines 36-42 – The first while loop finds the first digit in a byte. The table is initialized the first time sreadhex is called. calculating the value.} } *ptr++ = (val1<<4) +val2. which sets every entry to hex_none indicating that the corresponding character is not hexadecimal. –1 means no digit was passed in. goto ended.