You are on page 1of 8

2012 IEEE 14th International Symposium on High-Assurance Systems Engineering

Exception Handling Defects: An Empirical Study
Puntitra Sawadpong∗ , Edward B. Allen† and Byron J. Williams‡ Department of Computer Science and Engineering Missisippi State University Mississippi State, Mississippi 39762–9637 Email: ∗ ps402@msstate.edu, † edward.allen@computer.org and ‡ williams@cse.msstate.edu

Abstract—Exception handling mechanisms are a feature common in many programming languages. Improper handling of exceptions can cause failures in software systems. This is especially critical for high-assurance systems where software failures may have severe consequences. Understanding the impact of misusing exception handling is important for better utilization of these constructs. This paper presents an exploratory study to determine whether using exception handling is relatively risky by analyzing the defect densities of exception handling code and the overall source code. Also, statistics representing the prevalence of exception handling code are proposed. The study was conducted with six major Eclipse releases. Essential data was collected using custom scripts to extract exception handling information from the source code and exception handling defects information from bug reports. We found that the density of defects that are closely related to exception handling constructs is relatively high compared to the overall defect density. This implies a relationship between the use of exception handling constructs and the risk of defects. Further studies should be conducted to better determine proper ways to implement exception handling and the root causes of exception defects in the software systems. Keywords-empirical study; exception handling; measurement; defects; Eclipse

I. I NTRODUCTION Exception handling is one of the advanced topics in most programming languages [1], [2]. Exception handling constructs serve to process unexpected events so the program does not crash or experience other unintended behaviors [3], [4], [5], [6]. However, when exceptions are not handled properly, there may be serious consequences. In Java, the Java Virtual Machine (JVM) provides a default exception handling mechanism. In general, when an uncaught exception occurs and no handler exists, the stack trace is produced and the thread is terminated. This behavior has the same result as a try/catch block with only a print stack trace statement inside the catch block [7]. Such behavior is usually unacceptable in high assurance systems. One of the motivations of this paper is the case of the Ariane 5 [8], [9] in June 1996, a widely-known example of accidents in software engineering. The maiden flight of the Ariane 5 launcher angled off its path, broke up and then exploded around 40 seconds after ignition, causing a $500 million financial loss. The key to this tragic incident was misuse of the exception handling mechanism. The inertial
1530-2059/12 $26.00 © 2012 IEEE DOI 10.1109/HASE.2012.24 90

reference system, which is a part of the flight control system, did not send correct attitude data to the on-board computer software because the unit failed as a result of an exception in the inertial reference system. The exception occurred when a 64-bit floating point value was converted to 16-bit signed integer value. The value of the floating point number exceeded the maximum for a 16-bit signed integer value. This led to an operand error that was caught by the exception handling mechanism. However, instead of trying to give estimated attitude information to the onboard computer or perform other recovery actions, both inertial reference system software units were terminated. This incident could have been avoided by more cautious design and implementation of exception handling [10], [11], [12]. In 1980, Hoare talked about exception handling in his Turing award lecture, “...Gradually, these [Ada design] objectives have been been sacrificed in favor of power, supposedly achieved by a plethora of features and notational conventions, many of them unnecessary and some of them, like exception handling, even dangerous...” [13]. Back then, software systems were neither as complex nor as safety critical as software systems of today. Also, exception handling was not an enforced feature of any programming language, which is not the case today (e.g., Java). Therefore, we cannot conclude that exception handling mechanisms are unneccessary any more, because there are many more errors to handle especially in high-assurance systems or complex safety critical systems [1]. The main goal of this study was to answer the question “Is exception handling risky?”. Our literature review found that few studies have addressed this issue [14]. We conducted a preliminary study of exception handling use and defects from major releases of Eclipse. The defects related to exception handling will be called exception defects in this study. From the Oxford Dictionary, one of the meanings of “risk” is “the possibility that something unpleasant or unwelcome will happen”. In software engineering, the occurrence of a defect can easily be considered an unpleasant or unwelcome event. The risk of exception handling defects is summarized by the defect density of exception handling constructs. Moreover, the prevalence of exception handling can be measured to determine the frequency of the exception handling

a chi-squared test was performed to calculate the odds ratios between two categories of classes (exception classes and non-exception classes) for both pre-release and post-release defects. other forms of exception declarations and extending exception classes are not considered in this study because we want to focus on exception handling constructs only as defined above. expert developers think that exception handling is an important part in the development process. To better account for size. there is only one study that empirically addresses the relationship between exception handling and defects. We also propose a measure of prevalence of exception handling. The preceding studies warrant further examination of exception handling constructs as their benefits and risks have not been thoroughly vetted. etc. II. STUDY METHODOLOGY This study tries to answer whether using exception handling is risky by analyzing the defect density related to exception handling constructs compared to the overall defect density. She presents some techniques. Collecting Exception Handling Data Exception handling constructs in this study include all try/catch and throw statements which could be in the following forms [17]. Shah et al. • A try statement followed by a single catch statement and/or finally statements • A try statement followed by multiple catch statements and/or finally statements • Nested try/catch/finally statements • All throw statements located outside try/catch statements • All throw statements located in any try/catch statement are considered part of that try/catch statement Exception classes are those with one or more exception handling constructs. Two Python scripts were implemented to aid in this process. Her study analyzed three versions of Eclipse. The number of classes that used exception handling and the number of classes that did not use exception handling for each release were collected. Exception handling is risky if the defect density is high for exception handling constructs. A. and exception handling information. The rest of this paper is organized as follows. we investigated risk from a different perspective by analyzing the exception defect density and the overall defect density of the post-release defects. a new study methodology was designed and performed instead of repeating Marisnescu’s procedure [14]. exception handling is a concern when the prevalence is large. Then. each file is analyzed to obtain required information including the following information: • General information such as source lines of code (SLOC) and number of Java files. On the other hand. After that. 2. guidelines and design patterns for implementing exception handling.1 and 3. Therefore. RELATED WORK To the best of our knowledge. Some of them use exception handling mainly for debugging the code. Qualitative and quantitative analysis of the results are discussed in Section V. Threats to validity are addressed in Section VI. Source lines of code means only non-comment and non-whitespace lines of 91 . 2. The input of the script is a parent directory path of the project that we want to analyze. the script traverses through all directories and subdirectories to locate all Java files. Brock [16] points out that bad exception handling implementations can affect the quality of the software. Section II highlights related work. Marisnescu [14] conducted an empirical study to investigate whether classes that use exception handling are more defect prone than classes that do not use exception handling. The results show that classes that used exceptions were more likely to exhibit defects than classes that did not have exception handling constructs for both pre-release and post-release defects. Prerelease and post-release defect information of each release was extracted from CVS and Bugzilla to see whether defects were in an exception class or non-exception class. study subjects. However. tools used for collecting data.constructs in the source code. section VII summarizes the results. if the defect density of exception handling constructs is low. With a given directory path. Finally. gives conclusions of the study. Section III explains the study methodology. Statistics representing the prevalence of exception handling are presented in this paper. The first one analyzes a Java project and gives general metrics of that release such as source lines of code (SLOC). [15] studied the viewpoints of novices and experts in the software industry. number of Java files. but some researchers have pointed out their importance. namely. Section IV presents the quantitative analysis. If the defect density of exception handling constructs is high especially when compared to the overall defect density then implementing exception handling may be risky. and the quantitative results obtained. The second script was developed to fetch the defect information from Bugzilla. perform keyword searches and provide the number of defects related to exception handling. then implementing exception handling mechanism is less risky. However. and outlines possible future work. Both usual and exceptional behavior should be considered concurrently during developing the system. Exception handling mechanisms seem to be neglected in the software engineering literature.0.0. Also. III. The study indicates that novice developers are likely to ignore exceptions when developing a system because exception handling is complex.

no workaround Prevents function from being used. In order to obtain exception defect numbers reported in Bugzilla. no workaround. minor releases were not included. enhancement defects are not considered because we want to analyze only actual defects of the system not requests for enhancements. pre-release and post-release defects. The Eclipse project was first developed by IBM in 2001. B. The Eclipse Foundation is an independent not-for-profit corporation that administers the Eclipse community. Study Subjects Six Eclipse releases were chosen as the study subjects. At this stage. but a work-around is possible A problem making a function difficult to use but no special work-around is required A problem not affecting the actual function. number of files with exception handling constructs and number of exception lines of code. which is an open source community. Apart from categorizing defects by release numbers. high reliability is required so that it can be used to develop high assurance products. First. Eclipse source code has high prevalence of exception handling constructs. and thus some reported defects might have not yet been resolved by the time of our study in early 2012. All of this information is shown in Table II. Although a development environment is not a “critical” system on the surface. The script processes the list of defect identifers then fetches each bug reporting page and performs keyword searches by searching for terms “exception”. “throw” or “threw” to determine whether or not that defect is related to exception handling. Therefore. post-release exception defects 2500 2000 1500 1000 500 0 No. the Eclipse community follows standard programming practices and has well defined defect reports [19]. Also. Collecting Defects information In this study. We define exception defect as any defects that are related to exception handling constructs. The defect profile of overall pre-release. any event that raises an exception. the Eclipse releases are hosted by the Eclipse Foundation. For example. Each bug reported in Bugzilla has a severity field which indicates the severity of the impact of that defect. The builtin values and meaning of each severity are shown in Table I [21]. another Python script was implemented. code. In this study. we also categorize defects of each release into 2 phases.7 because the post-release period of release 3. but the behavior is not natural A problem not affecting the actual function. Invalid defects. which was created in 2004. post-release non-exception defects Figure 1. Also. There are both full-time staff and volunteers in the Eclipse community working on Eclipse projects [18]. Eclipse is primarily written in Java.1 through 3. Figure 1 shows the trend of postrelease non-exception defects and post-release exception defects over time. in C. We selected Eclipse releases 3. namely. we ruled out Eclipse release 3.Table I S EVERITY VALUES AND M EANINGS Value Blocker Critical Major Normal Minor Trivial Enhancement Intended meaning Prevents function from being used. We chose to evaluate Eclipse because it is a mature and relatively large product and we have full access to its source code and bug reports.6 as study subjects because we wanted to be certain that both prerelease and post-release defects that we analyze were all resolved. usually happened when the user reported an intended behavior as a bug or the bug does not belong to Eclipse. • Exception handling information such as number of exception handling constructs. Eclipse is a software development environment consisting of an extensible plug-in system and integrated development environment (IDE). pre-release defects are defects reported during the 6 months period before the official release date. lists of all defect identifiers of each version were obtained manually and stored in text files. blocking progress on multiple fronts Prevents function from being used. a typo would be an example Request for enhancement No. Post-release defects are defects reported during the 6 months period after the official release date [20]. post-release and exception defects are displayed in Table III. White spaces and lines containing comments only are not included. Moreover. marked as INVA. Table IV displays the numbers of exception defects by 92 . Comparison between total post-release non-exception defects and post-release exception defects this study we considered only major releases. terminates after an exception is caught. or any event that is thrown is an exception defect. all defects identified as duplicate and invalid were removed.7 ended late in December 2011. Currently.

We measure risk by calculating defect density. files without exception Total exception constructs Total SLOC SLOC of files with exception SLOC of files without exception Total exception SLOC 3.3 8 14 103 1071 92 46 3.5 (Galileo) 3051 909 3960 587 203 790 3. ANALYSIS Risk and prevalence are the main concepts addressed by this study. Moreover. we focus only on post-release defects due to space constraints. While post-release defects are mostly discovered by the users or external customers during operation. The size unit in this study is a thousand lines of non-comment source code. Overall Defect Density is used as a baseline for analyzing risk.Table II G ENERAL M EASUREMENTS OF E CLIPSE R ELEASES Release Metric Total java files No.RELEASE DEFECTS OF EACH RELEASE Release Defect Types Pre-release Post-release All Pre-release exception Post-release exception All exception 3.1 13 22 151 1274 109 44 3.6 (Helios) 2123 628 2751 479 152 631 Table IV T HE N UMBER OF E XCEPTION D EFECTS BY S EVERITY L EVEL Release Severity Blocker Critical Major Normal Minor Trivial 3.2 (Callisto) 14620 4767 9853 19380 1893437 1158570 734867 167756 3. Pre-release defects are usually found by the test team or internal customers during the development process.RELEASE AND POST. files with exception No.3 9 8 57 259 14 4 3.2 (Callisto) 5690 1884 7574 1104 360 1464 3.4 3 16 35 238 7 4 3.6 1 27 54 330 38 36 severity level of each release.3 (Europa) 4635 1685 6320 883 351 1234 3.4 (Ganymede) 3689 1314 5003 691 303 994 3.1 (N/A) 7437 1998 9435 1654 385 2039 3. Table V shows the numbers of non-exception defects by severity level.3 (Europa) 16036 5103 10933 20955 2087566 1262222 825344 180933 3.1 (N/A) 12300 4058 8242 16825 1595837 968714 627123 143160 3.5 0 5 25 164 9 0 3. Overall Defect Density = Total Number of Defects Total SLOC × 10−3 (1) 93 . This section presents the rationale behind our quantitative results. Therefore. Defect density is a measure of the number of known defects detected per unit of software or component size.4 (Ganymede) 18092 5644 12448 23216 2346114 1398714 947400 198424 3.5 1 8 61 541 63 32 3. IV.5 (Galileo) 18836 6047 12789 25547 2463001 1517064 945937 217769 3.4 5 16 73 812 77 28 3. post-release defects have more severe consequences for users and are far more expensive to fix than pre-release defects.6 (Helios) 20599 6638 13961 27978 2674520 1653658 1020862 236391 Table III N UMBER OF PRE . High defect density means high risk while low risk is indicated by low defect density. The Overall Defect Density is the ratio of the total number of post-release defects to the size of the software (see Equation (1)).2 4 23 42 284 6 1 3.2 11 35 136 1194 106 42 3.1 8 13 46 298 14 6 3.6 2 8 19 118 4 1 Table V T HE N UMBER OF N ON -E XCEPTION D EFECTS BY S EVERITY L EVEL Release Severity Blocker Critical Major Normal Minor Trivial 3.

689 0. This tells us the density of exception defects that are closely related to exception handling constructs. The prevalence of exception handling implementations can be determined by comparing the size of exception handling code to the total size.2 0.092 Exception Defect Density of Exception Classes Non-Exception Defect Density Exception Defect Density of Exception Handling Constructs Exception Ratio Defect 1. We discuss the application of the above formulas related to our research questions in the next section. If exception classes are widespread in the source code.231 0.242 small portion of the source code. DISCUSSION Is exception handling risky or not in terms of defect densities? How prevalent are the exception handling constructs in the source code? The prevalence gives perspective on the degree of risk.Next. This defect density is the ratio of the number of non-exception defects (defects that are not related to exception handling) to the portion of code that does not contain exception handling constructs. Other defects in exception classes that are not related to exception handling are excluded. the degree of risk is elevated. tells us if the exception handling constructs are a large portion of the lines of code in exception classes.110 2. A defect containing the keyword “exception” is called an exception defect.560 0.223 0.314 0.134 3. Exception Defect Ratio = Number of Exception Defects (5) Total Number of Defects Table VI D EFECT D ENSITIES Release Defects/KSLOC Overall Density Defect 3. Low prevalence of SLOC of exception classes means classes with exception handling constructs are a 94 .193 0.932 0. Prevalence of SLOC of Exception Classes = SLOC of Exception Classes (6) Total SLOC We also determine whether classes containing exception handling constructs are prevalent compared to the total number of classes (shown in Equation (7)).883 2.471 1.217 3.995 0. we can measure the risk of code that is not related to exception handling by calculating the non-exception defect density which is shown in Equation (4). the prevalence value is high.940 0.6 are displayed in Table VII.1 through 3.252 0. Non-Exception Defect Density = Number of Non-Exception Defects (4) (Total SLOC − Exception Lines of Code) × 10−3 Equation (5) calculates the number of exception defects out of the total number of overall defects by directly comparing both quantities.5 0.700 1. V. Exception Defect Density of Exception Constructs = Number of Exception Defects (3) Exception Lines of Code × 10−3 Moreover.208 0. The quantitative results of the risk or defect densities and the exception defect ratio are shown in Table VI The concept of prevalence is defined as how common exception handling constructs are in the source code.311 3.6 0.191 0. Equation (6) calculates the prevalence as SLOC of exception classes divided by total SLOC of all classes. Exception Defect Density of Exception Classes = Number of Exception Defects (2) SLOC of Exception Classes × 10−3 The exception defect density of exception constructs is measured by the number of exception defects divided by source lines of code of exception constructs as shown in Equation (3).397 3.3 0.369 0. a class with at least one exception handling construct is called an exception class.146 0. Exception handling is risky when the value of exception defect density is high. Prevalence of Exception Classes = Number of Exception Classes (7) Total Classes Prevalence of Exception Constructs in the exception classes as shown in Equation (8).527 0.235 0.4 0.807 0.1 1.643 0.195 0. This tells us the density of exception defects that reside in exception classes only.278 3. Prevalence of Exception Constructs = Exception Lines of Code (8) SLOC of Exception Classes The prevalence values of exception handling of Eclipse 3. If the prevalence of exception handling is also high. The exception defect density of exception classes is measured by the number of exception defects divided by source lines of code of exception classes as shown in Equation (2).

1 3. Thus. The defect density of exception classes only represents the density of the exception defects regardless of the other defects in the exception classes.618 0.5 3. Since the size of exception classes (shown in Figure 4) is considerably higher than the size of non-exception classes. 1400 1200 1000 800 600 400 200 0 Distribution of Exception Defects Based on Severity 3. This might be a result of better testing methodology or development processes. but the severity levels are also a concern.142 0. Figure 5 shows the defect densities of Eclipse over 6 years.235 defects per KSLOC.6 Figure 3. Considering the trends of defect densities.322 350 300 250 200 150 100 50 3. The fact that the number of defects decreases over time was reflected in the defect densities.605 0.330 0.143). Table VII also shows that every release has almost identical prevalence figures.643 defects per KSLOC) is much higher than the overall defect density and other defect densities. Likewise.143 0. the defect density of exception classes is very low.Table VII P REVALENCE OF EXCEPTION HANDLING Release 3. Distribution of Non-Exception Defects Based on Severity classes of release 3.6.1 3.616 0.6 0. considering the fact that exception handling is not the main purpose of the software. This implies that the exception defect density of exception handling constructs is not only high.5 3.4 3.326 0. In other words. Compared to the overall defect density.3 3.1 Prevalence of SLOC of Exception Classes Prevalence of Exception Classes Prevalence of Exception Constructs 3. the defect densities of the lastest release of the study subjects (3. This indicates that the defect density of defects that are closely relately to exception handling constructs is high and more risky than the baseline. On the other hand. Like defect densities.148 0. This is a result of large exception class size compared to nonexception class size. the prevalence of SLOC of exception Figure 2. Figure 2 and Figure 3 show that both exception defects and non-exception defects have similar distributions.2 3.6 has the highest value (0.4 3.144 0.145 0.143 0 In general.6) is discussed in this section to represent all the older releases which have a similar profile.618). The prevalence of exception classes is 0.3 3. all of them slowly declined. Based on the results.4 3. The overall defect density is used as a baseline in analyzing the risk of exception handling.195 defects per KSLOC) is slightly lower than the baseline defect density.6 gradually decreased over time. However. The majority of exception defects are normal defects while the second-most are major defects (see Table I).1 through 3. the non-exception defect density (0. the value of the defect density of exception classes is substantially lower.318 0. the number of both pre-release and post-release defects of Eclipse 3. the defect density of exception handling constructs (0.2 3. Therefore.321 0. the overall defect density is 0.612 0. every release also has a similar profile of exception defects at each severity level (see Table IV).6 0.322.596 0. the prevalence of exception handling lines of code is smaller but still substantial (0. which means that the classes that have exception handling constructs are common.3 3. Compared to the overall source code.2 3.5 3.607 0. only the results of the latest release are discussed. the reliability of the system has been improved over time.312 0. exception handling constructs are SLOC of classes without exception handling constructs SLOC of classes with exception handling constructs 3000000 2500000 2000000 1500000 1000000 500000 0 Figure 4. In Eclipse 3. Comparison between the size of classes with exception handling constructs and classes without exception handling constructs 95 .

The internal threats to validity are from the data collection process. Internal threats to validity are concerned about the appropriate use of methods and tools including the consistency of the measurements. especially high assurance or safety critical systems. design. there might be severe consequences when errors are not handled properly. However. Furthermore. exception handling is enforced [22]. We compared the exception defect densities to the overall defect density and compared the exception defect ratios. Figure 5. the generalization of the study and the applicability of the tool to other programming languages. because exception defects that are not caused by exception handling constructs. To analyze the source code. THREATS TO VALIDITY In this study there are several threats to validity. they were manually tested by sampling to make sure the results are acceptably accurate. C ONCLUSION Our goal was to determine whether using exception handling is risky by performing measurement of source code and analyzing bug reports. in some programming languages such as Java. the sizes of our subjects are considered medium to large. Although the overall number of defects is declining over time. both source code and bug reports. Even though the overall number of defects decreases over time. VII. We found that in Eclipse releases. The script also collects other general information from the code such as the number of Java classes and source lines of code. especially in safety critical systems [12].6 a small portion of the source code. such as defects found in the code that requires exception handling. have more complex functionalities. The results show that the exception defect density of exception handling constructs are approximately three times higher than the overall defect density.6 the overall defect density is 0. Defect Densities of Eclipse 3. implementation and testing are crucial. For example. Bug reports were investigated to determine which defects are related to exception handling. External threats to validity are concerned with the degree of generalization of the study. Exception handling mechanisms might not be implemented in all software systems or exception handling in other programming languages might have other issues. Even though the false positive rates of both programs have not been quantitatively tested. Eclipse releases which have a high prevalence of exception handling constructs were selected as the study subjects. This threat can 96 . The exception defect ratios are over 20%. Also. We also proposed statistics to calculate the prevalence of exception handling code. Eclipse is an opensource. All of these issues affect the generalization of our study. The six major Eclipse releases studied are not generalized to the population of every classification of software. the exception defect density of exception handling constructs is much higher than the overall defect density. The results tell us that using exception handling is risky and further studies should be conducted to find the root causes of exception defects and thoroughly understand the correlation between exception handling and defects in the software systems. the trend of the exception defect ratios slightly increases.235 defects per KSLOC and the exception defect density of exception handling constructs is 0.643 defects per KSLOC (see Table VI).1 – 3. Also. There are two external threats to validity. This study explored whether using exception handling has a high possibility of defect occurance. The tools implemented for this study were designed for analyzing Java code and fetching data from Bugzilla only. VI. We believe implementing exception handling mechanisms is essential for software systems especially safety critical and high assurance systems. the trend of exception defect ratio in Table VI is slightly increased. Exception handling mechanisms are advanced features of most modern programming languages that many programmers neglect [15]. The fact that the defect density of exception handling constructs is relatively high tells us that developers should pay more attention to implementing and testing exception handling code compared to normal code. both internal and external. Careful planning.džĐĞƉƚŝŽŶ ĞĨĞĐƚ ĞŶƐŝƚLJ ŽĨ džĐĞƉƚŝŽŶ ŽŶƐƚƌƵĐƚƐ KǀĞƌĂůů ĞĨĞĐƚ ĞŶƐŝƚLJ EŽŶͲ džĐĞƉƚŝŽŶ ĞĨĞĐƚ ĞŶƐŝƚLJ džĐĞƉƚŝŽŶ ĞĨĞĐƚ ĞŶƐŝƚLJ ŽĨ džĐĞƉƚŝŽŶ ůĂƐƐĞƐ ϯ ϬϬϬ ϯ͘ϬϬϬ Ϯ͘ϱϬϬ Ϯ͘ϬϬϬ ϭ͘ϱϬϬ ϭ͘ϬϬϬ Ϭ͘ϱϬϬ Ϭ͘ϬϬϬ ϯ͘ϭ ϯ͘Ϯ ϯ͘ϯ ϯ͘ϰ ϯ͘ϱ ϯ͘ϲ be addressed by extending the analysis to other systems and programming languages. namely. Our second external threat to validity is the applicability of the tool. It is as well written in Java. it is important to understand this feature because today’s software systems. In order to do so. a second tool was implemented to perform keyword searching on Bugzilla to determine whether that defect is related to exception handling. user support type of application used mainly by developers. might be included by the script. The use of a keyword searching strategy to extract the defect information is a threat to validity. we implemented a script capable of extracting the number of exception handling constructs and the number of source lines of code of exception handling code. in release 3.

“A side-by-side comparison of exception handling in Ada and Java. 411–414.html.di. 2010. “Exception handling design issues. de Dinechin. A. pp. “The emperor’s old clothes. vol. Hoare. Oct 1993.org/org/. no. 1994. T. K. Gough.” IEEE Transactions on Software Engineering. no. pp.” Computer Languages. Oct 2000. no. [5] R.jenkov . pp. pp. “Toward exception-handling best practices and patterns. and B. 1992. The findings and opinions in this paper belong solely to the authors. accessed: 04/09/2012. The false positive rates of both scripts can be quantitatively evaluated to mitigate internal threats to validity. [15] H. 393–401. 2. pp. 2011. J. no. 5. exception handling. pp. 69–87. and M. 11–13. 2000. 2. 2.” http://www.” IEEE Concurrency. 72–79. Wirfs-Brock. With an improved tool. vol. empirical study. 97 . Romanovsky. requirements and software lifecycle. February 1981.” IEEE Transactions on Software Engineering. [13] C.” http://wiki. 56–60. pp. 18. [9] “The bug that destroyed a rocket. Strohmeier and S. Gorg.oracle. “Exception handling: Expecting the unexpected. which software systems implement exception handling mechanisms and what types of exceptions are handled? An empirical study on whether exception handling code is more complex than non-exception handling code would help explain why using exception handling is risky. [3] J. “Are the classes that use exceptions defect prone?” in 2011 Proceedings of the 12th International Workshop on Principles of Software Evolution and the 7th annual ERCIM Workshop on Software Evolution. 5. Maxion and R. pp.” http://www. we hope this study benefits the software engineering research community by stressing the importance of the exception handling mechanism. B. Wong and E.org/Bug Reporting FAQ. Romanovsky. Recent Trends in Information Reuse and Integration. 8. vol. “Eliminating exception handling errors with dependability cases: A comparative.ing. Goodenough. Yung. Xu. J. 888–906. 2007.eclipse.” ACM SIGPLAN Notices. 301–301. We plan to repeat the same study methodology with other data sets to improve the generalization of the study. 150–161. A. 40. no.” in Proceedings of the 1993 IEEE Region 10 Conference on Computer. Springer. no. Further study can also be done to investigate why exception handling constructs cause defects? What are the root causes of exception defects? Is there any correlation between exception handling and defect location. [20] T. we would also like to thank Dae Glendowne and James Thomson for reviewing and providing valuable comments. Drew and K. no. Finally. A.” no.html. 24. [4] S. 173–174. “Exception handling in C++ without language extension. pp.html.it/damiani/ariane5rep . Marinescu. 10. [14] C. Communication.pdf.” in 2007 Proceedings of the 10th IEEE International Symposium on High Assurance Systems Engineering. B. vol.com/java-exception-handling/checked-or-uncheckedexceptions.” IEEE Transactions on Parallel and Distributed Systems. no. Cui and J. [2] C. Xu.” IEEE Transactions on Software Engineering. [12] R. ˜ [8] “Flight 501 failure. accessed: 04/01/2012. 41–56. 1. 41–45. and M.eclipse.” ACM SIGPLAN Notices. 20. 7. no.com/javase/specs/jls/se7/html/ jls-11. D. Kianmehr. This study is a start for research in this area. 9. 2006. Jiang and B. [7] “Exceptions. vol. 36. pp. Commercial software systems would also be a good source of projects to empirically analyze. 11. ACKNOWLEDGMENT We would like to thank the Empirical Software Engineering Research Group at Mississippi State University for weekly presentations that inspire and open our vision for software engineering research. Randell. a [10] A. 2011.” http://docenti. “C++ exception handling. Control and Power Engineering. R.Finally. vol. accessed: 05/06/2012. pp. “Concurrent exception handling and resolution in distributed object systems. vol. accessed: 04/02/2012. C. pp.” http://docs. [21] “TPTP development process.org/ tptp/home/documents/process/development/bugzilla. vol. accessed: 04/09/2012. similar studies can be performed with systems written in other programming languages to see whether our results are widely applicable. 1019 – 1032. Olszewski. 2005. vol. 2. Harrold. accessed: 04/09/2012. Nov. [11] J. 2001. vol.unipi. pp. J. 27–32. “Understanding exception handling: Viewpoints of novices and experts. Ozyer. 3. [18] “About the eclipse foundation. Chachkov. accessed: 04/09/2012. 4. 2000.” Communication of the ACM.it/ ˜ 009435/issw/extra/ariane5-benari. Gannon. “An efficient and reliable object-oriented exception handling mechanism. 26. ACM.unito. no. [17] A. Tan.” IEEE Software. 23. pp. pp. “On exceptions. [19] “Bug reporting faq. vol. [16] R.html.” http://www. J. Also. Shah. R EFERENCES [1] S. 75–83. 10.eclipse. [22] “Checked or unchecked exceptions?” http://tutorials. [6] Q. 1975. “Data-oriented exception handling.