Database Application Testing

Presented by Shreyas Kale

Agenda

Introduction AGENDA Approach DICFG Approach Discussion

Introduction

Database applications are commonly used across industry. Many companies are completely reliant on database software to be consistently correct Little research has been done on testing and verifying database interactions Several issues arise when trying to perform Database testing

Introduction (cont)

Many aspects of correctness in a Database application
• Does the application program behave as specified? • Does the database schema correctly reflect the real world data being modeled? • Are security and privacy protected properly? • Are the data in the database accurate? • Does the DBMS perform all database actions correctly?

AGENDA

AGENDA: A (test) GENerator for Database Applications Attempts to answer: Does the application program behave as specified? Written by David Chays et al.

AGENDA Introduction

This journal paper introduces a framework for testing database applications and describes a complete tool set, based on this framework, which has been prototyped. The main purpose of this prototype is to populate a database, generate test inputs to an application, execute the application on those inputs, and check for some aspects of correctness in both the application output and the resulting database state.

AGENDA Introduction

The components of this system are
• A Parser that gathers relevant information from the database schema and application • A State Generator that populates the database with meaningful data that satisfy database constraints • An Input generation tool that generates test cases for the application • A State Validator that checks the resulting database state after operations are performed by the application • A Output Validator tool that assists the tester in checking the database application's output.

AGENDA View

AGENDA Parser

The AGENDA Parser extracts relevant information from the application’s database schema, application query, and tester supplied sample values files. The parser makes this information available to the rest of the prototype by creating an Internal database called AGENDA DB The AGENDA DB is used and modified by the rest of the prototype tools

AGENDA Parser Details

The AGENDA Parser is a generic SQL Parser based on PostgreSQL Instead of storing the AbstractSyntaxTree that is generated by PostgreSQL, or a complex data structure which is dynamically allocated, the AGENDA Parser simply populates an internal database to store the Syntax tree and other relevant information about the tables, attributes and constraints of the application’s database.

AGENDA Parser Details

Although the design implemented in the AGENDA Parser is reliant on PostgreSQL, the Parser tool can be ported to another DBMS without requiring any changes for the other four tools in the AGENDA prototype. This provides significant flexibility for testing which is needed, since several DBMS are available and used in practice.

AGENDA Parser Details

The main focus of the AGENDA Parser is to extract information about the tables, attributes and constraints of the application schema. The current version of AGENDA can detect the following constraints:
• • • • Uniqueness constraints Referential integrity constraints Not-NULL constraints Simple boundary constraints

AGENDA Parser Details

Once the AGENDA Parser has parsed the application schema, the samplevalues file is parsed. This gives sample values with which to populate the application database, and this information is stored in the AGENDA DB.

AGENDA Parser Details

Once the sample files have been processed, the application query is parsed in, in order to extract information about the query’s parameters. This information is important throughout the rest of the prototype for test generation and output checking. The query is broken down into a query tree, and the input and output host variables are identified. Any associations between the input and output host variables are stored in the AGENDA DB.

AGENDA Parser View

State Generator

The State Generator tool does the actual population of the database. The State Generator uses the tables and constraints provided in the AGENDA DB to output a consistent populated database which adheres to all constraints identified by the AGENDA DB.

State Generator

The State Generator prompts the user for the desired heuristic with which to populate the database. The options are
• Boundary Values • Duplicates • Null values • All groups

State Generator

Boundary Values set certain constant values close to their boundaries. This approach assumes that more faulty behavior is shown when the database is populated with boundary values. Duplicates makes duplicates in the database to verify that the query can correctly obtain the needed information when duplicate entries exist. Nulls add in NULL values in various locations in the database. The idea is that NULL values can cause faults in certain database applications The allgroups simply combines all the above approaches and incorporates each.

Input Generator

The Input Generation Tool creates test cases by instantiation the application’s inputs with concrete values. The Input Generator requires a heuristic to be chosen by the tester. These heuristics guide the Generator to decide which inputs to place in the application’s query.

Input Generator

The heuristics are:
• • • • • Boundary values Duplicates from the Application DB state Nulls All groups All templates

These heuristics are similar to those used in the State Generator, except the “All templates”, which uses test templates for query input selection.

State Validator

The State Validator actually verifies the state of the database after the test queries have been run. It is near impossible to do full verification on a database without specification information State Validator provides the user to specify preconditions, postconditions and relationships between the old and new values.

State Validator

The State Validator provides automatic logging of all changes made to the application tables. This information can be very useful to a tester. The State Validator also provides checking state changes by means of database constraints, providing another layer of verification.

Output Validator

Similar to the State Validator. The Output Validator submits a query on the application tables and saves the affected rows into the log table. The query is based on the application query. After the execution of the query and storing the log files, the Output Validator applies the integrity constraint to the log table to check the resulting output.

Results

The authors performed tests to determine the overhead and time cost of the AGENDA prototype. Although the overhead was shown to be low on very small databases, the larger (more realistic) databases varied widely in their overhead. The Time cost seemed to be very low for most portions of the application. This may not be an accurate conclusion however. . .

Limitations

One of the blaring limitations of this prototype is its restriction to one query. This is not realistic in any database application. This also renders the results of the experiment useless, since it provides no information on scaling, or if scaling is even possible! Another major limitation described in two reviews was the entire lack of any tangible results. No case study was done. No useful experiment was done. No comparison to other approaches was done. This could be due to the limitation above.

AGENDA Conclusions

The proposed approach was very interesting, and many important problems in Database Application Testing seemed to be addressed in a accurate way. However, the limitations taken into consideration simplify the prototype to a point where its actual relevance is in question.

Family of Test Adequacy Criteria for Database-Driven Applications

This paper presents an approach to using test adequacy criteria to assess the quality of test suites for database-driven applications. The proposed approach is based on creating a Database Integration Control Flow Graph (DICFG) which provides the necessary information to analyze the test cases to be run. The results from this paper were that the DICFG approach does not take significant overhead or time cost.

DICFG

Conclusions

Database Application testing is still a large area for research. While certain approaches have worked in seclusion, most database applications are extremely immense and would take a significant amount of time to fully analyze. Very useful to industry and academia. . .Important problem to solve (or adequately address)

Discussion

??

Sign up to vote on this title
UsefulNot useful