Professional Documents
Culture Documents
me/PFbM0-4a
http://dbtestunit.wordpress.com/ Many
(but not all) applications under test use one or more databases. The purposes of using a database include long-term storage of data in an accessible and organized form. Many people have only a vague idea about database testing. If you are serious about learning database testing, then read on... Firstly, we need to understand what is database testing? As you would know, a database has two main parts - the data structures (the schema) that store the data AND the data itself. Let us discuss them one by one. The data is stored in the database in tables. However, tables may not be the only objects in the database. A database may have other objects like views, stored procedures and functions. These other objects help the users access the data in required forms. The data itself is stored in the tables. Database testing involves finding out the answers to the following questions: Questions related to database structure 1. Is the data organized well logically? 2. Does the database perform well? 3. Do the database objects like views, triggers, stored procedures, functions and jobs work correctly? 4. Does the database implement constraints to allow only correct data to be stored in it? 5. Is the data secure from unauthorized access? Questions related to data 1. Is the data complete? 2. Is all data factually correct i.e. in sync with its source, for example the data entered by a user via the application UI? 3. Is there any unnecessary data present? Now that we understand database testing, it is important to know about the 5 common challenges seen before or during database testing: 1. Large scope of testing It is important to identify the test items in database testing. Otherwise, you may not have a clear understanding of what you would test and what you would not test. You could run out of time much before finishing the database test. Once you have the list of test items, you should estimate the effort required to design the tests and execute the tests for each test item. Depending on their design and data size,
some database tests may take a long time to execute. Look at the test estimates in light of the available time. If you do not have enough time, you should select only the important test items for your database test. 2. Incorrect/ scaled-down test databases You may be given a copy of the development database to test. This database may only have little data (the data required to run the application and some sample data to show in the application UI). Testing the development or test or staging databases may not be sufficient. You should also be testing a copy of the production database. 3. Changes in database schema and data This is a particularly nasty challenge. You may find that after you design a test (or even after you execute a test), the database structure (the schema) has been changed. This means that you should be aware of the changes made to the database during testing. Once the database structure changes, you should analyze the impact of the changes and modify any impacted tests. Further, if your test database is being used by other users, you would not be sure about your test results. Therefore, you should ensure that the test database is used for testing purpose only. You may also see this problem if you run multiple tests at the same time. You should run one test at a time at least for the performance tests. You do not want your database performing multiple tasks and under-reporting performance. 4. Messy testing Database testing may get complex. You do not want to be executing tests partially or repeating tests unnecessarily. You should create a test plan and proceed accordingly while carefully noting your progress. 5. Lack of skills The lack of the required skills may really slow things down. In order to perform database testing effectively, you should be comfortable with SQL queries and the required database management tools. Next, let us discuss the approach for database testing. You should keep the scope of your test as well as the challenges in mind while designing your particular test design and test execution approach. Note the following 10 tips: 1. List all database-specific requirements. You should gather the requirements from all sources, particularly technical requirements. It is quite possible that some requirements are at a high level. Break-down those requirements into the small testable requirements. 2. Create test scenarios for each requirement as suggested below. 3. In order to check the logical database design, ensure that each entity in the application e.g. actors, system configuration are represented in the database. An application entity may be represented in one or tables in the database. The database should contain only
those tables that are required to represent the application entities and no more. 4. In order to check the database performance, you may focus on its throughput and response times. For example, if the database is supposed to insert 1000 customer records per minute, you may design a query that inserts 1000 customer records and print/ store the time taken to do so. If the database is supposed to execute a stored procedure in under 5 seconds, you may design a query to execute the stored procedure with sample test data multiple times and note each time. 5. If you wish to test the database objects e.g. stored procedures, you should remember that a stored procedure may be thought of as a simple program that (optionally) accepts certain input(s) and produces some output. You should design test data to exercise the stored procedure in interesting ways and predict the output of the stored procedure for every test data set. 6. In order to check database constraints, you should design invalid test data sets and then try to insert/ update them in the database. An example of an invalid data set is an order for a customer that does not exist. Another example is a customer test data set with an invalid ZIP code. 7. In order to check the database security, you should design tests that mimic unauthorized access. For example, log in to the database as a user with restricted access and check if you can view/ modify/ delete restricted database objects or view or view and update restricted data. It is important to backup your database before executing any database security tests. Otherwise, you may render your database unusable. You should also check to see that any confidential data in the database e.g. credit card numbers is either encrypted or obfuscated (masked). 8. In order to test data integrity, you should design valid test data sets for each application entity. Insert/ update a valid test data set (for example, a customer) and check that the data has been stored in the correct table(s) in correct columns. Each data in the test data set should have been inserted/ updated in the database. Further, the test data set should be inserted only once and there should not be any other change in the other data. 9. Since your test design would require creating SQL queries, try to keep your queries as simple as possible to prevent defects in them. It is a good idea for someone other than the author to review the queries. You should also dynamically test each query. One way to test your query is to modify it so that it just shows the resultset and does not perform the actual operation e.g. insert, delete. Another way to test your query is to run it for a couple of iteration s and verify the results. 10. If you are going to have a large number of tests, you should pay special attention to organizing them. You should also consider at least partial automation of frequently run tests. Now you should know what database testing is all about, the problems that you are likely
to face while doing database testing and how to design a good database test approach for the scope decided by you. Why don't you consider professional database testing for your application? Let me know your experiences. You can contact me at isingh 30 AT gmail DOT com (no spaces).
Isn't it time that we stopped talking about data quality and actually started doing something about it?
Here's a few interesting questions to ask someone who isn't convinced that you need to test the DB: 1. If you're implementing code in the DB in the form of stored procedures, triggers, ... shouldn't you test that code to the same level that you test your app code? 2. Think of all the data quality problems you've run into over the years. Wouldn't it have been nice if someone had originally tested and discovered those problems before you did? 3. Wouldn't it be nice to have a test suite to run so that you could determine how (and if) the DB actually works? I think that one of the reasons that we don't hear much about database testing is because it is a relatively new idea within the data community. Many traditional data professionals seem to think that testing is something that other people do, particularly test/quality assurance professionals, do. This reflects a penchant for over-specialization and a serial approach towards development by traditionalists, two ideas which have also been shown to be questionable organizational approaches at best.
Test-driven development (TDD) is an evolutionary approach to development which combines test-first development and refactoring. When an agile software developer goes to implement a new feature, the first question they ask themselves is "Is this the best design possible which enables me to add this feature?" If the answer is yes, then they do the work to add the feature. If the answer is no then they refactor the
design to make it the best possible then they continue with a TFD approach. This strategy is applicable to developing both your application code and your database schema, two things that you would work on in parallel.
When you first start following a TDD approach to development you quickly discover that to make it successful you need to automate as much of the process as possible? Do you really want to manually run the same build script(s) and the same testing script(s) over and over again? Of course not. So, agile developers have created OSS tools such as ANT, Maven, and Cruise Control (to name a few) which enable them to automate these tasks. More importantly, it enables them to automate their database testing script into the build procedure itself.
Agile developers realize that testing is so important to their success that it is something they do every day, not just at the end of the lifecycle, following agile testing strategies. They test as often and early as possible, and better yet they test first. As you can see with the agile system development lifecycle (SDLC) of Figure 3 testing is in fact something that occurs during the development and release cycles, not just during release. Furthermore, many agile software developers realize that you can test more than just your code, you can in fact validate every work product created on a software development project if you choose to. This philosophy is exemplified by the Full Lifecycle Object-Oriented Testing (FLOOT) Methodology.
4. How to Test
Although you want to keep your database testing efforts as simple as possible, at first you will discover that you have a fair bit of both learning and set up to do. In this section I discuss the need for various database sandboxes in which people will test: in short, if you want to do database testing then you're going to need test databases (sandboxes) to work in. I then overview how to write a database test and more importantly describe setup strategies for database tests. Finally, I overview several database testing tools which you may want to consider.
so to ensure that your changes integrate with the changes made by other developers. Every so often (perhaps once every six to twelve months) into production. The primary advantage of sandboxes are that they help to reduce the risk of technical errors adversely affecting a larger group of people than is absolutely necessary at the time.
Figure 4. Sandboxes.
3. Check the results. You'll need to be able to do "table dumps" to obtain the current values in the database so that you can compare them against the results which you expected. The article What To Test in an RDBMS goes into greater detail.
Where does test data come from? For unit testing, I prefer to create sample data with known values. This way I can predict the actual results for the tests that I do write and I know I have the appropriate data values for those tests. For other forms of testing -particularly load/stress, system integration, and function testing, I will use live data so as to better simulate real-world conditions.
Beware Coupling: One danger with database regression testing, and with regression testing in general, is coupling between tests. If you put the database into a known state, then run several tests against that known state before resetting it, then those tests are potentially coupled to one another. Coupling between tests occurs when one test counts on another one to successfully run so as to put the database into a known state for it. Self-contained test cases do not suffer from this problem, although may be potentially slower as a result due to the need for additional initialization steps.
To make a long story short, although we're starting to see a glimmer of hope when it comes to database testing tools, as you can see in Table 2, but we still have a long way to go. Luckily there are some good tools being developed by the open source software (OSS) community and there are some commercial tools available as well. Having said that, IMHO there is still significant opportunity for tool vendors to improve their database testing offerings. Table 2. Some database testing tools.
Category Description Examples IBM Optim Data Privacy tools
critical issue for many organizations. Many organizations must safeguard data by law due to regulatory
compliance
concerns. Tools simulate high usage loads on your database, enabling you to Testing tools determine for load whether your testing system's architecture will stand up to your true production needs. Developers need test data against which to validate their systems. Test data generators can be particularly useful when you need large amounts of data, perhaps for stress and load testing.
Empirix Mercury Interactive RadView Rational Suite Test Studio Web Performance
Test Data Your test data Management needs to be managed. It should be defined, either manually or automatically (or both), and then maintained under version control. You need to define expected results of tests and then automatically compare that
with the actual results. You may even want to retain the results of previous test runs (perhaps due to
regulatory compliance
concerns). Unit testing tools Tools which enable you to regression test your database.
AnyDbTest DBFit
DBUnit
NDbUnit
OUnit for Oracle (being replaced soon by Qute)
SQLUnit
TSQLUnit (for testing T-SQL in MS SQL Server)
XTUnit
Data inspection is more of a debugging technique than it is a testing technique. It is clearly an important technique, but it's not something that will greatly contribute to your efforts to ensure data quality within your organization.
8. Best Practices
I'd like to conclude this article by sharing a few database testing "best practices" with you: 1. Use an in-memory database for regression testing. You can dramatically speed up your database tests by running them, or at least portions of them, against an inmemory database such as HSQLDB. The challenge with this approach is that because database methods are implemented differently across database vendors that any method tests will still need to run against the actual database server. 2. Start fresh each major test run. To ensure a clean database, a common strategy is that at the beginning of each test run you drop the database, then rebuild it from scratch taking into account all database refactorings and transformations to that point, then reload the test data, and then run your tests. Of course, you wouldn't do this to your production database. ;-) 3. Take a continuous approach to regression testing. I can't say this enough, a TDD approach to development is an incredibly effective way to work. 4. Train people in testing. Many developers and DBAs have not been trained in testing skills, and they almost certainly haven't been trained in database testing skills. Invest in your people, and give them the training and education they need to do their jobs. 5. Pair with novices with people that have database testing experience. One of the easiest ways to gain database testing skills is to pair program with someone who already has them.
Current State of Data Management: July 2006 Survey Results Database Application Testing Fundamentals (A Course by the International Institute for Software Testing) Database Unit Testing Whitepaper (Sachin Rekhi) Database Testing (Mike Kelly) Database Testing Strategy in Build (Vincent Massol) Ensuring Database Quality Evidence that Agile Software Development Scales
On Relational Theory The Process of Database Refactoring Questioning Traditional Data Management Simplified Database Testing Using Enterprise Services Survey Results (Agile and Data Management) Test Driven Database Development (TDDD): September 2006 issue of Quality Software
and Testing.
Test Driven Development: A Practical Guide by Dave Astels Test Driven Development: By Example by Kent Beck Test Driven Development of Relational Databases (IEEE Software, May/June 2007) Testing 1, 2, 3... (Phil Zoio) Unit Testing a Database (Ronald Bradford) Unit Testing and Code Generation -- Grant Fritchey writes about unit testing T-SQL code. Unit Testing Database Code (Richard Dallaway) Unit Testing Stored Procedures in SQL Server ( Sudheer Palyam ) Visual Studio Team Edition for Database Professionals What Do You Test for in Database Testing Other Than Data? What to Consider When Testing Databases (DDJ Newsletter, June 2007) Whence Data Management? explores the current state of database management, including
database refactoring