M13 Final Test Critique

by Kristy S. Garner, EdS

COUN 7335
Summer 2013

1)

What is the test instrument name and what is its purpose?

The name of the test instrument is Diagnostic Achievement Battery (DAB-3). Its
purpose is gather recommendations concerning effective construction on the basis of
the students strengths and weaknesses in listening, speaking, reading, writing, and
mathematics. This assessment is designed to answer three fundamental types of
questions, those pertaining to the childs comprehension of the task, thought processes
and learning potential so that professionals can make the appropriate
recommendations to further ensure and improve student success (Newcomer &
Ginsburg, 2001). The instrument consists of the initial DAB-3 test and the follow-up
probes.
2)
Describe and evaluate the norm group. Do you think it is representative?
Do you think the norm group is current? Do you believe the size of the norm
group was large enough? Are the samples related to the population you intend to
use the test with? Explain your answer.
The assessment utilizes national norms which were standardized on 1,534
students across 13 states. The sample is representative of the national population with
regard to gender, race, ethnicity, geographic region, and urban/rural residence
(Newcomer & Ginsburg, 2001). I feel that I do not have enough information to
determine whether or not the norm group is representative. I have questions which I
have not been able to acquire answers. Which states were used in the norm group?
How were the participants identified? I would have to say that I do not feel the norm
group is large enough. In my opinion, the researcher would have to take a sample from
each state to attain an appropriate norm group; however, I do trust that the researcher
maintained an adequate standard by which the norm group was identified thereby
validating the assessment results. The data for the particular test I attained was
collected during 1997 to 2000 and this third edition was published in 2001. Since we
are now in 2013, I do not feel that the norm group is current and represented accurately
as it is 12 years old. The population has grown and the demographics have change for
our society. I feel a more recent assessment is needed to effectively evaluate our
students. With age not being a consideration, I do feel the samples are related to the
population I intend to use the test with. My intention is to perform the assessment at the
elementary or middle school levels which aligns with the age range of 6 to 14 for the
participants. The samples gathered are from an array of ages across the spectrum to
provide the examiner with a more comprehensive example of responses for any age
within the range.

3)
Describe and evaluate each method used to estimate reliability. Does the
reliability evidence support a decision to use the instrument? Explain your
answer.
The DAB-3 generates standard scores (M=10, SD-3 for subtests, and M=100,
SD=15 for the composites), percentile ranks, and age/grade equivalents (Newcomer &
Ginsburg, 2001).
Coefficient alphas for each subtest and composite according to age
are provided by the author as a measure of internal-consistency
reliability. Of the 126 subtest coefficient alphas, 102 meet or exceed .
80. Subtests having several lower coefficient alphas include
Synonyms, Punctuation, and Math Reasoning. Among the composite
scores, all have alpha coefficients that exceed .80, with the Listening,
Spoken Language, and Written Language coefficients exceeding .90.
The Total Achievement coefficients range from .98 to .99. Coefficient
alphas are also provided for gender and ethnicity groups, as well as
for students with learning disabilities. These reliabilities all meet or
exceed .80, except for Punctuation, Writing: Contextual Language,
and Math Reasoning among students with learning disabilities, as well
as Writing: Contextual Language among African American students
(DAB-3, 2001).
Using 65 Pennsylvanian elementary and middle school students who utilized
test-retest during a two-week period indicated adequate reliability. The test-retest
reliability (great than .80) for all subtests, except for Writing: Story Construction and
Writing: Contextual Language (DAB-3, 2001). I like how test-retest reliability was
conducted on a small sample size; however, I feel that perhaps it may have been too
small. I would like to know why Pennsylvania was chosen over any other state.
Nonetheless, I still feel as though this assessment is reliable because I have seen its
use in our own school and how the recommendations are useful in ensuring student
improvement.
4)
Describe and evaluate each type of validity evidence. Does the validity
evidence support a decision to use the instrument? Explain your answer.
There is evidence of validity based on relations with other measures. Predictive
criterion validity is utilized by comparing the DAB-3 and Stanford Achievement Test with
a sample of 70 Pennsylvanian students.
Seventy-five percent of the coefficients were in the "high" range (.60
to .80). Corresponding composite correlation coefficients (such as
reading with reading, math with math) ranged from .52 to .80. Higher
scores were obtained by older students than younger students, and
scores for students who were expected to score lower or higher due to

having a learning disability or being identified as gifted demonstrated

corresponding performance on the DAB-3 (Dab-3, 2001).
While there is no data provided to determine that DAB-3 scores are beneficial in
identifying academic difficulties or for monitoring intervention effects, I still feel the
assessment is a valid source of information. Research has been performed to compare
the DAB-3 with another valid standardized test to prove its beneficial effects. I do have
to ask why students from Pennsylvania are the ones being chosen for the tested sample
size. Also, 70 students seem to be a small group to be making generalizations about
the test. I still have to go back to the evidence seen in my own school and how it works
for our students with identifying academic issues.
5)
Describe the practical aspects of the instrument, focusing on issues
related to time required for administration, ease of administration and ease of
scoring.
The student may feel anxiety with the administration taking place in an individual
setting; therefore time should be taken before the assessment for the examiner to
establish a rapport with the child. A child who is relaxed and confident will be more
open to allowing access to their inner thoughts. The administration of the test seems to
be easy in terms of its flexibility with testing sessions. It can be given over multiple
sessions or in one based on the childs age, experience, and attitude at the time of
testing. If the probes are administered after the DAB-3, their duration should not
exceed 10 minutes which would make an approximate total assessment time of 40
minutes. If the probes are administered in a separate session, then a full 30 to 40
minutes can be devoted based on the willingness of the child. It is not recommended to
probe the child on the same day that the DAB-3 is administered because normative
scores would not be attained regarding the childs achievement (Newcomer & Ginsburg,
2001). The test does not appear to be very easy to score because it does not focus on
right or wrong answers. The examiner must score individual responses to the probes
with no objective or standard way to garner information about the childs thinking and
learning potential.
6)

Are there any multicultural issues? Explain your answer.

Yes, there are multicultural issues. The test is administered in English. If the
assessment is to be used with children who have limited proficiency in English, the test
must be translated into the childs first language. Relatively few items were identified
as being moderately to severely biased for different ethnic groups, and none were
identified as being gender biased (DAB-3, 2001).

7)
Based on your review of this test, would you adopt this instrument?
Explain your answer.
Based on my review of this test, I would adopt this instrument. I am most
impressed with how the assessment looks first at whether or not the student
comprehends the task. Students may be able to read the question perfectly, but they do
not have a clear understanding of what is being asked of them. The examiner can
make the distinction as to whether or not the issue can be solved by simply rewording
the task or the clarification of particular terms. I also like how the examiner utilizes such
questions as How did you do it? Tell me how you knew. What did you figure that out?
What were you saying to yourself? to explore the way the child thinks. In the end, the
learning potential is accessed and a determination is made as to the childs current level
of intellectual functioning and whether or not they can transcend that level (Newcomer &
Ginsburg, 2001).

References

DAB-3. (2001). Retrieved from

http://college.cengage.com/education/salvia/assessment/10e/resources/salvia_test_
diagnostic_battery.pdf

Newcomer, P. L., & Ginsburg, H. P. (2001). Diagnostic Achievement Battery DAB3

(3rd ed.). Austin, TX:
Pro-Ed.