COGNITIVE LEVELS ON THE MCA-IIs

Cognitive Levels Defined

Minnesota students are expected to acquire knowledge and skills at each grade level; the Minnesota Academic Standards identify these expectations. The No Child Left Behind (NCLB) Act of 2001 requires states to demonstrate that its test questions, or items, reflect the depth and breadth of the state standards; these standards are made up of skills called benchmarks. One of the ways Minnesota meets this requirement is by categorizing the difficulty of the task within an item. The Minnesota Comprehensive Assessment Series II (MCA-II) is the set of Minnesota reading, mathematics, and science tests that meets the requirements of NCLB, and its items are categorized in two ways: item difficulty and cognitive complexity. To fairly measure the educational achievement of students, tests must be of equivalent difficulty from one year to the next. Item difficulty is determined after items have been field-tested (used in tests with Minnesota students but not counting toward their score). At this point, we know the percentage of students who get a particular question correct. MDE uses this knowledge to help build a test that is equally difficult from year to year. Cognitive complexity refers to the cognitive demands of an item. Test makers use a classification system that helps them rate the complexity of the skill necessary to answer the item correctly. Beginning with the MCA-IIs in 2006, MDE has used an adapted version of Blooms Taxonomy to classify items. Teachers are familiar with this classification, and it is their expertise that helps MDE identify the complexity of each item used on the MCA-IIs.

The Purpose of Cognitive Levels

The rationale for classifying items by their complexity is to focus on the expectations of the item, not the ability of the student. The thinking an item demands of studentswhat the item requires the student to recall, understand, analyze, and doassumes that the student is familiar with the concepts of the task. Items are chosen for the MCA-IIs based on their fit with the Minnesota Academic Standards and their grade-level appropriateness. The complexity of the item is independent of the particular curriculum a student has experienced. The cognitive complexity of a multiple-choice item is generally NOT dependent on the distractors (answer choices). However, the distractor options may affect the difficulty of the item.

How Cognitive Levels Work

Blooms Taxonomy
Minnesota has based its cognitive levels on an adapted version of Blooms Taxonomy1, a classification of the levels of intellectual behavior. His taxonomy contains three overlapping domains: cognitive, psychomotor, and affective, as well as six levels of complexity ranging from simple recall to more complex mental levels (see graphic).

Test items are written to measure benchmarks. Item writers then assign a Cognitive Level to identify the cognitive skill level necessary to answer the item correctly. The three categoriesLevels A, B, and Cform an ordered description of the demands an item may make on a student. The distinctions made in item complexity ensure that items will assess the depth of student knowledge at each benchmark.

Level A items assess benchmarks at the lowest cognitive skill levels. These items typically require the student to identify or select an answer that is explicitly evident. For example, Level A items may require a student to recall a fact. Items at Level A consist of the Knowledge level in Blooms Taxonomy.

Level B items require greater cognitive skill. They typically require a student to complete a procedure or explain a phenomenon. For example, Level B items may require identifying similarities and differences. Items at Level B consist of the Understanding* level.
*MDE uses Understanding in Level B instead of Comprehension. The term comprehension is a more global skill necessary at all cognitive levels of reading. Understanding is used to avoid confusion with the cognitive skill comprehension that is necessary for any level of reading.

Level C items require the greatest cognitive skill. These items may require the student to evaluate the effectiveness of a solution or analyze the appropriateness of an argument. For example, Level C items may require a student to analyze and synthesize information. Items at Level C consist of the Application, Analysis, Synthesis, and Evaluation levels in Blooms Taxonomy.

Cognitive Level Examples

The pages that follow illustrate some of the varying complexity demands that math, reading, and science items might make at each level. An item that fits more than one description should be classified in the highest level of complexity demanded for the item. The last page breaks down the percentages of items by cognitive level for each content area. This information is also found in the test specifications for the MCA-IIs.

Bloom, B., Englehart, M. Furst, E., Hill, W., & Krathwohl, D. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York, Toronto: Longmans, Green.

Mathematics

The following items were written to the Benchmark V.C.1 in Grade 5: Students will find the area of a polygon and label the answer with appropriate units. These item examples are variations of an actual item on the MCA-II Grade 5 Mathematics Item Sampler (#39 on page 34). The actual item in the Sampler is a Level C item and was used on a previous test. The Level A and B items were written by MDE assessment specialists to demonstrate how the same concept can be assessed at different cognitive complexities.

Cognitive Level A: MCA-II in Mathematics

The Level A category relies heavily on the recall and recognition of previously-learned concepts and principles. Items typically specify what the student is to do, which is often to carry out some procedure that can be performed mechanically. It is not left to the student to come up with an original method or solution. Skills required to respond correctly to Level A items may include but are not limited to:

Computing a sum, difference, product, or quotient Evaluating a variable expression, given specific values for the variables Recognizing or constructing an equivalent representation Recalling or recognizing a fact, term, or property Retrieving information from a graph, table, or figure Identifying appropriate units or tools for common measurements Performing a single-unit conversion

Item

Grade 5

Benchmark V.C.1
1. What is the area of the rectangle? A. 20 square units B. 22 square units C. 24 square units D. 32 square units*

*Correct answer

Annotation
This question is a Cognitive Level A because: The student recalls that the area of a rectangle is found by counting squares within the rectangle. There are no partial squares. The problem clearly states to find the area, measured in square units, and the distractors are labeled square units. The graphic provides the exact information of what a square unit is in this figure.

Cognitive Level B: MCA-II in Mathematics

Items in the Level B category involve more flexible thinking and choice among alternatives than Level A items. The student is expected to decide what to dousing informal methods of reasoning and problem solving strategiesand to bring together skill and knowledge from various domains. Skills required to respond correctly to Level B items may include but are not limited to:

Solving a problem involving spatial visualization and/or reasoning Selecting and/or using different representations, depending on situation and purpose Retrieving information from a graph, table, or figure and using it to solve a problem Determining a reasonable estimate Extending an algebraic or geometric pattern Comparing figures or statements Representing a situation mathematically in more than one way Formulating a routine problem, given data and conditions

Item

Grade 5

Benchmark V.C.1
1. What is the area of the triangle? A. 4 square units B. 16 square units* C. 18 square units D. 20 square units

*Correct answer

Annotation
This question is a Cognitive Level B because: The student must make a connection between finding lengths and determining area. The answer options are labeled square units, but the graphic only provides information of what 1 unit is in the figure. This is similar to many problems students have seen in class and their textbooks. The problem clearly states to find the area.

Cognitive Level C: MCA-II in Mathematics

Level C items make heavy demands on student thinking. Students must engage in more abstract reasoning, planning, analysis, judgment, and creative thought. These high-complexity items require that the student think in an abstract and sophisticated way. Skills required to respond correctly to Level C items may include but are not limited to:

Solving a non-routine problem (as determined by grade-level appropriateness) Solving a problem in more than one way Describing how different representations can be used for different purposes Generalizing an algebraic or geometric pattern Explaining and justifying a solution to a problem Describing, comparing, and contrasting solution methods Providing a mathematical justification Analyzing similarities and differences between procedures and concepts Formulating an original problem, given from a situation Formulating a mathematical model for a complex situation Analyzing or producing a deductive argument

Item

Grade 5

Benchmark V.C.1
1. What is the area of the shaded figure? A. 4 square units B. 8 square units C. 16 square units* D. 24 square units

*Correct answer

Annotation
This question is a Cognitive Level C because: Students must break the problem into steps while applying what they know about area. They may do this by: decomposing the rectangle to determine first the area of the whole rectangle, then the area of the two triangles, and then the area of the shaded figure; drawing a line of symmetry and making the connection with a whole is equal to the sum of its parts. The problem clearly states to find the area. The graphic provides the information of what 1 square unit is in the figure.

Reading

The following items were written to the Benchmark I.C.6 in Grade 6: Retell significant sequences of events or ideas. These item examples are variations of an actual item on the MCA-II Grade 6 Reading Item Sampler (#3 on page 8) from the passage Marco Polo and the Million Lies. The actual item in the Sampler is a Level B item and was used on a previous test. The Level A and C items were written by MDE assessment specialists to demonstrate how the same concept can be assessed at different cognitive complexities.

Cognitive Level A: MCA-II in Reading

The Level A category requires students to recall, observe, question, or represent basic facts. For Level A items, the student would be expected to demonstrate simple skills or abilities. Level A items require only a basic understanding of the text, often verbatim recall from the text or simple understanding of a single word or phrase. Skills required to respond correctly to Level A items may include but are not limited to:

Identifying the correct meanings of gradeappropriate words Locating details in text Locating details on a graph, chart, or diagram Recognizing the correct order of events in a text Identifying figurative language in a text

Item

Grade 6

Benchmark I.C.6

1. Marco Polo worked for Kublai Khan A. when he was seventeen years old. B. during the years he was in prison. C. for more than sixteen years.* D. for more than three years.

*Correct answer

Annotation
This question is a Cognitive Level A because: The student needs to read for specific details to locate the fact in paragraph 3: Marco spent more than 16 years in Khans service. The student needs to identify and recall the stated sequence of who Marco Polo and Kublai Khan were, their roles, how they were connected, and their subsequent roles. The item does not expect the student to assimilate a major section of the reading, but rather the answer is found in the passage nearly word for word.

Cognitive Level B: MCA-II in Reading

Level B items may require a two-step process, comprehension and subsequent processing of text. Students are expected to make simple inferences within the text and may encounter items that include words such as summarize, infer, classify, gather, organize, compare, and display. Depending on the objective of a particular Level B item, students may also be required to explain, describe, or interpret. Skills required to respond correctly to Level B items may include but are not limited to:

Using context clues to identify the meanings of unfamiliar words Determining how details support the main idea Interpreting the information in graphs, charts, and diagrams Identifying cause and effect relationships Determining an authors main purpose or point of view Identifying similarities and differences Demonstrating an understanding of plot development Recognizing elements of plot Recognizing patterns of organization Summarizing the major points of a text Comparing word meanings

Item

Grade 6

Benchmark I.C.6

1. Before he met Kublai Khan, Marco Polo spent several years A. fighting in a war. B. traveling to China.* C. telling stories in prison. D. writing a book of fables.

*Correct answer

Annotation
This question is a Cognitive Level B because: The text must be reviewed to determine when Marco Polo met Kublai Khan. The item requires the student to recognize the sequence of the events to determine how Marco Polo spent his time prior to meeting Kublai Khan. The reader has to draw a conclusion to determine that Marco Polo traveled to China in the years prior to meeting Kublai Khan.

Cognitive Level C: MCA-II in Reading

Level C items make heavy demands on student thinking. Students may be encouraged to explain, generalize, or make multiple connections. Level C items require several steps involving abstract reasoning and planning. Students must be able to support their thinking. Items may involve identifying theme and implicit main idea and making complex inferences within or across text. Students may also be asked to take information from at least one portion of the text and apply this information to a new task. They may be asked to perform complex analyses of the connections among texts. Skills required to respond correctly to Level C items may include but are not limited to:

Analyzing the use of figurative language in a text Showing how graphs, charts, and diagrams contribute to a text Determining an authors purpose and/or point of view and describing how it affects the text Evaluating strong vs. weak arguments in a text Analyzing similarities and differences Describing and analyzing the characteristics of various types of literature Describing and illustrating how common themes are found across texts Analyzing cause and effect relationships

Item

Grade 6

Benchmark I.C.6

1. Why was it significant that Marco Polo shared his observations of China with the people of Europe? A. Marco Polo needed to clear his name which had been tainted by Marco Milione. B. Marco Polo predicted people would be doing daily chores in silken clothes. C. Marco Polo defended Rustichellos experience as a writer and historian. D. Marco Polo predicted, through Rustichellos book, developments still used today.* *Correct answer

Annotation
This question is a Cognitive Level C because: The entire passage must be read while employing multiple criteria. The log of Marco Polos travels to China and back to Europe, including his stories, has to be understood. The historical and sequential importance of Marco Polos travels and his acquired knowledge are necessary to answer the item. The student has to analyze the word significant in the stem of the item and how to employ that words meaning in determining the answer. This process involves more than a mere literal reading of the text. Analysis of the different experiences of Marco Polo is necessary to determine their significance and possible future importance. Comprehension strategies of sequencing are necessary. Recognition of the passages historical and sequential frameworks is necessary. Analyzing how this text is different from other texts is necessary to determine cause/effect, fact/opinion, or conflict/resolution. 10

Science

The following items were written to Benchmark II.A.3 in Grade 6: The student will know that the mass of a substance remains constant whether it is together, in parts, or in a different state. These item examples are sample items written by MDE assessment specialists to demonstrate how the same concept can be assessed at different cognitive level complexities. The items are examples of the kinds of items that may appear on the Grade 8 MCA-II in science.

11

Cognitive Level A: MCA-II in Science

Items in the Level A category rely heavily on the recall and recognition of previously-learned concepts and principles. Items typically specify what the student is to do, which is often to carry out some procedure that can be performed mechanically. It is not left to the student to come up with an original method or solution. Skills required to respond correctly to Level A items may include but are not limited to:

Identifying a common example or recognizing a concept Retrieving information from a chart, table, diagram, or graph Recognizing a standard scientific representation of a simple phenomenon Calculating or completing a familiar single step procedure or equation using a reference sheet

Item

Grade 6

Benchmark II.A.3

1. Which property stays the same when a liquid freezes? A. Mass* B. Volume C. Temperature D. State of matter

*Correct answer

Annotation
This question is a Cognitive Level A because: The item requires the student to recall that freezing is a change in the state of matter and that the mass of a substance remains constant in a different state. The item does not require the student to transform this recalled information. The item does not require the student to apply this information.

12

Cognitive Level B: MCA-II in Science

Items in the Level B category involve more flexible thinking and choice among alternatives than Level A items. They require a response that goes beyond the habitual, is not specified, and ordinarily has more than a single step or thought process. The student is expected to decide what to dousing informal methods of reasoning and problem-solving strategiesand to bring together skill and knowledge from various domains. Skills required to respond correctly to Level B items may include but are not limited to:

Applying or inferring relationships among facts, terms, properties, or variables Describing examples and non-examples of scientific processes or concepts Predicting or determining the logical next step or outcome Comparing or contrasting structures or functions of different organisms or systems Choosing the appropriate formula or equation to solve a problem and then solving it Applying and using concepts from a standard scientific model or theory

Item

Grade 6

Benchmark II.A.3

1. Which change will affect the mass of a sample of water? A. Freezing the water B. Dissolving salt in the water* C. Pouring the sample into another container D. Dividing the sample into two different containers *Correct answer

Annotation
This question is a Cognitive Level B because: The item first requires the student to recall that mass is a property that remains constant whether a sample of a substance is together, in parts, or in a different state. The item then requires the student to transform this recalled knowledge in order to identify that dissolving salt is the only change that does not keep the mass the same.

13

Cognitive Level C: MCA-II in Science

Level C items make heavy demands on student thinking. Students must engage in more abstract reasoning, planning, analysis, judgment, and creative thought. The items require that the student think in an abstract and sophisticated way often involving multiple steps. Skills required to respond correctly to Level C items may include but are not limited to:

Constructing models for research Generalizing or drawing conclusions Designing an experiment, given data and conditions Explaining or solving a problem in more than one way Providing a justification for steps in a solution or process Analyzing an experiment to identify a flaw and propose a method for correcting it Interpreting, explaining, or solving a problem involving spatial relationships; or predicting a long-term effect, outcome, or result of a change in a system

Item

Grade 6

Benchmark II.A.3

1. Two ice cubes with a mass of 5.0 grams each are melting. What will be the mass of the water formed from these ice cubes? A. 8.5 grams B. 9.0 grams C. 10.0 grams* D. 11.0 grams

*Correct answer

Annotation
This question is a Cognitive Level C because: The item first requires the student to recall that the mass of a substance remains constant whether it is together, in parts, or in a different state. The item then requires students to apply this principle to the mass of the sample of water.

14

Percentages of Items
This page breaks down the percentages of items by cognitive level for each content area. This information is also found in the test specifications of the MCA-IIs.

Cognitive Level Distribution of Items in Mathematics Grades 3-5 6-8 11 Level A 35 45% 35 45% 30 40% Level B 35 45% 35 45% 30 40% Level C 10 20% 10 20% 15 25%

Cognitive Level Distribution of Items in Reading Grades 3-5 6-8 10 Level A 35 45% 30 40% 25 35% Level B 35 45% 30 40% 25 35% Level C 10 20% 15 25% 25 35%

Cognitive Level Distribution of Items in Science Grades 5 8 1012 Level A 35% 25% 20% Level B 40% 45% 45% Level C 25% 30% 35%

For more information related to the nature of the MCA-IIs, please visit the Research and Assessment Web site for the MCA-IIs (http://education.state.mn.us/mde/Accountability_Programs/ Assessment_and_Testing/Assessments/MCA_II/). This site has a link to information about the MCA-II test specifications that describe the skills, or benchmarks, eligible for testing and the nature of the test items that measure them. It also has a link to the Item Samplers which are examples of the kinds of items that appear on the tests. Both of these links are particularly valuable to teachers and parents.

Thank you to the Florida Department of Education for their contributions to this brochure.

15