I read and re-read the letter from members of the US National Commission on Mathematics Instruction to Education Secretary Arne

Duncan, which is found here. I think the letter is excellent, and I hope it has an impact on Secretary Duncan and national policy. However, we humans have a tendency to explain away and dismiss facts that are not consistent with our beliefs, especially when we have strong feelings about the issue and are already committed to action based on those beliefs. Chris Mooney has a good article on this tendency called The Science of Why We Dont Believe Science. Given this tendency, can we expect much, if any, response from Duncan. I would say that, while this letter is not written in a hostile tone, it does take a frontal attack approach to change, directly promoting principles that contradict current beliefs and practices. Change would require a major rejection of current beliefs in which there has already been much emotional, financial, and reputational investment. I think it is important to argue the case made in this letter, but I would like to suggest that we can simultaneously take another approacha Trojan Horse approach which does not oppose the current system but seeks to improve it from within its current design, yet ends up undermining its poor practices. This approach targets an area where we are ALL in agreement but in which the current system is critically flawed. By concentrating on this flaw and making it obvious, we force change. Ive explained what I think the critical flaw is in this guest post on Sue VanHattums blog. (If you read my article, read the comments as well. I didnt discuss solutions to the Sarah problem in my post because I thought it was important to leave the problem open. The extent that readers have to work hard and struggle to find the solutions is the extent that my point is made, because the math is very basic and we should easily see both conceptual solutions. Theres closure to that example in the comments.) In essence, the flaw is how we think about and measure understanding. Everyone wants our children to understand concepts. No one is saying that computational fluency is sufficient without conceptual understanding. But how we teach and evaluate teaching, and our use of tests, is not always consistent with real understanding. When students compute the answer to problems we assume they also understand the concepts involved in the solution. Too often this is not a valid assumption. We need to directly test for conceptual understanding instead of inferring it when students can compute the correct answer, or worse, when they use test-taking tricks to select the correct answer in multiple choice tests. For example, there is more than one way to use the multiple choice answers given with the problem to select the correct answer without actually working through a solution. An example of a multiple choice test that does assess conceptual understanding is the Force Concept Inventory. A paragraph from the developers of the inventory:

The Force Concept Inventory is not "just another physics test." It assesses a students overall grasp of the Newtonian concept of force. Without this concept the rest of mechanics is useless, if not meaningless. It should therefore be disturbing rather than comforting that students with only moderate scores on the Inventory may score well on conventional tests and get good grades in physics. Of course, experienced teachers have learned to avoid problems that are "too hard" for the students. That includes most qualitative problems that seem so simple until student answers are examined. Students do better on quantitative problems where the answer is a number obtained by substitution into an appropriate equation, and even on harder problems that require some algebraic manipulation. So should we not be satisfied that they have developed quantitative skills? After all, physics is a quantitative science! Or do we have here a selection process that directs teachers to problems that students can answer with a minimum of understanding?
The Force Concept Inventory is a physics test. We need to develop math concept inventories. From Wikipedia:
A concept inventory is a criterion-referenced test designed to evaluate whether a student has an accurate working knowledge of a specific set of concepts. To ensure interpretability, it is common to have multiple items that address a single idea. Typically, concept inventories are organized as multiple-choice tests in order to ensure that they are scored in a reproducible manner, a feature that also facilitates administration in large classes. Unlike a typical, teacher-made multiple-choice test, questions and response choices on concept inventories are the subject of extensive research. The aims of the research include ascertaining (a) the range of what individuals think a particular question is asking and (b) the most common responses to the questions. Concept inventories are evaluated to ensure test reliability and validity. In its final form, each question includes one correct answer and several distractors. The distractors are incorrect answers that are usually (but not always) based on students' commonly held misconceptions.

Developing math concept inventories is a large project, but worthwhile. I think that until we make a clear distinction in our teaching and assessments between what Skemp calls Instrumental Understanding and Relational Understanding, we will continue to have our students getting the right answers despite not understanding the relevant math concepts. The ability of poor practices to nevertheless help students get right answers is what supports those practices. To many people, getting the right answer is proof of understanding. Concept inventories can dispel those people of this notion and reveal the inadequacy of poor practices. Then those who believed in those practices will be ready for significant change. Even individual teachers can include qualitative, conceptual questions in their own tests. Doing so may help both teachers and students pay more attention to concepts. But what we need in order to change the entire system is official, valid measures of conceptual understanding that expose poor practices for what they are, and shift the focus away from simply computing the right answer to really understanding the material. The first step in achieving this goal is to spread this idea as widely as possible, including through the US National Commission on Mathematics Instruction.