Two days ago I commented on a questionable item in the 10th grade TAKS mathematics test, for which scores were revised. The associated TEA press release refers also to a controversy over some science test items, and states that, upon review, these items were found to be correct. It is fascinating to see the items and to see how they are judged to be correct. The TEA (Texas Education Authority) put out an Additional Information Regarding Released Science Items for the spring 2003 testing cycle. Four controversial items are discussed.
Grade 5 Science, Item 13. Item 13 asked students which two planets are closest to Earth. Among the possible answers: Mercury and Venus, and Mars and Venus. The correct answer varies over time, and the question is plainly wrong or crazy. To add insult to injury: the intended answer was Mars and Venus, but on the day the test was given the correct answer was Mercury and Venus. Nevertheless, the TEA insists that for the purpose of the 5th grade test the question had only one correct answer - to wit, the wrong answer.
Grade 10 Science, Item 50. Item 50 looks crazy to me - they seem to be testing in a most convoluted way that the student knows that the element symbol K stands for Potassium. The TEA discussion indicates that the item is factually wrong to boot, but they insist that it is valid just the same.
Grade 11 Science, Items 11 and 45. Question 11 asks for the force exerted by a jumping frog on a leaf. The force has two components: one due to the weight of the frog and the other due to its acceleration. These are to be added vectorially, but the direction of the jump is not given. The TEA insists that therefore the correct treatment of the question must ignore the weight of the frog. Obviously the question is wrong and the TEA is wrong to insist that it is correct. Question 45 concerns a hypothetical situation in which a force is exerted on an object but no work is done. The question asks what can be concluded, and the intended answer is that the object is and remains at rest. This is wrong; the force may be perpendicular to the direction of motion. The TEA insists in effect that students don't know that, and that therefore the TEA's intended answer is, for the purpose of the test, the unambiguously correct answer.
The TEA has a bit of a quality control problem, obviously. In connection with the earlier 10th grade Math test problem Kimberly Swygert asked if the pre-testing might not have found the error. The same question could be asked for these science test items, but I think that it is too much to ask of the psychometric process that it correct for blunders of this kind.
I suspect that for many patently wrong questions students will nevertheless do what the TEA expects of them. The pernicious effect of the bad test items is indirect. It creates among the students and the public an impression (a correct impression) that the TEA doesn't have its house in order; that questions can't be read to mean what they mean; and that one should always be prepared to second-guess the clear meaning of a question.
A closing remark: the New York State Regents testing division has similar quality control problems. I remind the reader of the earlier discussion about the June 2003 Regents Math A exam, and my related Critique of the New York State Regents Mathematics A Exam
Man. Thats the kind of test that punishes students for being intelligent...
I remember in my math analysis class, my teacher warning all the sophomores (who had the WASL coming up) to be very careful because by the time they took the test, they'd be very far conceptually in math, and would struggle with the math portion of the test BECAUSE OF IT... these students are two years ahead of the curriculum... they're just "too smart"?
Posted by Stine at October 14, 2003 09:18 PMSome years ago (quite a few years ago),I failed to convince my intelligent boss that yhere would be a considerable force exerted on the wooden transmission line structure I was designing.He demurred noting that the weight of the wire was only a few pounds.I tried to explain that a large horizontal force was generated because of necessity the line could not sag much but without much success,
A technican hearing the conversion supported me by noting that when he was a telephone lineman in the army,as the telephone wires were made taut the poles would creak.Hearing this my boss agreed with me.Why the failure of my intelligent boss to see the "obvious"?
To begin with he was a graduate electrical engineer.Thus structural engineering was not his forte.
However I was an electrical engineer too so why didnt I fall into the same trap?
In college we did a simple but very informative experiment where the equilibrium of forces was demonstrated by balancing 3 hanging weights.
We were required to calculate the angles the lines of action made with the weights and compare them with the measured angles.
I was given a arrangement of weights and angles that resulted in two nearly equal weights acting almost opposed to each other with a third almost at right angles to the other 2.
I got very poor results naturally !
I did myself proud and did an analysis showing that very small changles in the angles would make very large changes in the forces.
My instructor gave me I think the lowest possible passing grade commenting that my results wer not accurate enough.
To "see" the components of a vector is not I believe intuitive.A very wrong answer in an exam question that reinforces that misperception-that fails to see - nay- fails to understand that a force perpendicular to the direction of motion does no work- reinforces the students misperception.
RE: Question 50. You're right; this question is ridiculous. They are apparently trying to test for some basic logic and knowledge of the periodic table and polyatomic ions. Unfortunately, at least some of their solubility rules are wrong. A student who had taken even basic high-school level chemistry would be able to come up with several exceptions to rules 2 and 3.
Posted by Daryl Cobranchi at August 11, 2003 03:02 PM