The New York Times has a misguided report by David Herszenhorn on NYC mayor Bloomberg's and schools chancellor Klein's poor marketing of their reforms.
When Mr. Bloomberg laid out the bulk of his education plans in a speech in Harlem in January, his proposals were received with general enthusiasm, even winning the initial support of the teachers' union president, Randi Weingarten.
But in the weeks after the mayor's speech, the administration failed to build the momentum, officials said, and instead became embroiled in an arcane debate over whether the proposed literacy curriculum had a strong enough phonics component.
Sadly, it seems entirely possible that mayor Bloomberg and chancellor Klein view the debate over how to teach reading as just arcane and petty. Their defence of the curricular mandates that chancellor Klein imposed on the New York City schools is never based on the substance of the specific choices, but only on the claimed need to have a unified curriculum throughout the city. A Freedom Of Information Law request done for New York City HOLD showed that there is no documentation, not even for the Department's internal reference, of the rationale for the specific choices of textbooks for reading and mathematics.
Readers may recall the controversy over the difficulty of the June 2003 New York State Regents Math A exam. The results of the exam were tossed for juniors and seniors, and a panel was appointed to study what went wrong. For reference, here are links to Commissioner Mills's earlier press release and the charge to the Math A panel. Also for reference, my critique of the New York State Regents Math A exam.
The Math A Panel has now produced an interim report, and it is receiving plenty of press attention. (Go to Google News and do a search on 'regents "math a"'.) The best summary that I've seen is that of Karen Arenson in the New York Times.
The panel's interim report deals with only a very limited part of the charge, and deals with it in a disappointingly limited way. The panel clearly thought it was important to have a recommendation out before the start of the school year about a rescaling of the test. I am surprised that they only found the time to compare the June 2003 and the June 2002 instances; in the 6 weeks that they've worked they really might have had a serious look at, say, the past 6 instances of the exam, and this both in a qualitative and a psychometric way. Who knows, maybe the June 2002 exam was exceptionally easy.
In fact, though, the conclusions of the panel regarding the difficulty of the June 2003 instance match all the informed speculations that I've seen, including my own speculations: Parts 1 and 2 of the exam were in line with previous instances, and parts 3 and 4 were more difficult. For my critique I looked at August 2002, January 2003, and June 2003; and found June 2003 the hardest and January 2003 the easiest.
The interim report does not specifically criticise any officials or any actions, but I draw from it the conclusion that inexcusable errors were made in the development of this June, 2003, instance of the exam. In my earlier commentary I quoted an article by David Hoff in Education Week in which he quoted deputy commissioner James Kadamus as saying that the June, 2003, exam had more problem-solving questions than previous exams, because the state is gradually raising its expectations. I wrote then that this is a remarkable statement, because all previous reports indicated that the added difficulty of the June exam was unintended and had taken the Department entirely by surprise.
Now here is Karen Arenson, writing on the basis of the interim report of the Math A panel:
Based on field tests before the actual test was administered, the Education Department expected the average score on the June test to be 46. The expected average for the test given a year earlier was 51 slightly higher, but still below the score needed to pass, which is 65 for students who entered ninth grade in 2001 or later, and 55 for everyone else.
Did commissioner Mills know that the average scaled score of the June, 2003, exam was expected to be 5 points lower than that of June, 2002? (Arenson is mistaken, of course, to describe 51 as "slightly higher" than 46; the difference is large.) Public indications are that Mills did not know this.
I am still surprised that the error of the added difficulty was made in such a blatant way. For myself I had been speculating that a subtle error would have been made: the department might have used for its psychometric evaluation of the difficulty of the test a rather different population of students than the population that really matters. They might have had a test population with lots of bright 9th and 10th graders, and perhaps for that group the difficulty of the June 2003 exam was in line with earlier instances, while for the struggling seniors the added "problem solving" (i.e., aptitude oriented) focus of the exam would have posed more severe problems. But apparently the department did not make a subtle error; they were just completely wrong and out of control.
New York City schools chancellor Joel Klein has been under some heavy and well deserved fire recently for his curricular policies. This blog entry is based on articles and opinion pieces by James Traub, Sol Stern and Andrew Wolf; and on the Web pages of New York City HOLD.
On August 2 the New York Times educational supplement offered New York's New Approach, by James Traub. (The original article has gone off-line, and the link is to a copy.) Traub focusses on the literacy part of New York's "Children First" initiative.
[...] All New York elementary and middle-school students will have lengthy "literacy blocks" each day to focus on reading as well as writing skills. Teachers will read books aloud, engage in "shared reading" with the whole class, "guided reading" with smaller groups and "independent reading" from classroom libraries whose books will be carefully calibrated by skill level.
[...] Here was a form of teaching that built on the child's innate knowledge and love of learning, required virtually no rote instruction and permitted children to acquire information and understanding as a painless byproduct of pleasurable activities. It sounded delightful. But would it be effective?
Traub presents Klein as perhaps an unwitting captive of the city's liberal consensus on pedagogical issues, and presents the deputy chancellor for teaching and learning, Diana Lam, as the real force behind the progressive pedagogy. Traub himself has no sympathy for the direction chosen by chancellor Klein:
Every new chancellor in recent years has come into office with a message of salvation for the schools. Once it was "school-based management," then it was "curriculum frameworks," and then data-driven instruction. None of it really mattered in the end, because chancellors couldn't impose their will on the system. Now, at long last, they can. Mayor Bloomberg and Mr. Klein have the power to reshape New York City schools.
But they have imposed a curriculum that scants content knowledge for personal experience and direct instruction for self-directed learning. With almost half of the city's fourth graders and two-thirds of its eighth graders reading below grade level, is this the direction they should go?
Traub's piece mentiones an earlier article by Sol Stern of the Manhattan Institute: Bloomberg and Klein Rush In (City Journal, Spring 2003). There, Stern wrote:
Unless Bloomberg and his handpicked schools chancellor, Joel Klein, admit to some monumental blunders, discredited progressive methods for the teaching of the three Rs such as "whole language," "writing process," and "fuzzy math" will soon be enforced in every single classroom in 1,000 New York City schools. This is a disaster in the making, not least because the children in the targeted schools are mainly poor and minority - the very population historically most damaged by such methods.
Mr. Stern is at it again in the online pages of City Journal with Mayor Bloomberg's Diana Lam Problem. (The article also appeared as an opinion column in the New York Post: Lam Excuses.) Stern first recalls the appointment - later put on hold - of Diana Lam's husband to a $100,000 per year job as regional instructional supervisor. He then addresses a new issue by which Ms. Lam has given the impression of being ethically challenged. With reference to Stern's earlier conclusion that Diana Lam is addicted to discredited "whole language" and "constructivist" methods for teaching reading and writing Stern writes:
Lam responded to these criticisms in a manner that raised new questions about her competence and integrity. In a Daily News op-ed, she trumpeted the results of a recent U.S. Department of Education study comparing the reading and writing scores of New York City's 4th-graders with those of five other urban districts: Atlanta, Chicago, Houston, Los Angeles and Washington.
In those tests, the city's 4th-graders ranked at the top of the six participating districts in writing and a close second to Houston in reading. According to Lam, "the results of this assessment show our pedagogical approach is sound."
But Lam neglected to inform her readers that the tests represented a random selection of the city's 4th-graders from January through March 2002. At that time, Lam was running the Providence, R.I., school system, Joel Klein was an executive with the Bertelsman publishing company, and newly elected Mayor Bloomberg hadn't yet convinced the state Legislature to give him control of the city's schools.
[...] I leave it to others to decide whether Lam's misrepresentations about those 4th-grade tests result from a blunder or from something worse. In either case, Mayor Bloomberg and Schools Chancellor Joel Klein now have a credibility problem on their hands.
In addition to the pieces by James Traub and Sol Stern there was a scathing op-ed by Andrew Wolf in the New York Sun. (No NYC journalist has been as consistently strong on the Bloomberg and Klein educational fiasco as Andy Wolf, as witness this collection of previous columns.)
In a remarkably intellectually dishonest opinion piece that ran last week in the Daily News, Ms. Lam had the chutzpah to declare that New York's "reading plan is working." She bases her claim on the results of the National Assessment of Educational Progress test, a voluntary exam given to compare the progress of students in the nation's cities. This test was administered to a sampling of fourth-grade classes more than six months before Mr. Klein and Ms. Lam took over the old Board of Education. New York City and Houston were shown to have the most effective programs among the six largest urban centers.
Now unless Mr. Klein was lying on January 21, when he stated that the city has been "using something along the lines of 30 different reading programs," the results of the NAEP test reflect that diversity. This is certainly no more an endorsement of Ms.Lam's controversial program than it is of any of the other 29 programs then in use. And what if Ms. Lam, as many of us feel, has chosen the wrong one of the 30 alternatives? She concedes that Houston did just as well, but with a "scripted" reading program that she has specifically excluded. But many of our New York City schools used such programs. How much of New York City's success can be attributed to those schools?
The cited articles of James Traub, Sol Stern, and Andrew Wolf all address primarily the reading component of chancellor Klein's Children First initiative. For critical perspectives on the mathematics component, please see the New York City HOLD Web pages, and see also my overview page Chancellor Joel Klein's "Children First" New Standard Curriculum for NYC Public Schools.
A further issue that has not received adequate attention in the press reporting is the secrecy of Children First. As a result of Freedom Of Information Law Requests we know that the primary Children First working groups operated without formal charge and did not produce reports. In a remarkable show of contempt for integrity of process and for careful policy chancellor Klein has arranged that there is no documentation, not even for the Department of Education's internal purposes, of the rationale behind his and Ms. Lam's choices for the literacy and mathematics curricula.
School's Out, by Carl Campanile (New York Post, July 28, 2003).
The city is opening a full-fledged high school for gay, lesbian, bisexual and transgender students - the first of its kind in the nation, The Post has learned.
Operating for two decades as a small alternative program with just two classrooms, the new Harvey Milk HS officially opens as a stand-alone public school with 100 students in September.
[...] The Hetrick-Martin Institute - the gay-rights youth-advocacy group that manages and helps finance the school in conjunction with the Department of Education - has hired the school's first principal.
[...] [Principal] Salzman said Harvey Milk will be an academically rigorous school that follows Schools Chancellor Joel Klein's mandatory English and math programs. It will also specialize in computer technology, arts and a culinary program.
The New York Post claims the story as an exclusive, and my first reaction was to wonder if it would hold up. However, there does exist a Hetrick-Martin Institute in New York City and it is indeed home of the Harvey Milk School, which is at present offering some path to an alternative high school diploma. They are going mainstream, then. Meanwhile also New York Newsday has picked up the story and they quote Mayor Bloomberg as to why it is a good idea.
The New York Times and other NYC newspapers report the retirement of Shelley Harwayne. As superintendent of New York City's community school district 2 Shelley Harwayne was among the most visible proponents nationwide of the educational reform movement associated with whole language reading instruction and constructivist mathematics teaching. In the new governance structure that took effect just the start of this month Shelley Harwayne held the position of superintendent for Region 9: the largest region by number of schools in the system and encompassing most of Manhattan including the old CSD2. Ms. Harwayne is retiring to deal with health and family issues.
Shelley Harwayne has written at least 6 books and is a frequent speaker at national events; for example, keynote speaker at the NCTE Whole Language Umbrella Conference in Nashville (2000) and at the National Conference of the Reading Recovery Council of North America (2002), and giving the opening talk at the NCTE Whole Language Umbrella Conference in Bethesda (2002). Before becoming superintendent of community school district 2 Ms. Harwayne was the founding principal of the Manhattan New School. One of her books, Going Public: Priorities and Practice at the Manhattan New School (Heinemann, 1999) is based on that experience, and offers insight into the educational philosophy that guided District 2 and that has been influential throughout the NYC school system.
My own interest is mathematics and science education. Last year I read Going Public with that perspective and used it for a Web article, Shelley Harwayne and Mathematics. The present contribution is based on that longer article.
Ms. Harwayne's book has one chapter where one may look for academic ambitions of the school: Chapter 6, Talking Curriculum and Assessment. The issue of mathematics education covers about half a page in that chapter, and there is nothing at all about science education. In the half page about mathematics Shelley Harwayne describes how she marvels at what her children are able to do, such as renaming numbers, seeing patterns in hundreds charts, and performing great amounts of mental math. With little attention to algorithms her students understand how knowing that 6 x 7 = 42 helps you to know what 60 x 70 is, what 12 x 7 is, what 3 x 7 is, and so on. Observing the teaching of mathematics she realizes how little she knows and how much there is to learn.
Ms. Harwayne's limitations in mathematics did not prevent the CSD2 superintendency from taking a very active and damaging interest in mathematics instruction, removing curricular choices from the schools and teachers and imposing a sequence of reform mathematics curricula throughout the District that are roundly rejected by mathematics professionals. These curricula include TERC: Investigations in Number, Data, and Space in grade school, Connected Mathematics Program (CMP) in middle school, and Mathematics: Modelling Our World (COMAP) in high school.
The educational reform in district 2 gave rise to an opposition, and especially to New York City HOLD: an advocacy organization for parents, educators, mathematicians and others focussed on improving the quality of mathematics education in New York City schools. In spite of the efforts of NYC HOLD and others, at present the District 2 philosophy holds sway throughout the New York City school system.
Ronald Brownstein writes a Washington Outlook column for the Los Angeles Times. The column of July 21, 2003, Failing Schools Need Courses in Readin', Writin' and Accountability (also here in case the LAT link disappears) made some laudatory references to the performance of NYC schools chancellor Klein in his focus on accountability. I think it useful to provide a counterpoint.
Ronald Brownstein wrote:
Joel I. Klein, the accomplished attorney who was Mayor Michael R. Bloomberg's unconventional choice as schools chancellor last year, understands he can effectively educate the 1.1 million students in his care only if he shatters the cozy arrangements that have kept the New York City school system focused more on providing jobs for adults than on opportunities for kids. After 11 months on the job, Klein has the scars to prove his commitment to that cause. [...]
In conversation, it's apparent his greatest challenge is to impose more accountability for results on principals, teachers and the rest of the school system's 150,000 employees. "In public education," he says in a measured understatement befitting his days as a federal prosecutor and an assistant attorney general under Clinton, "the normal merit approach to service is very limited."
It's achingly ironic that Klein's headquarters is now in the 19th-century courthouse built near City Hall by legendary political boss William Tweed. Tweed's power rested on a patronage system that guaranteed jobs for even his most unqualified supporters. Klein presides over a $12-billion system whose work rules and union contracts make it dauntingly difficult for him to fire even the most incompetent. Last year, Klein initially hoped to remove 50 principals in woefully under-performing schools; he was only able to dismiss one. Just 132 of the system's 78,000 teachers last year were removed for inadequate performance.
The article goes on to argue that, besides accountability, also more resources, and specifically federal funds, are required in order to help schools make the grade.
Brownstein is correct to point out that so far there is little to show for chancellor Klein's focus on accountability. Where our new chancellor has made an impact is in the choice of a system-wide mandated curriculum for reading and mathematics, and here the result is entirely negative. In a secretive Children First process a K-5 mathematics curriculum, Everyday Mathematics, was selected that was twice rejected in the California textbook adoptions process. For reading the chancellor and his deputy decided upon "instruction based on classroom libraries" (what may safely be understood as "do as you like whole language") supplemented by a program that has phonics in the name but that was severely criticized by reading researchers; later a further supplement to the supplement was identified. Recently the chancellor also decided to preserve New York City's failed bilingual education program. There will be a relentless focus system-wide on the subjects of reading and mathematics, with no apparent concern for science, history, or the arts. Finally, chancellor Klein's personnel decisions to-date, including his choice of deputy chancellor for teaching and learning and his choice of which of the 32 local district superintendents to promote to one of the 10 newly created regional superintendent positions, give not much hope for his future focus on accountability.
For ongoing commentary on the state of mathematics education in New York City, please visit New York City HOLD.
New York State Education Commissioner Richard Mills is under much pressure because of the high failure rate on the recent (June 17) New York Regents Math A exam. The latest and earlier instances of the exam and the associated scoring keys and conversion tables are posted on the Regents Examinations Web site, under the link to Mathematics A. Procedural information related to the exam is posted at the State Assessment site under High School General Information. I know of two reviews of the exam on the Web. There is my own Critique of the New York State Regents Mathematics A Exam, and there is an Analysis of the June, 2003, Administration of Physics and Math A Regents done by the New York State Council of School Supervisors (NYSCOSS).
In connection with the Math A flack the director of the testing division at the NYS Education Department was reassigned and chose to resign, but this is not presented as a cure for any problem. The State Assembly and Senate will hold hearings, according to a report in the Rochester Democrat and Chronicle:
[State Assemblyman Steven Sanders, D-Manhattan], who chairs the Assembly education committee, and Sen. Stephen M. Saland, R-Poughkeepsie, who chairs the Senate education committee, plan to hold public hearings for people to express their feelings about high-stakes testing in light of an estimated 37 percent passing rate on the June 17 Math A Regents exam. Those hearing dates have not yet been set, but Sanders said Rochester, Albany and New York City might be host cities.
Of course there are plenty of calls on editorial pages for the State to abandon its plan to require students to pass five Regents exams for graduation starting in 2004. However, according to an article in the Albany Times Union, Voided math test said to reveal systemic ills, the Regents and Commissioner Mills remain supportive of that plan.
Asked whether he and other Regents still stood behind Mills, board member Saul Cohen responded, "Sure," but added, "That doesn't mean we can't press him to do certain things as we did. We pressed him to nullify the results of the Math A, and I'm continuing to press him to re-examine the physics exam. But that doesn't mean we're not backing him."
A somewhat different take on the exam trouble is found in an article by Karen Arenson in the New York Times, Math Failures Are Raising Concerns About Curriculum.
But some explanations [of the high failure rate] touch on deeper issues, including whether the Math A curriculum is too broad, how much harder it is for students to solve problems than to manipulate equations, and whether unqualified teachers are even less likely to succeed in preparing students than they were with the old math curriculum.
[...] The shift from rote learning to a greater emphasis on mastery of concepts is welcomed by some college professors in math and science, who have been trying to accomplish the same shift. They say that although mastery of some facts is critical, students who focus on memorization may do well in a course but remember little of it six months later.
[...] Some educators say teaching students to be problem solvers takes more skill on the part of teachers, a challenge when there is a shortage of qualified math teachers.
"Teachers are not really prepared to prepare kids for this test properly," said Alfred S. Posamentier, dean of the School of Education, City College of New York, and the author of books on problem solving. "There is very little training for teachers in problem solving; it's assumed they will get it along the way."
My take on it is different. The low passing rate is a complicated affair in any case, and it isn't entirely clear from the data to what extent the exam was really more difficult than earlier instances. Students can take the exam three times per year over multiple years, in August, January, and June, and the character of the test taking population may vary greatly between the months. Many of last year's seniors may still have graduated on the basis of the easier Mathematics I exam. It is surprising that Commissioner Mills does not have the data to say anything more authoritative about the relative difficulty of the latest exam.
The main thing that can be learned from the low passing rate is that, in many cases, New York State high schools are failing to make up for the failures of elementary and middle school education. An eighth grader in a high performing country, say Singapore, or in a U.S. state that has high level content standards, say California, would be well placed to pass this exam. I find ony two kinds of questions that would probably be unfamiliar to such a student. One are the counting questions that require students to know something about permutations and combinations. The other are the very basic trigonometry questions: students must know the ratios in a right triangle that correspond to the sine, cosine, and tangent.
It does appear to me that the June, 2003, instance of the exam had a somewhat more difficult and less "standard" flavor than earlier instances, and in the open response section the June, 2003, exam tilts a bit more towards a test of aptitude rather than a test of school learning, relative to the previous two instances of the exam. This is not to say, however, that all the questions in August 2002 and January 2003 were of a standard and predictable form, and it is not to say that the June 2003 exam is plainly a test of aptitude and the earlier ones plainly a test of school learning.
The unintended shift in the character of the exam should be seen as a failure of the Regents testing division. In addition there are many mathematical flaws in the questions and dubious points in the scoring keys, as described in more detail in the Critique of the New York State Regents Mathematics A Exam and the Analysis of the June, 2003, Administration of Physics and Math A Regents, both mentioned earlier.
The Regents Math A exam that was given on June 17, 2003, has received much negative press attention. For example: New York Daily News, Test Mess Threatens Diplomas. New York Newsday, Math Test Too Tough?. New York Post, Testy Teachers Blast 'Too Hard' Math Exam. New York Times, This Year's Math Regents Exam Is Too Difficult, Educators Say. Rochester Democrat and Chronicle, Huge Numbers Fail Math Test. Buffalo News, Many Seniors Fail Crucial Test. (The links will disappear, but the titles are clear enough.)
I don't know that there is any serious analysis of the exam to be found yet, so the following brief comments may be of interest.
The June, 2003, Math A exam is not yet posted on the NYSED Web site. The previous instance of the exam was in January, 2003, and that one is posted; follow the link to the Math A exams here. I have a FAX copy of the recent Math A exam, but not of the scoring rubrics.
The format of the test is identical between January and June. There are 20 multiple choice questions worth 2 points each, and 15 open response questions, 5 at 2 points each, 5 at 3 points each, and 5 at 4 points each, for a maximum score of 85.
Question 14 on the June exam is plainly faulty, and the scoring rubric has already been changed to allow two answers. The question asks: "If the expression 3-4^2+6/2 is evaluated, what would be done last?" Could be either addition or subtraction, but in the initial rubrics only addition was considered correct.
The wording of several other questions makes it plain that the exam was not proofread by people with adequate mathematical training. This was also the case with the January exam. Examples: Both the January exam and the June exam ask for the "inverse" of a statement of the form "if A then B". I am inclined to assume that this concept of the "inverse" of an implication exists in the curriculum guide, but I am sure that many professional mathematicians would have to guess what is meant. The given answers (multiple choice) make it clear that it is not the negation.
Another example of a poorly worded problem: January Question 7 and June Question 20. They are very similar and have the same flaw. June Q20 asks: "How many different five-member teams can be made from a group of eight students, if each student has an equal chance of being chosen?" The "equal chance" bit does not belong in the question.
There are some minor irritants. The tests (January as well as June) use "equivalent" where mathematicians would use "equal". June question 4 asks "Which of the following does *not* have rotational symmetry: trapezoid, regular pentagon, square, circle?" A mathematician would not be happy with this formulation, although it is clear which answer is intended. (The circle has complete rotational symmetry, the square and regular pentagon have symmetry with respect to rotations over a multiple of 90 or 72 degrees, and only the trapezoid has, in general, no rotational symmetry at all.) The data analysis questions, on the latest exam as well as on earlier instances, are a further source of mild irritation. Mathematicians tend not to care about the "mode" of a data set, but it is on the curriculum and students can learn what it is. Likewise for reading a stem-and-leaf display and reading a box-and-whisker plot.
Part 4 of the June test, especially, has several multi-stage questions that are probably as much a test of intelligence as of learning. (I don't mean this as criticism.) I have not made a careful item by item comparison between the January and the June tests, but it looks plausible to me that the June test is indeed more difficult. Without the scoring rubrics I wouldn't try to say more.
I am convinced that the Regents Math A should be a predictable exam of which the level of difficulty is carefully matched between instances. It is possible that the June, 2003, instance failed on that measure, and the State Superintendent should study that very quickly and decide if an adjustment of the passing score is in order. The press reports, however, give a wrong impression. The exam is, on its own, not unreasonable and not wholly out of line with earlier instances.
[Addendum, October 24, 2003. TheJune, 2003, and earlier instances of the exam are posted on the Regents Examinations Web site, under the link to Mathematics A. Procedural information related to the exam is posted at the State Assessment site under High School General Information. I posted a Critique of the New York State Regents Mathematics A Exam on my Web pages, accompanied by a Detailed Critique of specific items on the June 2003, January 2003, and August 2002 exam instances. The New York State Council of School Supervisors produced an Analysis of the June, 2003, Administration of Physics and Math A Regents. I summarized the issues in a Blog entry Update on the Regents Math A. Commisioner Mills and the Regents appointed an independent panel to review the Mathematics A Regents exam. This panel provided a Report to the New York State Board of Regents and the New York State Commissioner of Education in October. At the same time the Association of Mathematics Teachers of New York State produced a Math A position paper for the New York Regents. Even before public release of the independent panel report commissioner Mills recommended and the Regents enacted changes in the future administration of the Math A exam. These changes are described in an October 2, 2003, press release (released October 9 or so): Four Policy Decisions on Assessment.]