Home » NYS APPR
Category Archives: NYS APPR
This article was originally published on 10 September in the Washington Post under the title Good or Bad? New rating system can’t decide about this principal.
Sean C. Feeney
Principal, The Wheatley School
President, Nassau County High School Principals Association
Co-author, Letter of Concern about NYS APPR Legislation
Beginning last week, superintendents in New York State’s public school districts received the “Growth Scores” for teachers of grades 4-8 and principals serving grades 4-8 and 9-12. These scores are based on the new Common Core test results, which saw a 30-point drop in students being classified as “proficient.” Although our Commissioner of Education has repeatedly tried to assure the public (http://www.engageny.org/resource/parent-resources-grades-3-8-ela-mathematics-tests) that the dramatic drop in student scores does not reflect negatively on student, teacher or school performance, this message flies in the face of reality. As the State continues to “build its plane in the air,” (http://www.washingtonpost.com/blogs/answer-sheet/post/the-dangers-of-building-a-plane-in-the-air/2011/09/30/gIQAojqWAL_blog.html), those of us who work in schools are seeing how destructive these poorly-planned initiatives have been.
As thousands of educators across New York State have publicly indicated, the excessive testing and use of student scores to rate teachers, principals and schools is misguided, not based in sound research and rushed in its implementation. These facts are ignored at the peril of our students and schools.
As we see more and more results of New York State’s initiatives, the more serious problems and contradictions are revealed. Looking deeper into the recent scores released, one can see at least three glaring problems:
1. State Provided Scores are Not Reflective of Reality
As the principal of an 8-12 school, I receive two separate growth scores from the State. According to the State’s growth measures for 4-8 principals, I have been classified as a “Developing” principal. This is one step above the “Ineffective” rating. According to the growth measure for 9-12 principals, I am an “Effective” principal. This notion that one can look at a single grade and extrapolate a rating for an educator is nonsensical and bad statistical practice. Additionally, it is not reflective of the educational offerings we provide our students across grades 8-12.
Our school programs and offerings reflect a supportive community that has high expectations for its children. We offer robust music, theatre and art programs, and students have ample opportunity to participate in student clubs and athletics. Our school has a strong commitment to community service with faculty and students all participating in a day-long a Day of Service and Learning. I have the blessing of working with a wonderfully talented faculty. As the principal, I have been smart enough to listen and support all the time, to stay out of the way some times, and to push hard other times.
So how has our school done preparing students for college? Our most recent State report card reflects a graduation rate of 100%, with 89% of our students earning the higher Advanced Designation diploma. These rates are far above the New York State average of 74% and 30%, respectively. Our school is highly ranked on the national lists of top high schools. Virtually every graduate attends college, with over 90% of them attending a four-year college. Students have the opportunity to participate in career mentoring, science research and mathematics research. Every school in our country should have the support and programs we are able to provide our students. How do these two scores, as well as the scores that my teachers received, reflect our work in preparing all students for college and careers?
2. State Provided Scores are Not Consistent
In one of the many memoranda to schools, Commissioner King boasts that “about three-quarters of individual teachers will earn the same or better HEDI rating than they did in 2011-12.” http://www.engageny.org/sites/default/files/resource/attachments/growth_score_release_letter_to_superintendents.pdf. This is not something that should make our Commissioner proud! If 75% of teachers earned the same or better rating than last year, that means about 25% of teachers earned a worse rating than they did in 2011-12. No matter how it is viewed, this amounts to an alarming amount of movement in a model that purports to measure “teacher effectiveness.” Did that many teachers become worse at their craft from one year to the next? Which measure is accurate: last year’s measure or this year’s measure? Of course, this lack of intertemporal stability in value added measures is one that has been identified by researchers for years (http://www.urban.org/UploadedPDF/1001266_stabilityofvalue.pdf). Clearly, there is a problem with the model. A system that purports to be objective but results in teachers bouncing from Ineffective to Effective or from Highly Effective to Developing in one year is evidence of a capricious, inconsistent system.
3. State Measures Contradict Each Other
My “Developing” rating as a 4-8 principal is based on the performance of our 8th Graders on the Common Core examinations administered in April 2013. As our Deputy Commissioner explained in the infamous March memo (http://www.engageny.org/resource/field-memo-transition-to-common-core-assessments) only students achieving at a level of 3 or 4 on these exams are considered to be on a college and career trajectory. (This is the same memo that informed us that only one-third of New York State students would demonstrate proficiency on the exams that were still weeks away from even being taken!) Well, how did our 8th graders do on the mathematics examination? Only 39% of them earned a level 3 or level 4 designation. Clearly, I must be an ineffective leader to have such low performance among my students!
Not so fast. At our school, 92% of our 8th graders also took the high-school Integrated Algebra examination. Our overall passing rate on this exam was 97% last year. The passing rate of the 8th graders who took the Algebra Regents examination was over 99%. A few years ago, the State established “College and Career Readiness” passing thresholds for these exams. Despite the fact that these thresholds and the correlational study on which they are based have been discredited (http://roundtheinkwell.com/2011/10/20/letter-to-the-new-york-state-regents-on-ccr), Commissioner King insists that only students who meet the Aspirational Index score of at least 80 on the Integrated Algebra examination are college and career ready. So how did our 8th Graders do against this measure?
Nearly 74% of them scored at this higher threshold. Of even greater concern is the fact that nearly 60% of the students who earned Level 1 or Level 2 scores on the 8th grade assessment passed the Integrated Algebra Regents at the “college and career ready” threshold of 80. How do I tell a student that he was not proficient in April, but met the Commissioner’s “College Ready” standard for graduation two months later?
In their desire to earn Race to the Top funding, our State Education officials have created a system of contradictions, mixed messages and harmful outcomes. This is what happens when one throws an entire state into a chaotic system of mandates and practices that are not thoughtfully planned and are simply not grounded in best practices. Yet I’m the one being labeled as a developing leader.
On Wednesday, April 10th, a panel of educators presented an overview of the impact that our current regimen of high stakes testing is having on our students and schools. Panelists included:
- Dr. William Johnson, Superintendent of Schools, Rockville Centre
- Dr. Carol Burris, Principal, South Side High School
- Dr. Sean C. Feeney, Principal, The Wheatley School
- Ms. Sharon Fougner, Principal, EM Baker Elementary
- Dr. Don Sternberg, Principal, Wantagh Elementary
- Nikhil Goyal, Student Author and Activist
- Dr. Lola Nouryan, Psychologist
- Leonie Haimson, Parent
The guiding presentation can be downloaded through the following link: More than a Number Updated 10April13
Originally published on-line at the Washington Post’s AnswerSheet Blog
New York State schools are back in session! With the new school year comes a new responsibility for principals across the state: the need to inform teachers of their “growth score” based on the New York State assessments their students took in the spring. This teacher growth score is one of the parts of the New York State APPR system that was implemented last year in a rushed manner against the very public objection of over one-third of the New York State principals along with thousands of other teachers, administrators, parents and concerned citizens (see www.newyorkprincipals.org for more information).
These state-supplied scores were the missing piece in a teacher’s final end-of-year score — potentially determining whether or not a teacher is deemed Ineffective and therefore subject to requiring a Teacher Improvement Plan (TIP) within 10 days of the start of the school year. These scores were not available to schools until the third week of August. So there you have it: high-stakes information that can potentially have a serious impact on a teacher’s career being supplied well past any sort of reasonable timeframe. Welcome to New York’s APPR system!
As a principal, I sat with each of the teachers who received a score from the state and tried to explain how the state arrived at these scores out of 20 points. One of the first teachers with whom I did this was Ashley.
Ashley is the type of teacher that all parents want for their child: smart in her content area and committed to making a difference in her students’ lives. Ashley works incessantly with her students, both inside and outside of the classroom.
During her free time, Ashley can always be found working with small groups of students in the hallways or any free space in the area. She has taken our school’s math teams on weekend trips as our mathematics team has found success in various competitions. Over the past four years, 91% of her 179 Algebra 1, Geometry or Algebra 2/Trigonometry students have passed the corresponding Regents examination on their first attempt.
At the end of every year, students and parents send in countless notes of thanks to Ashley for her tireless efforts. Ashley has worked with our highest achieving students as well as many of those who struggle with mathematical understanding. For those who struggle, Ashley has a well-deserved reputation for making them more confident, successful and comfortable with the material. Last spring, Ashley was recognized as the Parent Teacher Organization teacher of the year.
So what score did the state assign Ashley? Well, she earned a score of 7 out of 20 points. According to the state’s guidelines, this makes Ashley a Developing teacher. Goodness. To those of us who know Ashley and have had the pleasure of working with her over the years, this is a jaw-dropping result. Ashley’s score defies all understanding of who she is as an educator. Her score flies in the face of how she is valued in our school and what she has done for students in our school. Her score contradicts the thoughtful evaluations given to her over the past five years.
How, then, is one to understand this score?
Officials at our State Education Department have certainly spent countless hours putting together guides explaining the scores. These documents describe what they call an objective teacher evaluation process that is based on student test scores, takes into account students’ prior performance, and arrives at a score that is able to measure teacher effectiveness. Along the way, the guides are careful to walk the reader through their explanations of Student Growth Percentiles (SGPs) and a teacher’s Mean Growth Percentile (MGP), impressing the reader with discussions and charts of confidence ranges and the need to be transparent about the data. It all seems so thoughtful and convincing! After all, how could such numbers fail to paint an accurate picture of a teacher’s effectiveness?
(One of the more audacious claims of this document is that the development of this evaluative model is the result of the collaborative efforts of the Regents Task Force on Teacher and Principal Effectiveness. Those of us who know people who served on this committee are well aware that the recommendations of the committee were either rejected or ignored by State Education officials.)
One of the items missing from this presentation, however, is an explanation of how State officials translated SGPs and MGPs into a number from 1 to 20. In order to find out how the State went from MGPs to a teacher effectiveness score out of 20 points, one needs to refer to the 2010-11 Beta Growth Model for Educator Evaluation Technical Report. Why a separate document for explaining these scores? Most likely because there are few State officials who are fluent in the psychometrics necessary to explain how this part of our APPR system works.
It is incredulous that the state feels that it is perfectly fine to use a statistical model still in a beta phase to arrive at these amorphous teacher effectiveness scores. I make it a point not to use beta software on my computer, for I do not want something untested and filled with bugs to contaminate the programs that are working fine on my machine. It is a shame that the State does not have the same opinion regarding its reform initiatives.
As explained in the technical paper, the SGP model championed by New York State claims to account for students who are English Language Learners (ELL), students with disabilities (SWD) and even economically disadvantaged students as it determines a teachers adjusted mean growth percentage. While the statistical explanation underlying the SGP model is carefully developed, nowhere do the statisticians justify the underlying cause for any change in student score measured. In other words, what is the research basis for attributing any change in score from year to year to the singular variable of a teacher? The reason why this is never explained is because there is virtually no research that justifies attributing the teacher as the sole cause of a change in student score from year to year.
So if it is not solely the teacher who caused the change in score, to what should one attribute a change in student score? Well, that is a question that continues to challenge statisticians and educational researchers. Despite the hopes and declarations of so many of our present-day “reformers,” we simply do not have to tools necessary to quantify the impact a single teacher has on an individual student’s test score over the course of time. Derek Briggs presented a critique of the use of SGPs in this paper.
How can one explain Ashley’s shockingly low score, however? As a principal who has always availed himself of data when evaluating teachers, I would sit down and have a conversation about the test results so that I could put them in context. Here is what we know about the context of Ashley’s score:
* This year, Ashley’s score was based on her two eighth grade classes, not the results of her Regents-level classes
* The two eighth grade classes were different curricula: one was an Algebra course and the other was a Math 8 course.
* The Algebra 8 course is geared towards the Regents exam, which is a high-school level assessment that is beyond the mathematical level of the NYS Math 8 examination. Ninety one percent of Ashley’s students in this class passed the Regents Algebra 1 examination. There is different content on the Math 8 exam, which can make it a challenge for some of our weaker Algebra students. In fact, of the students who took the Algebra course, one-quarter of them passed the Regents examination but scored below proficiency on the Math 8 exam.
* In the two weeks prior to the three-day administration of the Math 8 exam in April 2012, students in Ashley’s class had one week of vacation followed by three days of English testing. In the two weeks leading to the beginning of the Math 8 exam, Ashley saw her class only three times.
Rather than place the student results in context, the State issued a blind judgment based on data that was developed through unproven and invalid calculations. These scores are then distributed with an authority and “scientific objectivity” that is simply unwarranted. Along the way, teacher reputations and careers will be destroyed.
Despite the judgment of the New York State Education Department, Ashley remains a model teacher in our school: beloved by students and parents; respected by colleagues and supervisors. She continues to work on perfecting her practice and helping her students gain confidence and skills. My hope, of course, is that she will continue to feel that she is part of a profession that respects teachers and students alike, not one that reduces them to a poorly conceived and incoherent number.