A student of mine just sent me this poster pic. I just had to share it!
Category Archives: Year
Differentiated Instruction
One of the buzzwords in education these days is differentiated instruction. In the field of teaching this means that we create different lessons for our different types of students.
“Differentiated instruction is the way in which a teacher anticipates and responds to a variety of student needs in the classroom. To meet student needs, teachers differentiate by modifying the content (what is being taught), the process (how it is taught) and the product (how students demonstrate their learning).” (from http://education-portal.com/academy/lesson/what-is-differentiated-instruction—examples-definition-activities.html)
The idea of differentiation is that some of our students are different—e.g. some are second language learners, some have learning disabilities, some are just behind, some are “gifted” etc. These students need either different lessons, or more commonly alterations and adaptations of the lesson that the “regular” students get. This is seen as an advance from the one-size-fits-all structure of many lesson plans and textbook lessons. In fact most textbooks now come with suggestions for such adaptations. All of this sounds very good—we are taking seriously that not all students are the same and helping teachers to support such students.
But is this really such a good idea? I am going to critique this theory on a few levels. One, is it realistic? Elementary school teachers already have a massive job on their hands making lessons for each different subject area. Now they have to multiply that by how ever many different types of student they have. An adaptation for their second language learners. But can they assume all their second language learners need the same adaptation? One might be a new comer, another an intermediate speaker. Then there are the students with learning disabilities. But again—each one of these is likely to have a slightly different disability. And we move on to the “gifted” students. And what if some students fall into more than one category? When and how does a teacher find time to create all these adaptations and manage them?
There is also the factor that this approach singles out some students as “normal” and others as “different” needing differentiation. There is a lot of evidence that labels often become self-fulfilling prophesies for students. It also sends a message about normalcy to both groups. How will this effect the self-identity of these students, and the view of them by other students?
So, should we return to the one-size-fits-all approach so as not to single out students and to make teachers jobs easier? This is one of those false choices. These two choices assume a teacher (or textbook) centered approach to learning.
Another option, that progressive educators have been practicing successfully for over a century is to have lessons and teaching units with activities that are open-ended and allow students to find their own approach that meet their individual interests and abilities while still helping them develop necessary skills and abilities. This actually mirrors how people have learned effectively outside of school since time immemorial where people of differing abilities, backgrounds and interests all work together on common tasks. Thematic instruction, project-based approaches often fit this. One common example of this approach is the Writer’s Workshop. All students work on writing a class book, maybe even within a certain genre or topic, but they all get to write what they want within that framework. They each can work at their own pace and ability, and the teacher and their peers all help each other refine their writing.
Larger projects can use this approach as well. Such a project might be the study of an ancient civilization. It might be the investigation of one’s community. It might be an examination of the physical environment. In this approach students investigate, build, write, read, observe, and create around the theme, each at their own level.
I am not going to say this approach is not a lot of work for the teacher, but it is not about creating lots of individual lessons, but rather creating a climate for learning, making the materials and resources available, and then knowing how to support each student to do their best within that framework.
To see a wonderful example of this approach at the elementary level see the video “We All Know Why we are Here”
Relationships
The more I think about education and learning, the more I see relationships as the key to what really matters. If I think about all the movies I have seen about “great teaching,” both fictional and those “based on a true story,” while the actually teaching going on in them varies enormously, what they all have in common is a teacher that builds caring strong relationships with their pupils, from “To Sir with Love” “Up the Down Staircase” of the 60s, to more recent movies such as “Mr. Holland’s Opus” and “Dangerous Minds.” But of course that portrayal could just be the license of the writers and directors.
But I would say I have found the same in my experience as a teacher. I worked with teachers with many different pedagogical approaches. If you have read my previous columns, you will see it is clear I have strong beliefs abut which are more effective. However, the most consistent thing that I noticed of teachers that appeared to me as more effective was that those teachers all had strong relationships with their students. The students knew their teacher expected them to learn, and was there to help them succeed in doing so.
It was really much for this reason that I decided to become an elementary rather than high school teacher. I did not see it as possible to really build those relationships if every hour I had a new group of students. With elementary school kids I had the same ones all day long. (It is also a reason I have never liked “regrouping” with other teachers—I never saw the trade off as worth the loss in knowing my students fully).
One anecdote. At one place I taught, we were using the Reading Recovery program for our struggling first grade readers. Reading Recovery is a strongly researched based program giving intensive support to the lowest readers in first grade, based on some of the best research of learning to read, with a strong research record of its own, and all the practitioners of it have to be credentialed teachers who have gone through an intensive training in the model. However, as a second grade teacher, my struggling readers did not qualify. So instead we used instructional assistants, who had a rudimentary training in more traditional phonics approaches to work with them. I would argue that second graders who are still struggling with reading are probably actually more difficult candidates, as they have a longer history of failure to overcome.
Yet, in the decidedly non-random and small sample that this consisted of, my instructional assistant succeeded with every one she worked with to at least getting them to the point of breaking the code in learning to read. The same cannot be said of the Reading Recovery program that had about a two-thirds success rate with our students. I attribute it to the strong relationship she built with each of them—letting them know that she believed they each would and could learn to read.
This, maybe, is what worries me most about many of today’s’ educational reforms. They make those relationships more difficult. Scripted curriculum, larger classes and school consolidation. use of technology for instruction, and worst of all, the tactics of fear—trying to scare teachers and students into doing a better job. Each of these, in a different way, makes it slightly more difficult for teacher and students to develop strong relationships.
I am about to embark on teaching an all on-line teacher education course. I will see to what degree this mode allows for and interferes with such relationships.
Poverty and Education
Progressive educator Deborah Meier and Michael Petrilli of the Fordham Foundation have been debating on EdWeeks’s Bridging Differences blog.
In his most recent post Michael claims that poverty is not the issue (even though, as another commentator to his post mentioned, the issue was not originally framed as poverty, but inequality, even in his own words).
From Petrilli’s Bridging Differences blog:
- “Most were born to single mothers, and their fathers have been absent from the start, or by the time they turn two or three;
- Most of their mothers were teenagers or in their early 20s when they gave birth;
- Most of their mothers have very little education—a high school diploma or less–and thus few marketable skills;
- Many of their mothers suffer from mental illness or addiction or both;
“If we give these families more money…will it erase the huge gaps….between these kids and their age-mates born into two-parent families? With highly-educated mothers and fathers? To believe so, you’d have to put as much faith in cash transfers and social services as some reformers put in schools. You’d have to believe in miracles.”
In response, I would say that tackling poverty and creating full employment and tackling society’s inequalities would actually help solve those too. Fathers are absent because they cannot support their families. Young motherhood is often a symptom of hopelessness, as is drug addiction. Michael shows data that America’s poverty is not really that much worse than other countries (though still worse even by his figures), so the problem cannot really be poverty (since he also accepts the data purporting that they do better academically). What he leaves out is that even if its true, those other countries do a better job of providing the supports for the poor that he derides as useless–housing, medical care, food, pre- and post- natal care—than the U.S. It may be those supports that keep fathers at home, create less single motherhood, and provide the supports needed for those who are single mothers.
While most critics of our current economic system and I think giving the poor more money and supports is a good idea, we do not see it as the solution either, but rather a band-aid, and when you are bleeding a band-aid is good to have! What is needed is a society that can provide meaningful employment for its citizens, that can provide decent housing, food, medical care, etc. It needs a society organized for a more equitable distribution of the resources.
A better education for the poor helps the individual student succeed, but it does not create more jobs nor reduce the overall rate of poverty nor solve the issues of inequality in a developed country such as ours. It does not change the number of winners and losers, though it just might even the odds a bit as to who gets to be winners and losers.
As John Dewey noted almost a century ago, a certain type of better education, i.e. one that help students participate and understand democracy and develop certain habits of mind, can be one of the aspects to creating that society, but it alone cannot do the job. And most of the reforms that Petrilli supports—more testing and top-down “accountability” based on that testing—actually create a less, not more democratic culture in schools, especially schools for the poor.
The Hardest Questions Aren’t on the Test
This is a wonderful book about the Boston Arts Academy written by the principal. In the genre of Deborah Meier’s The Power of their Ideas, discussing the history of the school to illustrate what it takes to make a powerful school. She describes the ups and downs. She gives stories of particular students. She describes how the school has developed, going through changes and struggles. She uses these stories to help us understand what schools could and can be like, how they can be powerful places for all to learn.
The Hardest Questions Aren’t on the Test:
Lessons form and Innovative Urban School
by Linda Nathan
Beacon Press
189pp
Common Core
I have been hearing from many friends who work as K-12 teachers, as well as some teacher educator colleagues here in California, that they are excited to see the coming of the Common Core standards. They see in them a move away from an emphasis on teaching by rote and a move toward emphasizing higher order thinking skills. I truly hope that they are right about this. A shift in balance from a preponderance of rote and conformist styles of teaching to more emphasis on creativity and the other aspects of what are called the higher order thinking skills in Bloom’s Taxonomy is needed.
On the other hand, I have been hearing some critiques from other colleagues, especially early childhood educators, about some of the specific standards that they say are developmentally inappropriate. One of those that I hear mentioned often is having young children doing more expository reading and writing.
However, what I want to address here is that regardless of the specifics of the content of the new Common Core standards, they are actually a continuation of what I see as a dangerous trend in the educational policies of this nation.
Common Core standards are now nationally mandated standards. In order to justify such a mandate several assumptions have to be accepted. One is that a lack of uniformity in the curriculum nationwide is part of the problem in today’s education. Such standardized curriculum have often been the mark of totalitarian governments—we used to make fun of how in the Soviet Union on a certain day at a certain time every fourth grader in that nation would be studying the same thing. Is this what we need?
The argument for national standards is that many nations that outscore us on international standardized tests have national standards. First off this connection breaks one of the first rules of research—confusing correlation with causation. Furthermore, other nations that don’t have national standards also beat us, and we beat many that do have them. Not to mention all the problems with comparing the quality of educational systems based on those tests (e.g. who is being tested in each country is not comparable, and is what is being compared what really matters?). Also the meaning of national standards varies enormously. In many countries they are just a general guideline, much like the California frameworks used to be.
The other argument for national standards is that standardization means everyone gets an equal education. The problem, according to this argument, is that in communities serving the poor the children get a watered down curriculum and national standards will mean the poor get the same education as everyone else. This assumes many things—that we want all students to end up the same, that equal inputs equals equal outputs, that as long as the standards are the same, all other differences for rich a poor kids are erased (opportunities beyond school, resources, nutrition, not to mention funding, as we have the least equal school funding of any industrialized nation in the world).
However, the main issue that I want to discuss here is the assumption of a national consensus on how children learn and what they should learn. In a totalitarian state or dictatorship, the right of the State to dictate such matters is assumed. However, in a democracy, to mandate something nationally should only happen in cases where there is a strong national consensus. Even if there were a majority opinion, such a mandate would just be a form of tyranny of the majority.
Do we have a national consensus on what children should learn and how they should learn it? To many people, at first glance people seem to think it is obvious. We teach kids reading, writing, and ‘rithmetic. Then I guess history, science, and social studies. But, is there really any such consensus? They have to learn to be fluent in reading and writing, and to know algebra, and the important facts of history and science.
But what does it mean to be fluent in reading and writing? Who says algebra is that important? What are the facts of history and science that are most important? And then there is how do we teach these things, and how do we know if they were learned?
Given that there are major debates on each of the above regularly, it is clear we do not have any actual consensus. Ask a variety of friends regarding how schools should teach, what they should teach, and what the main goal of schools should be, and I bet you will get a wide variety of answers. And that is your friends—once you start asking those in different communities, of different political, religious and cultural persuasions, the diversity increases.
If you ask the leading educational experts of today, you will probably get even less consensus. There is still huge debate among educational theorists about approaches to education. Direct Instruction versus constructivist and discovery approaches. What content is most important. How we measure success. It is just for that reason that most standards have been long comprehensive lists. To reach consensus, the committees making the standards just all accept each other’s ideas.
Now let us look at some things we do know about learning. As all of us notice, children always tend to want to be and act like the important people in their lives, imitating those they see as successful—especially those they want to and think they can be like. We learn from the company we keep and the experiences we have, to sum up and simplify the theories of such giants in learning theory such as Bandura, Vygotsky and Piaget.
Democracy is only partially a set of rules and procedures for making decision and electing representatives. More centrally, it is a way of thinking, a habit of mind. Therefore if we want our students to be democratic citizens they need to be socialized in a democratic environment. Are our classrooms or schools such environments? I doubt many could claim they are. A nationally mandated curriculum makes that virtually impossible. States are required by the Federal Government to impose the standards on their schools. This is done through the school districts who then order their principals to carry it out, who then order the teachers to do so, who then impose the curriculum on the students. We thus end up with people all down the line who are powerless except to carry out the designs of those above them. Those who oppose or disagree with the curriculum will either choose not to be part of such a system, learn to keep quiet about their opposition, or will likely lose their jobs. Children cannot learn democracy in such a culture. A few privileged students may learn it elsewhere or perhaps later, but public schools will not be part of that lesson except as a negative example.
If we look at the most successful schools around the country they do not share a common set of beliefs between them, except maybe a belief that all their students can and will succeed. They each have a very different set of ideas about what is important for children to learn and how best to carry it out. There are the Met schools in which students mostly work with an adviser while engaged in internships with minimal formal classes. There is High Tech High, where students develop projects and inventions often using computer technology. There are the schools of Deborah Meier (CPEI, Mission Hill) based on their five habits of mind and using a graduation by portfolio design. There are the KIPP schools based on a philosophy of no excuses, a longer school day and year and family involvement. There are the Montessori schools based on their particular form of pedagogy and curriculum. Waldorf has theirs, and I could go on and on. Each of these schools or programs has a record of success. Those who work at the school often have developed together those standards or at least chosen to be there because of them, as have the families. It is this freedom that we should be striving for in a democratic nation.
Even if you disagree with me about the central purpose of public education being to prepare students for democracy, or if you disagree about how that is achieved—in fact especially if you do not agree—you are in fact bolstering my argument against the Common Core standards (or any set of national standards). That is, we clearly do not have a consensus, and who has the right in a democracy to impose such a consensus by fiat?
Schools and the Business Model
Policy makers and the current so-called reformers talk about the need to run our public educational system more like a business. From what I see, that is being done, with the same disastrous results.
We see businesses paying their CEOs exorbitant salaries while reducing their labor forces and cutting the wages of those who actually do the work. Often these CEOs are hired irrespective of their knowledge or experience in the field or of whatever product or service that company is engaged in. Then when their policies fail, they are paid huge sums to buy out their contracts, while a replacement CEO is paid even more.
In education we see the same rise in wages for superintendents, especially of large school districts, while cuts are being made everywhere else. Often these new leaders come with no educational backgrond at all, their only experience being in the business world. And when they fail, they too see their contracts paid off while a new superintendent is hired at a higher salary.
In business more attention is being paid to short term profits to give investors a quick return on their money, often at the cost of long-term quality or stability.
In education we see schools forced to find ways to get short term test score gains per Federal and State mandates, which are often made at the cost of building a solid educational foundation and understanding.
In business companies engage in cooking the books to make their profits look better than they might actually be, and we read about these scandals almost daily. In education we see districts and schools cooking the books and engaging in practices to make their test scores and other data look better and we act just as surprised as the same scandals appear in education.
In business quick profits are the goal, at whatever cost, legal or illegal. In public schools better test scores are the legal tender to be attained at whatever cost.
Schools are learning to act more like contemporary businesses. In business such practices have taken the world’s economies to the brink of disaster and brought us the worst recession since the Great Depression, while the gap between rich and poor continues to widen. Such practices and mindset are doing the same to our public educational system, widening the gap between the quality of education for rich and poor.
One might notice that it is practically unheard of for the private schools where the elite send their children to use the rhetoric of business practices to describe their own schools.
Standards-Based Education
Currently in education there is a lot of talk about standards-based education and the need for high standards. I will discuss in this column where that concept came from and how it has been distorted from its original use.
The idea of a standards-based educational system came from the work of Ted Sizer (1932–2009). In the early 1980s he was involved in a nationwide study of high schools that resulted in his book Horace’s Compromise (and later Horace’s School and Horace’s Hope). In Horace’s Compromise, Sizer describes the work of a typical teacher, and how no matter how willing, well-meaning, and hardworking, the teacher cannot meet the needs of the over hundred students he sees everyday, and how students by the same token cannot do deep quality work while jumping from one subject to another each with a different teacher and mostly sitting there being expected to soak up facts and concepts. In other words, the lack of real quality learning going on in schools was not the fault of teachers or students, but the design of the institution and the compromises teachers and students made with each other to survive in such an institution.
Sizer proposed that instead of students being rewarded for successfully passing a certain number of courses, and being in school a certain amount of time, they be required to demonstrate the knowledge and abilities of a successful high school student through some sort of performance assessment where students actually showed they could apply what they had learned. He also posited certain attributes that schools would need in order to carry out such an education. What came out of that directly from Sizer and likeminded educators was an organization, the Coalition of Essential Schools, which holds a set of ten common principles that schools doing such work adhere to. This organization supports schools in trying to make the changes to move toward applying these ideas. According to Sizer, how schools would measure this success, and how each school would carry out those principles in practice, needed to be locally decided.
This idea of Sizer’s that students should graduate by being measured against a set of standards rather than just seat time became popularized in the 1990s. However, in many ways the concept got turned on its head. For one thing, the term “standards” took on a new meaning from its usual everyday meaning of a level of quality. Instead “standards” became laundry lists of facts and concepts, both broad and discrete, to be learned, as well as levels of performance. These standards, rather than being locally decided as Sizer proposed, have been mandated by State authorities (and now we are moving to National mandates). In most states these laundry lists of standards are typically so long that one expert declared that it would take over 20 years if students were just exposed to the material for each standard, and much more if they were really expected to master them.
The other distortion is that meeting these standards is measured by standardized tests. Performance has come to mean not what Sizer had in mind—the ability to carry out real world tasks that used the knowledge and abilities that schools decided were important—but how one “performs” on a standardized test. These standardized tests are designed to test students’ recall of a random sample of what is on that laundry list of facts and concepts. High standards have come to mean high scores on such tests.
Sizer’s idea was that graduation by standards should free up schools to look and act differently, and free up students to demonstrate their knowledge in a variety of ways. The current practice of “standards” has meant the standardization of curriculum, pedagogy, and assessment of the students, as well as their teachers, schools, districts and states by the use of standardized tests.
So far there is no evidence that the current use of standardized curriculum and the high stakes use of standardized tests has improved the quality of education. The achievement gaps these so called reforms were to solve are as great or greater than before these changes. Graduation rates are overall no better, and we do not hear high school teachers claiming that students are coming in more prepared than they used to be. So far the only response the education establishment has offered to this lack of results is that we need more standardization, more tests, and higher stakes.
On the other hand, Sizer’s ideas of standards without standardization have also been tried out, at times with astounding success. One of the first schools to implement Sizer’s ideas was Central Park East Secondary School (CPESS) a public school of choice in New York’s East Harlem. Deborah Meier, building on her work at Central Park East Elementary School, collaborated with Ted Sizer on how to meld his ideas and hers to develop a secondary school on the Coalition principles. They came up with a school where students studied fewer topics, and worked with fewer teachers more intensively. All faculty and administrators worked as advisors who stayed with students over time and met with their advisory group daily. Students took part in internships in real world professional settings. The standards of the school were upheld through a series of portfolios and defenses of those portfolios in front of a committee. Students graduated, not after a prescribed number of years or prescribed number of completed courses, but when they had successfully passed and defended those portfolios. The standards of Central Park East were built around certain “Habits of Mind” that the faculty believed were important in all facets of life and in all disciplines. To a large extent, the demonstration of the use of those habits was the rubric used to decide if the portfolio or defense of the work met the schools standards. The students of CPESS had success at graduating high school and going on to, as well as succeeding in, college far beyond their demographic equivalents in other public high schools in New York (see David Bensman’s fascinating book Central Park East and its Graduates which documents his study of CPESS alumni).
After CPESS, a whole network of such schools sprung up all over New York City, and to some extent nationwide. Schools such as Urban Academy, the International High Schools, the Met schools, High Tech High, and Boston Arts Academy, to name just a few, continue in this tradition of high standards without standardization, of depth of knowledge over coverage, and of the importance of relationships with students as essential to successful education. While each of these school looks very different, in each school one will see students who are passionately following their own interests while being held to a common set of high standards in a non-standardized curriculum. These schools have shown that they help students beat the odds in terms of graduation and getting into college. Even more importantly, these schools produce graduates with positive attitudes toward learning and their ability to shape their own futures and contribute to the larger society.
Assessment in California Teacher Education
(This column is adapted from a talk I gave at the University of Kyoto in January, 2012)
Those in the field of assessment often refer to two important standards that assessments are expected to meet, reliability and validity. Reliability meaning that the same results would be obtained if the assessment were given again, or if a different person was scoring the assessment.
Validity means that the assessment actually measures, assesses, what it claims to be measuring/assessing—and whether it predicts how one will perform in the future (Ormrod, 2005).
One type of validity is “face validity”—that is, it is accepted that the assessment actually does measure what it claims to measure, without needing statistical proof that it does. The road test portion of the driving test might be an example of that: We can easily agree that if we want to know if someone knows how to drive, we can sit in a car with them and watch them drive. Now, what constitutes good enough driving to pass the test, that is where things might get more difficult to agree. Both how good is good enough, and which things matter most, e.g. how well the student parked, used turn signals, obeyed signs, and how much should each count, can be controversial.
Other tests need to have their validity demonstrated. The paper pencil portion of the driving test might be one of those. Do we have any evidence that those who do better on the written portion are actually better drivers?
However, while we accept that the road test has more face validity, we might wonder about its reliability, the possible subjective nature. The written portion is more reliable, you either filled in the correct bubble/answer, or you did not. However, on the driving portion, maybe the traffic conditions were more difficult when you took it than when your friend did, maybe one instructor is tougher grader than another. Maybe he had a fight with his spouse that morning! Despite these shortcomings, we accept the trade-offs as worth the advantages of such an authentic assessment. A built in safeguard is the opportunity for second, third, and as many opportunities as needed to retake the test.
It is easy to create paper and pencil assessments that are reliable and easy to administer. However, how one’s score correlates to real life application of the knowledge or skill that the assessment is designed to measure is harder to determine. Some, such as myself, argue that there is a built in tension between reliability and authenticity. Real life tasks and situation are by their nature not standardizable: Conditions vary, there is ambiguity, and there is more than one right way to approach a situation or problem. Creativity, a very important human trait, cannot be measured, and one’s ability to act effectively in novel situations is also by its nature not standardizable. Therefore to assess one’s ability to use one’s knowledge and skills in real life situation is likely to have a degree of unreliability, unpredictably.
Furthermore, what one person views as good enough, as quality, in most real life applications also varies. A movie I thought was well acted and crafted, my best friend thought was poorly acted and rang false. And that is in movies made by highly paid seasoned professionals! Multiple publishers initially turned down a number of best selling classics in literature.
Compulsory public schooling in the United States was instituted at a particular point in history, with other changes and advances happening. Part of that was the belief in scientific experts and the new field of psychology as a science rather than philosophy, and the invention of standardized intelligence tests. Americans often want to find the one right way (Smith, 1988). Americans are known for their obsession with measuring everything, and putting numbers to everything. This has played into schools in the forms of tests that can be reduced to numerical scores, and a belief that if everyone takes the same test at the same time in the same way, and test is designed by outside experts, it is therefore objective.
Critics of the standardized tests of today point out the shortcomings of such tests: they don’t really have reliability at the individual level, they are culturally biased, and their inauthenticity—their lack of actual validity in terms of measuring any important, useful skill, ability or knowledge beyond the school house walls. They also object to the indirect influence of these tests in encouraging the teaching of discreet skills and rote knowledge that is quickly forgotten once the test is over (Hursh, 2005; Kohn, 2000; Meier, 2002; Ohanian, 1999).
However, it must be remembered that standardized tests were put in place in part as a seemingly fairer alternative to an aristocratic system, where social position and money was what decided who got into the best schools and got the best jobs. Standardized tests were seen as scientifically objective tests, and therefore gave an equal chance to all. One could rise by one’s merit, not relying on family name or wealth (Smith, 1988).
What authentic assessment is proposing to do is to let people show what they know and can do based on merit, but also more accurately than standardized tests reflect the skills and abilities the person should have by seeing how they apply that knowledge in a realistic situation.
Of course even “authentic assessment” is always a matter of degree. Authentic assessments are generally applied in somewhat contrived or hypothetical situations. In school situations it is rarely practical or even possible to have students demonstrate in the real life situation, and even authentic assessments give us just a sample of the full skill being assessed. To go back to the driving test example, even on the road test, not nearly every possible driving situation is encountered. The driver is asked to carry out a predetermined set of maneuvers at the direction of the tester over a relatively short period.
A large issue for authentic assessment is to overcome the issues of “bias,” which is really an issue of reliability—would a different scorer give that person the same score? One way to address this is through multiple assessors. For instance, at some high schools that use portfolio or exhibitions for graduation, such as was developed at Central Park East Secondary School, they use multiple assessors, while also having outside experts examine their system, and watch it in practice to help them improve and refine it (Gold, 1993; Meier, 1995; Meier 2002).
Another common system to obtain more reliability that is used in authentic or performance based assessment systems is to have scorers be calibrated. A set of benchmarks are set up—examples of the performance assessment carried out at different levels, and the scorers are first trained on what qualities to look for, and then they are asked to score these benchmark examples to see if they give them the expected scores. In theory, only when they can consistently give the expected scores are they considered calibrated, and therefore the scores are considered reliable.
I will now discuss efforts in California to bring a more authentic, yet standardized, assessment in a systematic way to credential teachers.
California teachers are given their credential based a variety of factors. Some have been (and still are) standardized paper and pencil tests. However, as we have discussed, there is a sense that these are not good indicators of how well they would actually teach. These tests are used as measures of minimum knowledge of basic skills. On the more authentic side these candidates are placed in classrooms to learn to teach alongside practicing teachers. In most teacher education programs in California, this is a semester long placement. In some, such as where I currently teach, we require two semesters of student teaching. However, some worry about the standards of those assessing that experience. Were they tough enough? Are they consistent? There is no standard set of measures for that experience. The same could be said of the other criteria, that they pass their college courses to become a teacher. Were the standards from one program to another, even one class to another, consistent (Chung, 2005)?
The legislature of the State of California decided to institute a performance based assessment system on top of the other criteria to both provide an authentic, yet valid and reliable way to measure whether a candidate was ready to become a teacher.
Linda Darling Hammond of Stanford University led a consortium of universities with foundation support to develop such a system, called Performance Assessment of California Teachers (PACT) (another similar system was also developed by the Education Testing System, the CalTPA). In the PACT assessment teacher candidates develop a 3-5 day lesson plan in mathematics or reading, they carry out the lessons in their placement, and videotape those lessons. They document all of this, providing a detailed description of the context where they taught the lesson, describing the school, the classroom and what they know about the students. They provide the lesson plans, and some discussion about those lesson plans. They reflect on what happened when they gave the lessons, what changes they made along the way, and what changes they might make if they were to give these lessons again. They select a 20-minute portion of the video for the portfolio, and discuss what is in that portion. They also provide examples of the assessment used in the lesson from three students of varying abilities. They discuss what they saw overall in reviewing the student assessment, and what they learned about the three students in particular.
This portfolio is then read and scored on a set of 12 rubrics. Several rubrics address issues of planning, several look at the execution of the lessons, several others look at the issue of assessment. The issue of how the lessons helped student access and learn “academic language” is also assessed by two of the rubrics. The people who score these assessments go through a two day scoring and calibration training, and must re-calibrate every year.
In practice, despite the training and calibration, there are still sometimes disagreements (if a student fails, it automatically gets scored by a second scorer. Randomly ten percent get two scorers to check reliability). While in the large majority of cases we probably score the candidates similarly, there are cases where we have scored them quite differently. In such a system, there is room for interpretation. If the rubric asks us if the lesson was appropriate for the students, or the teacher gave clear feedback, what one of us interprets as appropriate or clear may not be the same as another.
These are the trade-offs for a more authentic system. For everything we do, that we add, something is also lost, traded. On the positive side, in my institution it has meant that we have had dialog among the faculty about creating a more cohesive experience for the student. However, as many high stakes assessment systems can do, preparing our students for the assessment itself has taken significant university class time, time that used to be spent on content. In that way students may be losing out. Some also wonder to what extent is the ability to write well, to theorize being assessed, rather than the actual ability to teach. Though assessors are told that the writing itself is not being assessed, it is for the most part a written assessment, albeit of a performance (along with the short video clip).
It is certainly a system that is more uniform than what was in place before. From my experience with the system, it does appear that the stakes have been raised for student teachers. Are the teachers who have now gone through this system, better prepared? Are we better at keeping out unprepared teachers, while not excluding prepared ones through this system? That is a much more difficult question to answer for which there are no solid “facts.”
The problem in the United States is that people are looking for a foolproof “fair” system. The attempt is to avoid human judgment, which by its nature full of biases and well, judgment! Standardized tests, paper pencil tests, offer us the illusion of avoiding judgment, but it just moves such judgment to the creator of the test. It offers reliability often at the cost of meaningfulness.
In the United States we rely on human judgment for our criminal justice system, our courts—very important high stakes decision—and while mistakes are made, maybe even often, it is seen as better than the alternative. Authentic assessment systems at heart require the same faith. A faith that the trade-off of allowing for human judgment is better than the reductionism required to assess in a standardized form. I believe we need to bring more of such human judgment back to our educational system.
References:
Chung, R. R. (2005). The performance assessment for California teachers (PACT) and beginning teacher development: Can a performance assessment promote expert teaching practice? Stanford University. Proquest dissertations and theses, 598p.
Retrieved from http://search.Proquest.Com/docview/305434959?Accountid=10355 Unpublished Dissertation, Stanford University.
Gold, J. (Producer & Director), & Lanzoni, M. (Ed). (1993). Graduation by portfolio: Central Park East Secondary School [Videotape]. New York: Post Production, 29th Street Video Inc. http://vimeo.com/13992931
Hursh, D. (2005). The growth of high-stakes testing in the USA: Accountability, markets and the decline in educational equality. British Educational Research Journal, 31(5), 605-622.
Kohn, A. (2000). The case against standardized testing: Raising the scores, ruining the schools. Portsmouth, NH: Heinemann.
Meier, D. (1995). The power of their ideas: Lessons for America from a small school in Harlem. Boston: Beacon Press.
Meier, D. (2002). In schools we trust: Creating communities of learning in an era of testing and standardization. Boston: Beacon Press.
Ohanian, S. (1999). One size fits few: The folly of educational standards. Portsmouth, NH: Heinemann.
Ormrod, J. E. (2005). Educational psychology: Developing learners (4th ed.): Prentice Hall.
Smith, F. (1988). Joining the literacy club: Further essays into education. Portsmouth, NH: Heinemann.
Why Go to College?
I usually discuss k-12 schooling in this space. However this time, I would like to address higher education. In several of my columns here I have addressed the issue of the purpose of public schools, and I am going to bring that question to this next level. One of the purposes of k-12 schools we are now told is to prepare students for college and university work. What, therefore is the purpose of college and university work? The obvious or cliché answer is to get ahead in life. However, this assumption that the purpose of college is to get ahead in life needs further examination.
In k-12 schools we are told that students must spend more time studying academic subjects. Those involved in higher education are often referred to as Academia. But the people of the United States have contradictory attitudes toward things being academic. On the one hand we have the expression “It’s academic,” meaning it has no practical significance—maybe interesting philosophically or intellectually, but when real or practical decisions matter, irrelevant. On the other hand, in schools today, especially k-12 schools, if it is not academic it is not important. Arts, music, physical education, recess, everything that is not “academic” must be thrown out in this time of crisis. (That such a crisis actually exists is also a matter for much debate, but the media ignores that fact as well—reminds me of how at airports for the last 10 years we are in a constantly reminded over the PA system that we are in a state of emergency. Deborah Meier points out in a recent column that such a crisis seems to have always existed in our public education system.)
It was not long ago that Colleges and Universities were purposely seen as “only academic.” They made no claim to study anything practical, to prepare anyone for the workforce. To do so was seen as beneath them—that was the job of trade schools. Colleges and Universities were where the elite went to improve their minds, to become cultured and well educated—but not trained for the work world. Back when medical schools were being established as the main way for doctors to be certified, many universities resisted having medical schools attached them precisely for that reason—not wanting to be seen as trade schools.
In many fields, the more practical the study, the less esteemed it is—this is certainly true in the mathematics and hard sciences. And even within the hierarchy of university departments, it has historically been that the more practical the less esteemed, with teacher education being at the bottom of the totem pole, and theoretical sciences at pretty much the top.
However, college is more and more being sold to the public as an economic necessity. We are bombarded with charts of the higher earnings that college graduates make. We are told there is no future for those without a college degree. High schools are measured by how many graduates get in to college. Colleges advertise how well they do at getting their graduates good jobs.
For most of the first half of the 20th century most students did not finish high school. But there was no shame in that. Not only no shame, finishing high school was not seen as necessary or even useful in getting a good working class job—one on which someone could end up supporting a family and buying a home, living the American dream.
By the second half of the 1900s, a high school diploma appeared to be necessary. Now in the 21st Century, it seems the college diploma has become the new high school diploma. Whether jobs actually require the higher skills and knowledge obtained in college, or whether it is just that one needs a higher diploma to beat out the competition (or a combination of the two) is a matter of debate. Few employers actually claim that what their employees learned in school in terms of any content was really that helpful for what they needed to do the job.
Despite the debate of the actual practical knowledge learned, colleges and universities are certainly sold as the keys to an improved financial future. This has led to a crisis in identity for higher education. Many, in if not most college professors still think of themselves as teaching students in order to broaden their minds, get them intellectually interested in topics for their own sakes, improve their ability to communicate and think about the larger issues of life.
More and more though, professors are being asked, not just by students, but by administrators and the university system as a whole, to justify the purpose of what they are teaching in practical terms. Departments that are seen as practical, as leading directly to a job, are growing, such as business and engineering schools, while ones that are seen as purely “academic” shrink. And universities, in the competition for students and for money, steer toward those that are attractive in that way.
When college was just for the elite, to better their minds, professors could just say to students, if you are not interested, you do not need to be here. Now professors need to sell their students, and the university as a whole, on the purpose and practicality of what they are teaching. Students complain about having to take courses to fulfill breadth requirements, and professors dread having these students in their courses.
If colleges and universities are just places to further train our youth for a workforce that (supposedly) requires higher levels of education in our technological information rich society, why have such requirements at all? Why not just let students take those practical courses they need for the field they have chosen to pursue? Is there a place in society for the idea that an educated populace is well rounded in terms of the arts, humanities, literature, languages, and sciences? Does such a notion fit with what we want our society to look like?
Maybe we need to actually go back to differentiating between the two? Maybe to most people we offer college as a trade school, clearly focused on preparing them for a professional or economic field. Then reserve, for those who want it, an education to broaden their minds? Is there a way to separate the two without the latter being purely elitist; can we make it affordable and realistically open to all who were interested? Or is this broader purpose something we really want to insist (or encourage at least) for most people, rather than just leaving it up to see who might want it?
I believe that these are issues that our k-16 educational system is ignoring. If not addressed explicitly, the tensions will not go away, but the conflict will be resolved without a serious consideration of the trade-offs we are making, but rather will be the result of small moves seemingly made to meet the current “demands” of “the system.”
Nicholas Meier 