Bloom’s Taxonomy

On the theme of popular ideas that I feel a need to critique….

Bloom’s taxonomy has been around for a long time as an aid to teachers, presented as a hierarchy of sophistication of thinking. I was first introduced to it when I started teaching in the 1980s. The college where I currently teach asks all of their instructors to keep it in mind when developing and teaching their courses.

I have two central problems with Bloom’s taxonomy, both of which I will discuss. Then I will mention how it can be used positively.

First of all, when I have used Bloom’s list, or the new revised list, to analyze a lesson and to think about which categories are being tapped into, I find it hard to pigeonhole activities or questions. My teacher education students and I often find that we can put the same questions into multiple categories depending on how we interpret them. In other words, real ideas and lessons do not seem to fit neatly into these categories, and getting agreement on categories is not intuitive, making them less useful. For instance the top of the six categories is: Creating, putting information together in innovative ways. But “Applying” (third level from bottom) on one chart I am reading is listed as using the knowledge gained (level 2) in new ways. What is the difference between applying knowledge in a new way and being creative or innovative?


A bigger problem I have with Bloom’s taxonomy is that they are presented as a hierarchy, and Bloom meant them that way. In this hierarchy the first stage is knowledge or remembering. In other words, rote learning comes first. The next level, understanding, is that then we learn the meaning of what we memorized. Next we learn to apply the knowledge. After that we can analyze it—break it into parts. Then we can judge it. And finally we can use it creatively. For instance, I was told by someone instructing college professors that they would not get to the highest category with undergraduates, but should reserve that for graduate students!

If Bloom’s taxonomy were used just as a taxonomy—in other words a description of different types of thinking, I can see them as interesting and possibly useful. But generally they are used in the former way as developmental steps to be gone through, as Bloom designed them to be used.

My experience in elementary school bears this out. The “low achievers” and “remedial students” we are told first need to get the basics, and they are given tasks that focus on rote memorization, factual recall and following instructions. The “advanced” or “gifted” students are given assignments that allow them to be creative and analytical. They are asked to evaluate the characters in the story, to do the creative extension activities, and in math they get to do the extra “thinking” problems in the textbook.

This hierarchical idea of thinking ignores what Piaget demonstrated so long ago. Even babies are engaged in all levels of this taxonomy, and the different forms work together synergistically not separately. Learning by rote is the most inefficient way to learn. We are more likely to remember something when it has meaning attached to it. Then even more likely when we apply the knowledge, which is why hands-on and authentic activities are so often recommended. I would want most of the activities that students are engaged in at school to have them using all of these levels of thinking as they carry them out.

Any kindergarten teacher will tell you her kindergartners are creative. In fact there is some research that shows that we actually become less creative and innovative as we progress through school, less able to see the world in new and novel ways, not more so. I hope we do not wait until our students are in graduate school before we let and encourage them be innovative again, the way they naturally were as children.

It is possible to use Bloom’s taxonomy in a positive way. For instance, we create a teaching unit (level 6). We can analyze our unit to see it if includes all the levels of Bloom’s taxonomy (level 4). We apply the unit as we carry out our lessons (level 3), for which we must have understanding and knowledge of our content and pedagogy (levels 1 & 2). As we go along and at the end we evaluate the unit and how it went (level 5).

For a full look at this whole idea of teaching children to think, one should read Franks Smith’s book, To Think.

Differentiated Instruction

One of the buzzwords in education these days is differentiated instruction. In the field of teaching this means that we create different lessons for our different types of students.

Differentiated instruction is the way in which a teacher anticipates and responds to a variety of student needs in the classroom. To meet student needs, teachers differentiate by modifying the content (what is being taught), the process (how it is taught) and the product (how students demonstrate their learning).”  (from—examples-definition-activities.html)

The idea of differentiation is that some of our students are different—e.g. some are second language learners, some have learning disabilities, some are just behind, some are “gifted” etc. These students need either different lessons, or more commonly alterations and adaptations of the lesson that the “regular” students get.  This is seen as an advance from the one-size-fits-all structure of many lesson plans and textbook lessons. In fact most textbooks now come with suggestions for such adaptations. All of this sounds very good—we are taking seriously that not all students are the same and helping teachers to support such students.


But is this really such a good idea? I am going to critique this theory on a few levels. One, is it realistic? Elementary school teachers already have a massive job on their hands making lessons for each different subject area. Now they have to multiply that by how ever many different types of student they have. An adaptation for their second language learners. But can they assume all their second language learners need the same adaptation? One might be a new comer, another an intermediate speaker. Then there are the students with learning disabilities. But again—each one of these is likely to have a slightly different disability. And we move on to the “gifted” students. And what if some students fall into more than one category? When and how does a teacher find time to create all these adaptations and manage them?

There is also the factor that this approach singles out some students as “normal” and others as “different” needing differentiation. There is a lot of evidence that labels often become self-fulfilling prophesies for students. It also sends a message about normalcy to both groups. How will this effect the self-identity of these students, and the view of them by other students?

So, should we return to the one-size-fits-all approach so as not to single out students and to make teachers jobs easier? This is one of those false choices. These two choices assume a teacher (or textbook) centered approach to learning.

Another option, that progressive educators have been practicing successfully for over a century is to have lessons and teaching units with activities that are open-ended and allow students to find their own approach that meet their individual interests and abilities while still helping them develop necessary skills and abilities. This actually mirrors how people have learned effectively outside of school since time immemorial where people of differing abilities, backgrounds and interests all work together on common tasks. Thematic instruction, project-based approaches often fit this. One common example of this approach is the Writer’s Workshop. All students work on writing a class book, maybe even within a certain genre or topic, but they all get to write what they want within that framework. They each can work at their own pace and ability, and the teacher and their peers all help each other refine their writing.

Larger projects can use this approach as well. Such a project might be the study of an ancient civilization. It might be the investigation of one’s community. It might be an examination of the physical environment. In this approach students investigate, build, write, read, observe, and create around the theme, each at their own level.

I am not going to say this approach is not a lot of work for the teacher, but it is not about creating lots of individual lessons, but rather creating a climate for learning, making the materials and resources available, and then knowing how to support each student to do their best within that framework.

To see a wonderful example of this approach at the elementary level see the video “We All Know Why we are Here”


The more I think about education and learning, the more I see relationships as the key to what really matters. If I think about all the movies I have seen about “great teaching,” both fictional and those “based on a true story,” while the actually teaching going on in them varies enormously, what they all have in common is a teacher that builds caring strong relationships with their pupils, from “To Sir with Love” “Up the Down Staircase” of the 60s, to more recent movies such as “Mr. Holland’s Opus” and “Dangerous Minds.” But of course that portrayal could just be the license of the writers and directors.


But I would say I have found the same in my experience as a teacher. I worked with teachers with many different pedagogical approaches. If you have read my previous columns, you will see it is clear I have strong beliefs abut which are more effective. However, the most consistent thing that I noticed of teachers that appeared to me as more effective was that those teachers all had strong relationships with their students. The students knew their teacher expected them to learn, and was there to help them succeed in doing so.

It was really much for this reason that I decided to become an elementary rather than high school teacher. I did not see it as possible to really build those relationships if every hour I had a new group of students. With elementary school kids I had the same ones all day long. (It is also a reason I have never liked “regrouping” with other teachers—I never saw the trade off as worth the loss in knowing my students fully).

One anecdote. At one place I taught, we were using the Reading Recovery program for our struggling first grade readers. Reading Recovery is a strongly researched based program giving intensive support to the lowest readers in first grade, based on some of the best research of learning to read, with a strong research record of its own, and all the practitioners of it have to be credentialed teachers who have gone through an intensive training in the model. However, as a second grade teacher, my struggling readers did not qualify. So instead we used instructional assistants, who had a rudimentary training in more traditional phonics approaches to work with them. I would argue that second graders who are still struggling with reading are probably actually more difficult candidates, as they have a longer history of failure to overcome.

Yet, in the decidedly non-random and small sample that this consisted of, my instructional assistant succeeded with every one she worked with to at least getting them to the point of breaking the code in learning to read. The same cannot be said of the Reading Recovery program that had about a two-thirds success rate with our students. I attribute it to the strong relationship she built with each of them—letting them know that she believed they each would and could learn to read.

This, maybe, is what worries me most about many of today’s’ educational reforms. They make those relationships more difficult. Scripted curriculum, larger classes and school consolidation. use of technology for instruction, and worst of all, the tactics of fear—trying to scare teachers and students into doing a better job. Each of these, in a different way, makes it slightly more difficult for teacher and students to develop strong relationships.

I am about to embark on teaching an all on-line teacher education course. I will see to what degree this mode allows for and interferes with such relationships.

Assessment in California Teacher Education

(This column is adapted from a talk I gave at the University of Kyoto in January, 2012)

Those in the field of assessment often refer to two important standards that assessments are expected to meet, reliability and validity. Reliability meaning that the same results would be obtained if the assessment were given again, or if a different person was scoring the assessment.

Validity means that the assessment actually measures, assesses, what it claims to be measuring/assessing—and whether it predicts how one will perform in the future (Ormrod, 2005).

One type of validity is “face validity”—that is, it is accepted that the assessment actually does measure what it claims to measure, without needing statistical proof that it does. The road test portion of the driving test might be an example of that: We can easily agree that if we want to know if someone knows how to drive, we can sit in a car with them and watch them drive. Now, what constitutes good enough driving to pass the test, that is where things might get more difficult to agree. Both how good is good enough, and which things matter most, e.g. how well the student parked, used turn signals, obeyed signs, and how much should each count, can be controversial.

Other tests need to have their validity demonstrated. The paper pencil portion of the driving test might be one of those. Do we have any evidence that those who do better on the written portion are actually better drivers?

However, while we accept that the road test has more face validity, we might wonder about its reliability, the possible subjective nature. The written portion is more reliable, you either filled in the correct bubble/answer, or you did not. However, on the driving portion, maybe the traffic conditions were more difficult when you took it than when your friend did, maybe one instructor is tougher grader than another. Maybe he had a fight with his spouse that morning! Despite these shortcomings, we accept the trade-offs as worth the advantages of such an authentic assessment. A built in safeguard is the opportunity for second, third, and as many opportunities as needed to retake the test.

It is easy to create paper and pencil assessments that are reliable and easy to administer. However, how one’s score correlates to real life application of the knowledge or skill that the assessment is designed to measure is harder to determine. Some, such as myself, argue that there is a built in tension between reliability and authenticity. Real life tasks and situation are by their nature not standardizable: Conditions vary, there is ambiguity, and there is more than one right way to approach a situation or problem. Creativity, a very important human trait, cannot be measured, and one’s ability to act effectively in novel situations is also by its nature not standardizable. Therefore to assess one’s ability to use one’s knowledge and skills in real life situation is likely to have a degree of unreliability, unpredictably.

Furthermore, what one person views as good enough, as quality, in most real life applications also varies. A movie I thought was well acted and crafted, my best friend thought was poorly acted and rang false. And that is in movies made by highly paid seasoned professionals! Multiple publishers initially turned down a number of best selling classics in literature.

Compulsory public schooling in the United States was instituted at a particular point in history, with other changes and advances happening. Part of that was the belief in scientific experts and the new field of psychology as a science rather than philosophy, and the invention of standardized intelligence tests. Americans often want to find the one right way (Smith, 1988). Americans are known for their obsession with measuring everything, and putting numbers to everything. This has played into schools in the forms of tests that can be reduced to numerical scores, and a belief that if everyone takes the same test at the same time in the same way, and test is designed by outside experts, it is therefore objective.

Critics of the standardized tests of today point out the shortcomings of such tests: they don’t really have reliability at the individual level, they are culturally biased, and their inauthenticity—their lack of actual validity in terms of measuring any important, useful skill, ability or knowledge beyond the school house walls. They also object to the indirect influence of these tests in encouraging the teaching of discreet skills and rote knowledge that is quickly forgotten once the test is over (Hursh, 2005; Kohn, 2000; Meier, 2002; Ohanian, 1999).

However, it must be remembered that standardized tests were put in place in part as a seemingly fairer alternative to an aristocratic system, where social position and money was what decided who got into the best schools and got the best jobs. Standardized tests were seen as scientifically objective tests, and therefore gave an equal chance to all. One could rise by one’s merit, not relying on family name or wealth (Smith, 1988).

What authentic assessment is proposing to do is to let people show what they know and can do based on merit, but also more accurately than standardized tests reflect the skills and abilities the person should have by seeing how they apply that knowledge in a realistic situation.

Of course even “authentic assessment” is always a matter of degree. Authentic assessments are generally applied in somewhat contrived or hypothetical situations. In school situations it is rarely practical or even possible to have students demonstrate in the real life situation, and even authentic assessments give us just a sample of the full skill being assessed. To go back to the driving test example, even on the road test, not nearly every possible driving situation is encountered. The driver is asked to carry out a predetermined set of maneuvers at the direction of the tester over a relatively short period.

A large issue for authentic assessment is to overcome the issues of “bias,” which is really an issue of reliability—would a different scorer give that person the same score? One way to address this is through multiple assessors. For instance, at some high schools that use portfolio or exhibitions for graduation, such as was developed at Central Park East Secondary School, they use multiple assessors, while also having outside experts examine their system, and watch it in practice to help them improve and refine it (Gold, 1993; Meier, 1995; Meier 2002).

Another common system to obtain more reliability that is used in authentic or performance based assessment systems is to have scorers be calibrated. A set of benchmarks are set up—examples of the performance assessment carried out at different levels, and the scorers are first trained on what qualities to look for, and then they are asked to score these benchmark examples to see if they give them the expected scores. In theory, only when they can consistently give the expected scores are they considered calibrated, and therefore the scores are considered reliable.

I will now discuss efforts in California to bring a more authentic, yet standardized, assessment in a systematic way to credential teachers.

California teachers are given their credential based a variety of factors. Some have been (and still are) standardized paper and pencil tests. However, as we have discussed, there is a sense that these are not good indicators of how well they would actually teach. These tests are used as measures of minimum knowledge of basic skills. On the more authentic side these candidates are placed in classrooms to learn to teach alongside practicing teachers. In most teacher education programs in California, this is a semester long placement. In some, such as where I currently teach, we require two semesters of student teaching. However, some worry about the standards of those assessing that experience. Were they tough enough? Are they consistent? There is no standard set of measures for that experience. The same could be said of the other criteria, that they pass their college courses to become a teacher. Were the standards from one program to another, even one class to another, consistent (Chung, 2005)?

The legislature of the State of California decided to institute a performance based assessment system on top of the other criteria to both provide an authentic, yet valid and reliable way to measure whether a candidate was ready to become a teacher.

Linda Darling Hammond of Stanford University led a consortium of universities with foundation support to develop such a system, called Performance Assessment of California Teachers (PACT) (another similar system was also developed by the Education Testing System, the CalTPA). In the PACT assessment teacher candidates develop a 3-5 day lesson plan in mathematics or reading, they carry out the lessons in their placement, and videotape those lessons. They document all of this, providing a detailed description of the context where they taught the lesson, describing the school, the classroom and what they know about the students. They provide the lesson plans, and some discussion about those lesson plans. They reflect on what happened when they gave the lessons, what changes they made along the way, and what changes they might make if they were to give these lessons again. They select a 20-minute portion of the video for the portfolio, and discuss what is in that portion. They also provide examples of the assessment used in the lesson from three students of varying abilities. They discuss what they saw overall in reviewing the student assessment, and what they learned about the three students in particular.

This portfolio is then read and scored on a set of 12 rubrics. Several rubrics address issues of planning, several look at the execution of the lessons, several others look at the issue of assessment. The issue of how the lessons helped student access and learn “academic language” is also assessed by two of the rubrics. The people who score these assessments go through a two day scoring and calibration training, and must re-calibrate every year.

In practice, despite the training and calibration, there are still sometimes disagreements (if a student fails, it automatically gets scored by a second scorer. Randomly ten percent get two scorers to check reliability). While in the large majority of cases we probably score the candidates similarly, there are cases where we have scored them quite differently. In such a system, there is room for interpretation. If the rubric asks us if the lesson was appropriate for the students, or the teacher gave clear feedback, what one of us interprets as appropriate or clear may not be the same as another.

These are the trade-offs for a more authentic system. For everything we do, that we add, something is also lost, traded. On the positive side, in my institution it has meant that we have had dialog among the faculty about creating a more cohesive experience for the student. However, as many high stakes assessment systems can do, preparing our students for the assessment itself has taken significant university class time, time that used to be spent on content. In that way students may be losing out. Some also wonder to what extent is the ability to write well, to theorize being assessed, rather than the actual ability to teach. Though assessors are told that the writing itself is not being assessed, it is for the most part a written assessment, albeit of a performance (along with the short video clip).

It is certainly a system that is more uniform than what was in place before. From my experience with the system, it does appear that the stakes have been raised for student teachers. Are the teachers who have now gone through this system, better prepared? Are we better at keeping out unprepared teachers, while not excluding prepared ones through this system? That is a much more difficult question to answer for which there are no solid “facts.”

The problem in the United States is that people are looking for a foolproof “fair” system. The attempt is to avoid human judgment, which by its nature full of biases and well, judgment! Standardized tests, paper pencil tests, offer us the illusion of avoiding judgment, but it just moves such judgment to the creator of the test. It offers reliability often at the cost of meaningfulness.

In the United States we rely on human judgment for our criminal justice system, our courts—very important high stakes decision—and while mistakes are made, maybe even often, it is seen as better than the alternative. Authentic assessment systems at heart require the same faith. A faith that the trade-off of allowing for human judgment is better than the reductionism required to assess in a standardized form. I believe we need to bring more of such human judgment back to our educational system.


Chung, R. R. (2005). The performance assessment for California teachers (PACT) and beginning teacher development: Can a performance assessment promote expert teaching practice? Stanford University. Proquest dissertations and theses, 598p.
Retrieved from http://search.Proquest.Com/docview/305434959?Accountid=10355 Unpublished Dissertation, Stanford University.

Gold, J. (Producer & Director), & Lanzoni, M. (Ed). (1993). Graduation by portfolio: Central Park East Secondary School [Videotape]. New York: Post Production, 29th Street Video Inc.

Hursh, D. (2005). The growth of high-stakes testing in the USA: Accountability, markets and the decline in educational equality. British Educational Research Journal, 31(5), 605-622.

Kohn, A. (2000). The case against standardized testing: Raising the scores, ruining the schools. Portsmouth, NH: Heinemann.

Meier, D. (1995). The power of their ideas: Lessons for America from a small school in Harlem. Boston: Beacon Press.

Meier, D. (2002). In schools we trust: Creating communities of learning in an era of testing and standardization. Boston: Beacon Press.

Ohanian, S. (1999). One size fits few: The folly of educational standards. Portsmouth, NH: Heinemann.

Ormrod, J. E. (2005). Educational psychology: Developing learners (4th ed.): Prentice Hall.

Smith, F. (1988). Joining the literacy club: Further essays into education. Portsmouth, NH: Heinemann.

Computer Use In Public Schools

(click here for the full published version of this article)

In this column I am going to present the results of a small study I conducted with some Masters in Education students in regards to computer use in schools. Computers have become ubiquitous in our society. Shopping and planning travel arrangements, social networking and entertainment are often done through the computer. Jobs from mathematics, the sciences, and even the arts usually require creative and skilled use of computer applications. Groups such as MoveOn have even created new forms of political organizing and activism. Blogging and other Web2 applications are changing the way people get their news. Most people now agree that our schools should therefore be preparing students to be technologically competent.

In considering what such a shift might mean for education, educational theorists interested in the topic have tended to predict one of two types of changes. Some have focused on the ability of the computer to empower students. Others have focused on the power of the computer to effectively and efficiently deliver instruction.

When personal computers were first invented, some claimed that computer technology would transform schools and education as we know it, bringing on new ways of teaching and learning that were not possible in the past. They argued that computers made the traditional role of teachers as lecturers—the sage on the stage—obsolete. Others, while not claiming the inevitability of such a change, promoted the idea that computers could be used to make constructivist, learner-centered teaching easier. With the use of such computers, teachers can and should now play more the role of guide, coach and facilitator.

Another view has been that computers would or should transform schools, not by changing our basic paradigm of learning and instruction, but as a more effective and efficient way to deliver instruction, or at least as a strong supplemental aspect to the curriculum. The idea of using technology for programmatic instruction goes back at least to the 1960s. According this view, the promise of programmatic instruction is now possible with the powerful computers of today. Computers can now assess the individual learner, and tailor the instructional pace and problem presented to that student. No longer will each teacher need to be the expert in instructional techniques, since it will be programmed into the computer. Once we have identified the steps, any skill can be taught most efficiently and effectively this way. While this approach could significantly alter the teacher’s role as deliverer of instruction or information, it does not substantially alter the role of the student.

As of yet, there is not much evidence of either of these becoming realities. There are many individual examples of teachers using computers in creative ways that do speak to the claim of a more constructivist paradigm (see Coppola’s book Powering Up for an example of this). On the other hand, these appear to be the exceptions that prove the rule (read Larry Cuban’s book Oversold and Underused for a full treatment of this). While there is some evidence that many schools are using computers in ways that match the programmatic instructional idea—that is for teaching basic skills—there is of yet little evidence that it has improved learning beyond small-scale examples.

Another issue that has concerned many in terms of technology use is the digital divide. Not surprisingly, those with more money and resources, and those of higher socio-economic-status, are more likely to have computers at home, and use them more powerfully. Potentially, public schools could be the place where those with fewer resources could get that access. However, often resources at schools mirror the resources of those in the community. Therefore, schools, rather than leveling the playing field for disadvantaged students, may exacerbate those differences.

In looking at 16 local public schools, evenly divided between primary and secondary, and between schools serving predominantly low-income or more middle and upper income students, we asked the following questions:

  • For what purposes do the students use the computer technology at the schools?
  • In regards to the above question, what differences do we see among schools? Do socio-economic factors correlate with those differences? Is the age level of students a factor?

In particular, for our analysis we divided computer use into two basic categories. On the one hand were uses we saw fitting more of the constructivist paradigm, where computers were tools the students use to enhance productivity and creativity. On the other hand were uses with skill and drill programs, or as assessment tools of basic skills, fitting the programmatic instruction model.

Our study did find that computers were used differently based on the socio-economic make up of the student body, and based on the grade level of students served. There did not appear to be any consistent factor related to quantity or quality of hardware available to students. However, schools serving middle- and high-income students were more likely to have well-trained computer technicians and teachers to help make the computers more useful. Schools that served low-income students mostly used computers for drill and practice type programs and as an assessment tool. Schools serving middle- and high-income students were more likely to be using computers in ways that built computer literacy, though still not to any large degree. We also found that high school students were more likely to use computers in ways that built their computer literacy skills than elementary students.

The data suggest that schools serving low-income students use the computers mostly for drill and practice due to pressures of the standardized testing. Most of these schools are Program Improvement schools. As such, boosting standardized test scores is their top priority. They are then likely to use the computer programs designed as test preparation. Schools serving middle- and high-income students, not being under those same pressures, may feel the freedom to use computers in ways that are more creative.

In regards to high school versus elementary student use, the findings suggest that high school students are already likely to have basic computer literacy skills, allowing teachers to assign more creative projects without having to spend much time teaching how to use the technology itself, especially among middle- and high-income student bodies.

This study suggests that if we want to create equity for students from all backgrounds we need to rethink what opportunities we provide for low-income students to use computers in ways that prepare them to be able to use them in as powerful ways as their more well-to-do peers.

Given that high SES students tend to have more opportunities and access to powerful technology at home, and that high SES students have more opportunities to use computers in ways that build computer literacy, current school practices are likely to exacerbate rather than mediate the digital divide between low and high SES students.

To change such practices a serious reconsideration of what it would take to really bridge the gap needs to be undertaken. Such an examination is unlikely at most schools serving low-income students, given the pressures on district administrators, principals, teachers on down to students, to raise short-term standardized test scores. With such pressures, almost everything else becomes at best secondary, if considered at all. Such pressures are only increasing under the current Federal policies.

It would also take an enormous input of resources. The real cost of having enough up-to-date computers, the software to use them well, the personnel to keep them running, and the professional development so that teachers would know how to use them effectively certainly does not exist in these times of economic crisis and education budgets cut to the bone.

These are real difficulties that all of us who are committed to equity face. Those of us who do work with low-income students are therefore forced to think of creative ways to overcome these difficulties. While it is true that all students need to learn to read and do basic arithmetic, it is not true that for some students this should be done at the expense of learning other things, including being powerful users of technology. The brains of poor kids do not learn and function differently than those of rich children. Therefore, we do not need to teach them in fundamentally different ways. Without being prepared with equal technological skills, this lack will be just one more division and barrier when these students leave school, leaving them less prepared not just for the world of work, but the world of social empowerment, and access to information to improve their lives and make informed decisions.

Even without a change in resources, it is possible to use technology differently than is now often the case. As the data showed, the difference in resources in the schools between types of schools was minimal. The real differences were in how they were used. These differences underscored an implicit or hidden curriculum. The use of computers as programmatic instruction treats students as passive recipients of knowledge and instruction whose job is to input the correct answer. The uses that the the International Society for Technology in Education (ISTE) standards promote, for example, ask students to be active participants in their own learning, using computers as a tool to create and convey knowledge. When these uses are promoted differently, for different types of students (which may coincide with the non-computer based instruction they are receiving), students come to view learning and the purpose of school in fundamentally different ways.

The argument for the need for this different instruction is, as mentioned earlier, the need to raise test scores. However, many schools have been effective using constructivist approaches to learning effectively with low- and high-income students alike. When we ask and support students to use their minds creatively and constructively, they not only do they do better on standardized tests of knowledge in the short term, but they also develop the abilities necessary to succeed in many arenas, in and out of school.

(click here for the full version of this article)

The Big Idea and Thematic Planning

In this essay I want to think aloud about approaches to planning and carrying out thematic instruction. Those of you who have read my previous essays know I usually have strong opinions on the educational topics I raise, be it policy, instructional pedagogy, assessment…. In this essay I am going to explore my conflicting thoughts on different approaches to such planning, particularly looking at idea of backward planning.

When teachers thought of doing a “thematic unit,” (and in the few places where they still do) they often thought about all the interesting activities that somehow would be fun or interesting connected to that theme. “We’ll study farms… I could take them to a farm. We could study different animals. We can look at the food chain. We can look at how foods are processed…”

Currently another approach is being advocated to replace this type of thematic planning. This newer approach is known generically as backward planning, or as promoted by Grant Wiggins and Jay McTighe, Understanding by Design. In this approach, one doesn’t start with a topic per say, but with a big idea or basic concept that one wants the students to understand. In science, this might be, for example, “the food chain” or “properties of solids,” or “the effects of humans on their environment,” in social studies, students it might be understanding revolutions. In this approach it is only after deciding on the central idea, that you decide what topic you might use to learn about that. Then after deciding the topic do we finally get to thinking about the activities that would help lead to an upstanding of the concept or big idea. We could study the food chain in ocean life, in the desert or other environments. In this approach we are studying the ocean, not so much to learn about the ocean, but as an example of the food chain. The topic is an example of a bigger idea, not the topic itself. We could pick from a variety of historical revolutions to understand revolutions in general. Once we know what main idea we want to get across, we think about how our topic and activities will help the students achieve that understanding of the big idea or central concept.

The backward planning approach may fit more easily into a discipline or subject area approach than a traditional thematic approach. The big ideas and concepts come from the what that discipline sees as core and important. It also may be easier to implement within a standards-based system. The big idea or concept can come from the list of standards that the teacher is expected to cover. It also keeps things focused. When one plans under this approach, one is constantly asking oneself, how does this advance the big idea or major concept? How does this prepare the student to be able to show they have this understanding? As one who prepares teachers to enter the field and works with others returning for further education, this has been an approach I have often advocated when I have students plan units and lessons.

Recently, I have been asking myself, what might be lost in this approach. I was brought to think about this in reading Deborah Meier’s January 2010 Column. In it she was briefly mentioning her school’s approach to thinking about a theme. What are all the types of connections that can be made to this topic? One thing leads to another. This approach has a more playful, spontaneous feel to it than the Backward Design approach. In the topic approach, the teacher has more freedom to see where things lead, where the students might want to go, to go off on tangents as the class or individuals discover new things or opportunities arise. This is connected to the second advantage which is connections. In the topical approach, one can look at how this topic connects to lots of concepts in different fields, area, disciplines. In studying ancient Egypt, we might look at Egyptian myths, the culture, fiction about Egypt, the historical implications, the science of the Egyptians, their influence on mathematics. This approach is more likely to integrate the disciplines. I like the sense of intellectual play (see my earlier column on play) that such an approach can encourage.

I just learned about an endeavor to transform teaching using this topical approach at a workshop. In this approach, called Learning in Depth, students are given a topic, such as apples, dust, or circuses, which will become their individual specialty for the rest of the k-12 school experience. “The aim is that students, by the end of their schooling, will know as much about that topic as almost anyone on earth.” The student will study this topic in whatever direction the topic and their interest takes them.

In the backward planning or Understanding by Design approach, many of those topics would be seen as moving away from the central concept that studying of it was an example of. In a topic based approach, the students become knowledgeable about a broad range aspects and connections within that topic. However, we cannot be assured that they will hit on the particular central concept that we or the discipline sees as key.

I do not mean to imply that even under the Understanding by Design approach one does not find tangents, or make connections to other areas. Quite the contrary, this approach encourages making connections. If the connections to other examples weren’t made, the topic would lose its power as an example of a big idea that would transfer to other examples. However, the connections would be focused on connections to the concept, not to the topic.

Really, in the end, maybe it doesn’t have to be either/or. Maybe sometimes we can take the Understanding by Design approach and gain from the advantages of that focus on developing a deep understandings of a concept. Other times, we can explore a topic and gain from the advantages of looking at a topic in all of its dimensions and connections to a variety of disciplines and aspects.

Best Practices

The term “best practices” has become popular over the last decade. For me the term is problematic in a number of ways. First, it leaves off the essential question: “best” for what? Despite statewide standards and the current move toward national standards, we do not all agree on the aims and purposes of public education. Far from it, as I have discovered every time I teach a new group of teacher candidates.

The other problematic assumption is that there is a best method for whatever our purpose is. While there are practices that are generally more effective then others, human beings and the teacher/students relationship, not to mention all the other contextual variables, are so complex that no one practice is likely to always be the best, if even effective at all.

Let us consider an analogy. Let us say I want to find the “best” shoe size, so I can provide all my students with the right shoes. I do a controlled study, and find that when I give size 10 shoes, more students have shoes that fit them than any other size. Now I can mandate that everyone be given size 10 shoes. But men’s and women’s feet are different you complain. Okay, I may need to do some differentiation. Women get a women’s size 8 1/2. How about ethnic groups? Mexican Americans tend to be smaller. Okay, Mexican-Americans men get a size 9…..

We can all see the utter absurdity of this. But this is what we are doing to our school children, especially to the most needy and disadvantaged school children. I spend a lot of time in a lot of different schools as a researcher and as a supervisor of student teachers. In schools that are considered “Program Improvement” under No Child Left Behind, I see teachers mandated to give lessons where every child is on the same page at the same time doing the same exercises, often not just in the one classroom, but in every class at that grade level. There is a pacing guide to keep up with. The students must move on, whether they got it or not (and do it whether they already know it or not). A few “differentiated” students may be allowed to get special help (by missing out on some other activity, or after school). Extensive data is kept on how the students are doing, with unit tests every few weeks that are diagnosed, often through sophisticated computer programs, Students’ scores get displayed in staff lounges (part of the data driven philosophy). Yet the teacher really cannot make much use of the data, since no matter what it says, they must keep to the pacing guide. This is seen as “equity” under NCLB. All children are afforded the same curriculum, the same instruction. After all, this curriculum has been designated as “research based,” since it uses strategies that the Reading Panel found to be most effective. We must have equally high expectations for all! Most of you probably think I am exaggerating. I assure you I am not. If you think so, find a school that has been designated as a “Reading First” school, and serves predominantly low income students. Maybe it is different in your state, but here in California, what I described above, I have seen over and over again.

The problem is that educational experts are being asked the wrong question: Which is the best method? Such a question was asked of the recent federal National Reading Panel—to come up with the best method for teaching reading. Textbook publishers created the materials that are used by “Reading First” schools, supposedly based on the recommendations of this Reading Panel. However, such one-size-fits-all thinking is equally absurd for teaching as it is for shoe size. Instead we need to be asking, what is the best way to support classrooms and teachers where each child will be best supported to learn in the most effective way? No two children are the same, and even the same child may need something different from day to day.

The best schools, schools that succeed with large percentages of students, are ones where teachers work together collaboratively getting to know the students. In these school they devise curriculum that allows all students to find ways into it, no matter what their learning differences styles and abilities are. These schools honor these differences, while expecting, cajoling, pushing, all students to do their best.

Play is the Work of Children

Two first graders come tearing down the hall and run into their teacher.
“Children, what was the head master just talking about today? Remember, no running in the halls.”
“But Mrs. James, we weren’t running, we’re horses.” At that, they galloped off into the yard.

This anecdote (much abbreviated and paraphrased here), comes from a talk by Michael Armstrong, author and former teaching principal, at this year’s North Dakota Study Group meeting of progressive educators. He used it to illustrate the essential aspect of playing with language. We learn language by playing with language. It is language that allows humans to be creative beings.

The title of this column is attributed to Friedrich Froebel (1782–1852), the German philosopher, and coiner of the term kindergarten, which literally mean child’s garden. Over the last couple hundred years, other philosophers, psychologists and educators have come to the same conclusion. These range from theoretical giants such as Jean Piaget, Lev Vygotsky and John Dewey to modern practitioners and thinkers such as Benjamin Spock, Vivian Paley, Eleanor Duckworth, Deborah Meier, and Michael Armstrong, listing just the tip of the iceberg.

What these and many other educators and psychologists have discovered is that play is an, maybe the, essential manner in which humans learn. In fact not only humans, but many mammals learn the important skills they will need for adult survival though play, as anyone who has watched kittens or puppies knows.

Piaget, one of the first to empirically study how children use play to learn, demonstrated that children use play to discover how the world around them works, to develop their schema, and organize the structure of their thinking. It is through such play that children develop intellectually. Piaget demonstrated that while with didactic teaching we can get students to parrot back scientific or logical principles, such rote learning does little to really help individuals internalize and use such rules and theories. According to his research, only when we can play and experience these things for ourselves, test out theories of how the world works, can we make these ideas our own and apply the principles appropriately in real world situations.

Vygotsky saw imaginative play as important beyond the logical-mathematical realm that Piaget focused on. Through his research, Vygotsky demonstrated that through play and imagination children try out new roles. They imitate, creatively, the roles they see of those who are older than them, the adults around them. They develop socially and culturally through this process. In all realms, through play children can go above their current level of being and development. Like an actor, play frees children to act in ways they may not yet be able to do outside of play. Play creates a safe place to task risks, try out new ideas, and take on new roles.

Benjamin Spock, the famous child psychologist of the 1960s, in reaction to rigid and controlling theories of child rearing that were popular at the time, advocated for the importance of play in children’s lives. He explained that children do not engage in play because it is easy, but precisely because it offers meaningful challenges. Human being like to be challenged and solve problems. Otherwise they get bored. It is through such challenges that children learn, learn at just the level that is appropriate to their particular level and skill, what Vygotsky would call, their Zone of Proximal Development.

Many other practicing educators have described the power of play in educational settings, such as long time elementary teacher and author, Vivian Paley, in her book A Child’s Work: The Importance of Fantasy Play (excerpt), Eleanor Duckworth (who worked with Piaget and Inhelder) in The Having of Wonderful Ideas, Michael Armstrong in Closely Observed Children and Elizabeth Jones and Gretchen Reynolds in The Play’s the Thing: Teachers’ Roles in Children’s Play, to name just a few. When you read these books, you see the joy and excitement that can fill classrooms when play takes a center stage. You see that rather than making school less “rigorous,” play actually makes them a more challenging and intellectually demanding place.

Deborah Meier has developed many schools based on the theory of play as central to learning. Meier’s small public schools of choice serve predominantly low-income minority students. Her work has documented how, when an entire school is designed with the importance of play as central, both in the literal sense of blocks and doll centers for young children, and the intellectual aspect of playing with ideas and theories for people of all ages (including the teachers and other adults) the levels of success for these students skyrockets.

As in the first anecdote, through playing with language, children not only have fun (though the importance of, and their right to, have fun should not be underestimated. Remember, this country was founded on the idea that we have the unalienable right of the pursuit of happiness), but they learn to frame their reality. The anecdote with which I started, in a charming way, illustrates how playing with language is a political tool—we use language to frame an issue. By defining themselves as horses, they find a way of framing reality where they are not breaking the rules—the rules apply to kids, not horses, and horses gallop, not run. These children are becoming powerful users of language, which we claim is a major purpose of education.

Play is a form of practice. But it is not rote practice. It is intellectual practice. Through play children are, in Piagetan terms, developing the schema of how the world works. Through play children have the freedom to discover how materials can be used, the limits and possibilities of those materials. Through play, in Vygotskian terms, children create a low-risk setting to move to the next level in their Zone of Proximal Development intellectually and socially. Through play comes discovery and invention. Through play comes power over language.

Despite all the evidence built up over several centuries, both theoretical and empirical, on the importance and power of play, it appears to be disappearing from the lives of children, especially their school lives, a place where they are compelled to spend 6 hours a day, 180 days a year.

This month, I observed one of my student teachers teach to a class of low-income Latino second graders. She did an hour and a half Reading First language arts lesson (a curriculum that is mandated in Program Improvement schools), following the lesson plan faithfully. The children responded dutifully and more or less competently to the assignments. Not once during that lesson were the children allowed to, much less expected to, use their imagination, to answer any question that did not have one right answer, nor did it allow them to explicitly use any experiences they might have had outside of school.

This experience is not the exception, but rather becoming the rule in schools serving low-income students. If you go into most public schools today, you will see that even in kindergarten (a term which is fast becoming an oxymoron), children spend most of their time sitting in seats filling in worksheets. When not doing worksheets they are engaged in other teacher directed activities. There is almost no time for children to choose their own activities or to freely explore materials. Where there is time to play, it is usually for those who finish their worksheets early, as a way to keep them busy.

Preschools, despite several recent large studies that show that children who attend developmentally oriented preschools outperform those in academically oriented preschools in later years (for more detail on this, read my March 2007 column), are more and more looking like the “traditional” first grades of my childhood—children spending their time doing teacher directed work sitting in chairs. Well-to-do parents are being convinced that only by engaging earlier and earlier in such “academic” work can their children win the race to get into the best schools later in life. Low-income parents are being told that only through such early preparation will their children have the skills that are required for entering kindergarten (when did we start expecting children to enter kindergarten with academic skills?).

There is also evidence that children have less opportunities and experiences engaging in imaginative play outside of school. Homework expectations are rising. Children, especially those of the middle and upper classes, spend more time than ever before in organized structured activities, such as sports, ballet lessons, or music lessons. While these activities are undoubtedly good for children’s development in many ways, they do not engage children in imaginative play. During the unstructured time they do have, children are spending more and more of it in front of electronic media, either television or computers. Neither television nor computer games stretch children to actively use their own imagination. At best they allow children to enter the imaginative worlds of the creators of these shows and games.

To those reading this column who are classroom teachers, I urge you to think about your own teaching. What are you doing in your classroom that allows children to use their imagination? Where can they express their own voice? What percentage of the day? Where are the spaces within the scripted curriculum that you may be mandated to use, where you can allow creativity and student voice?

To all of you, I ask, if you agree with the above critique, what are you doing to change the current system under No Child Left Behind that is driving out much of the space for imaginative work in schools? Preparing students to do well in high stakes standardized testing is often the reason teachers are told they may not allow creative lessons. The only national organization that is actively working to oppose the misuse of such tests for high stakes decisions is FairTest, working on a shoestring budget against the multimillion dollar budgets of the test publishers. At a minimum you can let your elected representative know.

One organization that is currently specifically working on saving the space for play in children’s lives is the Alliance for Childhood. Their site provides articles and links for more information and resources about play in children’s lives.

If you want to read some wonderful books on how to do such curriculum, you could start with any of the following books, some of which were mentioned above:

• Vivian Paley, A Child’s Work: The Importance of Fantasy Play
• Eleanor Duckworth, The Having of Wonderful Ideas
• Michael Armstrong, Closely Observed Children
• Elizabeth Jones and Gretchen Reynolds, The Play’s the Thing: Teachers’ Roles in Children’s Play

A useful teaching video is:
• Play, a Vygotskian Approach, by Elena Bodrova; Deborah Leong; John Davidson; Frances Davidson (Davidson Films).

For more scholarly academic work on the theory and evidence you might start with:
• Bodrova, E., & Leong, D. J. (1996). Tools of the mind: The Vygotskian approach to early childhood education: Merrill.
• Vygotsky, L. S., & Cole, M. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.

New City School

In my role as professor, I often supervise student teachers in schools serving some of the most disadvantaged students. For the most part, such education has been reduced to scripted curriculum that turns learning into a lifeless chore for students and teachers alike. I especially find it appalling to enter kindergartens, where I see these young children engaged in either whole class teacher centered instruction, or individual worksheets at their desks for the large majority of the day. All those things that the author of “Everything I need to know, I Learned in Kindergarten” learned are no longer being taught, in our rush to start our children on what we call “academics.” There is no time for creativity, children following their interests, learning to be social beings, or the engaging in the real work of childhood, which is play. Only the future will tell us what the cost of this will be to the next generation.

However, I actually am writing this column to talk about a school visit that renewed my spirit of what is possible! This was an impormptu visit last June to one of my favorite schools. On a vacation to visit friends and family in southern California, my wife and I were driving past Long Beach, when I suddenly blurted excitedly to my wife, “Can we stop here and see my friend’s school?” My wife agreed, somewhat reluctantly, as she has little interest in the topic of education, and few positive memories of her own schooling. I called up my friend, Stephanie Lee, the director of this small charter school in downtown Long Beach. She was more than happy to let us visit.

The New City School, or actually 2 sister schools now, are a pair of two-way immersion (Spanish-English) democratically run small charter schools (K-8), with multi-graded classrooms, and team teaching, using a project-based, constructivist approach to learning. With students from all walks of life, and a large proportion being from low-income Latino families, it represents virtually all the ideas I respect and admire in education.

Their new site, which I had not visited before, is a converted factory floor. The largely open space has mostly partial walls dividing the “classrooms,” with a couple of closed off rooms for particularly noisy or particularly quiet activities. The office is the old office of the factory, which looks down from above on the goings on below from its glassed in window, letting one see all the buzz of exciting activity. In walking around the school, we saw children engaged in projects of all sorts, from making life-sized anatomical drawings of the human body, to creating model cars. Each of the projects had some sort of “academic” aspect, often a written product, to demonstrate the learning of subject area content, and acknowledging that students did need to learn to become proficient in academic writing. Every wall was covered with creative works, arts and crafts. Wherever we went, students were busily engaged in work they cared about, and that had a purpose to them. My wife was amazed that this could be done in a public school, and was now glad she had agreed to come along. She loves music and dance, and was really excited to watch a group of kindergartners engaged in a hip hop dance routine as part of their elective in the afternoon as they learned their second language (dominant English speakers choose activities that took place in Spanish and vice versa).

While recognizing that standardized tests are not accurate measures of meaningful learning, they have managed to do what they need to do to keep their scores high enough to not fall under the thumb of state or federal sanctions. This is more than can be said for most of the schools I visit who use the scripted curriculums specifically designed to raise such test scores, yet still end up missing their targets. The success of The New City School has meant that they have a waiting list as big as the school (which was why they opened a second campus). Since under the requirements of NCLB and the State of California, the required scores are a moving target (each year the scores requirements are raised), how long they can sustain this is anyone’s guess. But for the meantime, this is one school where children are learning that education can be meaningful and worthwhile, and that it can allow for creativity, social learning, and be done in an environment where people treat each other with respect.

If you are in Long Beach, you might see if you can get a chance to visit! It restores my faith in what education can be. Maybe it could do the same for you!

Reading First

[Click here for the full version of this article as published in Critical Literacy Vol. 3, No.2]

A front page article in Education Week  (May 7, 2008) proclaims that “Reading First Doesn’t Help Pupils ‘Get It.'” This assessment is based on the U.S. Department of Education’s Reading First Impact Report. For those of you who are not aware, Reading First is a Federally funded grant program for “failing” school districts that use textbooks approved by the Federal government as being based on “scientifically based reading instruction.” What makes it scientifically based? That it presumably follows the advice of the National Reading Panel. The question becomes, why haven’t such programs shown effectiveness if they are scientifically based?

It turns out that where these reading programs are failing is in the area of “reading comprehension.” The report documents that schools using the program are increasing their use of the recommended practices. These programs do appear to help at so called decoding skills. However, the use of these recommended practices and these gains in decoding skills do not appear to translate into improvements in actual reading—that is, making meaning of text. Those who actually read the Reading Panel’s report should not be surprised. The fact is that the report did not have any evidence that the recommended strategies would help in reading comprehension. The only “scientifically based evidence” the panel found was that a limited amount of systematic phonics and phonemic awareness instruction would raise scores on tests of phonics and phonemic awareness for “regular” beginner readers(1). Just as in the evidence from the programs used in the field, there was no evidence in the Reading Panel’s report that such practices improved reading comprehension.

That advancement of such skills would raise actual reading ability is based on a theory of reading that is in fact quite controversial among reading researchers and specialists. Many of the foremost reading researchers, theorists and specialists have always contended that only a minimal amount of “skills-based” teaching is helpful, and that reading is most effectively learned through… reading(2)! (With support and help from those who already know how.) Moreover, the Reading Panel report found evidence for even this limited effectiveness of the skills-based approach only for students who were not shown to be problem readers, have learning difficulties nor to be second language learners. Yet these Reading First programs are often used for students of all types. In California, teachers are often mandated to use these programs in schools serving overwhelmingly Latino students whose dominant language is Spanish—in the name of scientifically–based curriculum. These skills–based strategies are applied in these programs for a much larger proportion of the teaching day than the research supports (more than a minimal amount is overkill—it’s like trying to pour more water into an already full glass). And at grades that the research has no evidence for effectiveness (the research on these approaches only looked at first or second grade). Students, whether they are already reading fluently for meaning or not, at all grade levels, are spending hours every week on decoding and phonics skills in these programs.

A friend of mine teaches kindergarten at one of these Reading First schools. The school is made up of over 90% Latino students. On the phone with her just the other day, she was telling me how they are constantly advised to examine the data on the students (another educational buzzword currently popular is “data-driven instruction”). She tells me that she is all for examining and basing instruction on data about her students. In fact the school spends considerable staff development time doing just that—examining the scores of the students so they know just where each student is. She can tell you exactly where each of her students measures up on all of the assessments which are carried out at the end of each six-week unit. Yet then she is told to keep all the students on the same page at the same time, and that she should not deviate from the script in the textbook (see, we’re not leaving them behind, they are on the same page as all the other students!). So what good does it do her to have this data? This practice ignores the research on the uselessness of teaching above students level of understanding(3). If you move on when students don’t get it, they certainly aren’t going to get the next lesson which builds on knowledge from the previous lessons, especially in a skills–based approach(4)! Her story of being mandated to use a one-size-fits-all approach is one I see and hear repeatedly from many of the student teachers and the experienced teachers I work with as a professor of teacher education, particularly those working with low-income minority children.

One of the worst problems of such programs is that they not only ignore the expertise that teachers bring to teaching their actual students—they try to prohibit it! Good teachers have always known that different children learn in different ways. Anyone who has taught knows that. Any parent with more than one child knows that. Good teaching is about figuring out that way for each student. If we really want to “Leave No Child Behind,” we have to stop tying teachers hands with scripted one-size-fits-all programs. We must allow them to do what they are trained to do, and spend a career getting better at—figuring out how the actual students sitting in front of them learn, and adapt their teaching to the students, not the other way around! (Which is part of the argument for small class sizes, but that’s another topic).

[Click here for the full version of this article as published in Critical Literacy Vol. 3, No.2]


1. Elaine M. Garan, “What Does the Report of the National Reading Panel Really Tell Us About Teaching Phonics.” Language Arts 79, no.1 (2001): 61-71.

2. Edward A. Chittenden, Terry S. Salinger, and Ann M. Bussis, Inquiry into Meaning: An Investigation of Learning to Read (New York: Teachers College Press, 2001). Gerald Coles, Misreading Reading: The Bad Science That Hurts Children (Heinemann, 2000); Kenneth S. Goodman, In Defense Of Good Teaching: What Teachers Need to Know about the “Reading Wars” (York, Me: Stenhouse Publishers, 1998); Stephen D. Krashen, Three Arguments against Whole Language & Why They Are Wrong (Heinemann, 1999); Jeff McQuillan, Literacy Crisis: False Claims, Real Solutions (Heinemann, 1998); and Frank Smith, Understanding Reading: A Psycholinguistic Analysis of Reading and Learning to Read. 6th ed. (Lawrence Erlbaum Associates, 2004).

3. John D. Bransford, Ann L. Brown, and Rodney L. Cocking, How People Learn: Brain, Mind, Experience, and School (Washington, DC: National Academy Press, 2000); and Linda Darling-Hammond, Barbara Low, Bob Rossbach, and Jay Nelson. The Learning Classroom: Theory into Practice (Burlington, VT: Annenberg/CPB, 2003).

4. James H. Block & Robert B. Burns, “Mastery Learning.” Review of the Research in Education, 4 (1976) 3-49; and J. Ronald Gentile & James P. Lalley, Standards and Mastery Learning (Corwin Press, 2003).