A Bubble Bath
My teaching experience in fourth grade between Fall, 1986, and Spring, 1989, put me in touch with transformative, emancipatory writing pedagogy. I lived, breathed writing instruction in and out of my classroom with my colleagues, my principal, parents from all grade levels, and area teachers in after school workshops. I was a participant at the Area III Writing Project Summer Invitational in 1987; Laura Stokes directed the festivities, and I was transformed. I was invited back for the summer of ‘88 to support the mission as a second-time-arounder, a designated mentor and writing group leader. Kristi Kraemer took the helm, and I was transfixed.
Sometime in Spring, 1989, I got a message that a woman named Carol Tateishi had called the school and wanted me to return her call.1 That evening Carol explained the intention of the State Department of Education to commission a design team of practicing teachers to work on a template or a skeleton of a reading test called CLAS assessing a student’s reading performance directly through collecting evidence from their writing and talking before, during, and after reading a passage. The idea was that the reading portion would be integrated with talk through a small-group discussion guide with a stand-alone individual writing prompt all thematically related. Could there be a talk score? We talked about whether I thought this sort of test would even be possible.
An orientation to the project to be held in Oakland was already on the calendar, and I could definitely make it. I had a ticket to ride beyond the bubble.
A Brief History of California State Education Assessment
State-level centralized public school testing has a long and storied history in the Golden State, and policy makers have not been shy about abruptly changing it. The relationship between property values and test scores was a hot button issue in the 1960s when ordinances forbade the publication of test scores. I read somewhere about a reporter in LA losing her job over publishing some test scores.
Over the past fifty years the state has legislated and implemented no fewer than five “systems,” depending on your definition. The most recent one, CAASP circa 2014, came about after California adopted the Common Core Standards. What has prevailed through all the systems is the sacred commitment to individual achievement scores—the this-is-how-you’d-do-if-you-were-shipwrecked-alone-no-cheating score or the this-is-all-you-not-your-school-not-your-experiences score, your choice.
In the 1970s the California Assessment Program (CAP) tested students across the state annually using a battery of standardized multiple-choice tests. In 1987 (see Peckham, 1987)2, English teachers finally broke the stranglehold of the multiple-choice standardized testing regime and opened the gates for a Trojan horse in the citadel of individual achievement scores. Writing instruction was enjoying a renaissance thanks to the Bay Area Writing Project, the active voices of writing teachers, and the maturing of research in composition theories. Teachers found little value in multiple-choice editing tests.
Enter a fine-tuned writing assessment program with eight genres of writing, banks of field tested writing prompts, and genre specific scoring rubrics. The CAP Direct Writing Assessment brought unprecedented coherence, the devilish angel, to teaching and learning writing.
The California Learning Assessment System (CLAS) emerged in legislation around 1987 to expand and deepen its predecessor, the California Assessment System (CAP). CAP as a multiple-choice system but with this avant-garde writing component seemed inadequate in 1990. The urge to morph the mission of assessment more toward making tools for teachers to use rather than tools for administration to hold teachers accountable was palpable. CAP’s writing assessment tools and strategies were to be embedded in the new CLAS Reading test design.
CLAS was ambitious and hungry, reaching for an authentic assessment system with performance tasks in math, science, and literacy and a robust portfolio assessment system. Giving teachers an authorized window into students in the act of learning (rather than after the fact) was new and enlightening. Here’s one teacher’s words interviewed in 1996 and cited again in 20143 on this point:
Another teacher reported a brand new respect for assessment:
Then suddenly, tragically, CLAS collapsed, vanished into sand, left us blindly here to stand but still not sleeping. In 1994, Governor Pete Wilson eliminated funding for CLAS in response to a public backlash and a lack of vision. Take him at his word. CLAS could not generate individual student achievement scores. It’s true that parents resisted CLAS with charges of invasion of privacy, indecent and controversial passages, and the like. But it was really about going beyond the bubble. It was honoring the bubble that ended it.
Soon, the state had a stopgap back-to-the-future system called STAR testing, and in 1999 legislation made clear that the state would take a hard line and test annually to hold districts accountable for scores. The public deserved objective evidence that its investment was producing the goods. No child left behind…
Gone were the days helping districts improve. Instead school test scores would arouse parents. Bad schools would be reconstituted. The tool was to be the Academic Performance Index (API) where schools were placed in zip code comparison bands and given a single number to sum up quality, a way to inform local parents about local school (in)effectiveness, a way to pit high-poverty schools against high-poverty schools in high-poverty comparison bands so no matter how under-resourced or factory-managed you were, your school could score well—relatively. The Matthew Effect was baked into the API.
State Responsibility for Public Schools Today
Common Core assessments have become a proxy for state-level assessment in California. Of concern to me is the expectation that districts are responsible for getting assessments up and running with little centralized state help, no vision or leadership, and no development and maintenance funding.
As near as I can tell, much energy is devoted to connecting assessments to fine-grained standards. I wonder how much energy is devoted to teachers’ collaboratively examining student work and co-creating learning environments tied to a coherent central hub with feedback and feed forward loops?
The following is excerpted from the same internal brief from which the above teacher quotes were drawn, published as a working document in 2014 pulling together California’s approach to new demands for the individuals charged with making the Common Core it happen as a driver of assessment.
CLAS was talking radically about school, district, county, and state level scoring of local portfolios. CLAS was interested in performance-based assessments in math and science, curriculum-embedded assignments of the sort the New Standards Project (NSP) in a multi-state consortium would design and field test from 1991-1996. CLAS was a full out effort to put teachers in the driver seat of school evaluation with professional accountability. Without robust political and economic support, CLAS would never work. Will districts ever have the human and capital resources to transform public schools county by county?
CLAS made people nervous. Fears emerged that high-scoring schools in affluent communities would see scores plummet with the new, invasive, overly sophisticated tests. Property values could fall. In fact, scores in affluent areas did fall, but CLAS never found legal footing to assert reliability and validity for the scores. How has Common Core dealt with this problem?
These issues were acknowledged in the 2014 brief. Responsible parties working within the Common Core framework knew that lack of political support and widespread understanding could spell the end of the Core. Reading between the lines, one empathizes with the agnostic briefer who knows full well the Core is a stopgap measure inching along a cliff.
More explicitly, the agnostic briefer in 2014 makes no bones about the assessment waste land we live in.
The Knight of Right Answers
CLAS went toe to toe with the standardized testing regime, a perennial foe for progressive educators. Benjamin Bloom in the 1950s created his taxonomies of higher-order behaviors precisely to expand the meaningfulness of such tests. As it happens, unfortunately, multiple-choice questions aren’t at all like catching these behaviors in the wild. Writing reliable items is a huge accomplishment. Science itself operates according to the principle of falsifiability—something is true until we have more data.
MC tests produce individual student scores on comparable forms of a test built from psychometrically equivalent items. A good item has several distractors and one right option that will fool some of the mudsills all the time. Learning to rule out distractions is a coachable art, though private coaching is expensive and taking classroom time for test prep is ethically questionable (I’m not agnostic). Answers to these items are right or wrong—not necessarily true or false, important or trivial—and we use them to generalize about teaching quality school wide.
CLAS could not produce both school-level and individual-student scores. The idea was to create evidence of learning for teachers to use as collaborators and co-creators of a powerful lived curriculum—a measure of how well intentional student opportunities to learn were working. Such an assessment would be made for teachers by teachers and examined by teachers as a foundation for coherent and purposeful instruction.
The new California Learning Assessment System—elegantly named ‘CLAS’—was setting sail for a place “beyond the bubble,” dealing a potential blow to the #2 Black Pencil industry. WYTIWYG was the CLAS mantra: What You Test Is What You Get. If it’s important enough to test, it’s important enough to teach.
In fact, standardized tests had been extolled as mission critical not because they measured what was taught, but because they did not. They came from the work of agnostic technicians hired by third-party disinterested private corporations like CTB McGraw-Hill in Monterrey.
These tests were billed as curriculum-neutral, unbiased toward any particular pedagogy or textbook series. They had an uncanny force, however, that drew curriculum and instruction to them like iron filings to magnets. The bubble, of course, referred to the empty circles on scantron answer sheets used as a collection device in a teacher surveillance system.
Envisionment Building
This post isn’t the place to go into details about the final design of the CLAS reading test. I plan to discuss this topic at some point. Suffice it to refer readers to Judith Langer’s work on reading as envisionment as a point of departure. When the CLAS design team met in Oakland for orientation, we began with Langer’s theory.
Instead of approaching reading passages as blueprints or programs that you follow carefully to build a meaning identical from bow to stern to everybody else, an approach that subjugates a reader to the status of a slave in a diamond mine (thanks Samuel Coleridge), Langer takes a view that brings the reader to the heart of the process, taking up a stance, entering a text, moving through it, stepping back, even stepping out and coming back, finally objectifying its meaning and locating it with reference to other texts and meanings.
Had California managed to instantiate its vision of authentic assessment in the life cycle of schooling, flawed as it was logistically and politically in the moment, I’m convinced three decades of collective experience would have revealed the obvious crack at the center of the factory model/metaphor, the deep and widely shared cultural model for public schooling that failed society when COVID-19 stress-tested the system. COVID showed us a public school system of individuals without a center. The CDC was stressed maximally, but at least public health had a beating heart with a Dr. Fauci at the federal level. Not so for public schools.
https://rethinkingschools.org/articles/taking-a-chance-with-words/
Irvin Peckham
The English Journal
Vol. 76, No. 8 (Dec., 1987), pp. 30-33 (4 pages)
Published By: National Council of Teachers of English
https://doi.org/10.2307/819410https://www.jstor.org/stable/819410
https://files.eric.ed.gov/fulltext/ED557612.pdf
https://test-learnermedia.pantheonsite.io/wp-content/uploads/2019/07/engaging-with-literature-a-video-library-grades-3-5Improving-Literary-Understanding-Through-Classroom-Conversation.pdf
https://files.eric.ed.gov/fulltext/ED397133.pdf
https://cacollaborative.org/sites/default/files/CA_Collaborative_CLAS.pdf
https://www.latimes.com/archives/la-xpm-1994-06-20-me-6218-story.html