Assessing School Writing: Navigating the Universe of Discourse
How well are students doing in their academic writing in school in 2024? Which students are better writers than others? What kinds of writing opportunities do they experience? What evidence of preparation for college level writing can be cited? Who does the American public turn to for answers? Who is minding the literacy store?
One has to admit: The irony is rich. The education system as a whole is terrorized by AI for its role in stoking literary armageddon when so precious little is known in practice about how well students can write, much less what they are doing with their literacy—which is not to say researchers haven’t been poking around. NAEP provides a suite of analytical tools with access to its database of surveys, etc., here. It’s far easier to buy a new car that actually has a dashboard than to find answers easy answers. It is after all quite a complex question.
With NAEP, however, one at least gets a sense of what is meant by “writing” in the assessment design as follows:
“Eighth-grade students writing at the Proficient level should be able to develop responses that clearly accomplish their communicative purposes. Their texts should be coherent and well structured… [with] appropriate connections and transitions. …[I]deas…should be developed logically, coherently, and effectively. Supporting details and examples should be relevant…and contribute to overall communicative effectiveness. Voice should be relevant to the tasks…. Texts should include a variety of simple, compound, and complex sentence types combined effectively. Words…should be chosen thoughtfully. Solid knowledge of spelling, grammar, usage, capitalization, and punctuation should be evident... There may be some errors, but these errors should not impede meaning.”
Translated, eighth graders should be able to write five paragraph essays their teachers can easily read and understand without being obnoxious or cute. Since 2013 twelfth grade students nationally are being held to the following clear and universal standard designed to cover the entire universe. From the Common Core:
“Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence. Introduce precise, knowledgeable claim(s), establish the significance of the claim(s), distinguish the claim(s) from alternate or opposing claims, and create an organization that logically sequences claim(s), counterclaims, reasons, and evidence.”
The developmental leap from NAEP’s expectations for eighth grade writing of a five-paragraph essay to the Common Core’s writing of legal briefs senior year is astonishing. If these statements refer to reality, at a minimum one might conclude that writing instruction is in chaos. The foundation laid in the middle grades is functionalist and communicative, but the pinnacle in high school is critical and argumentative. Given current political winds driving a back-to-basics regime in elementary school, the novice stage is orthographic and lexical.
*****
Charles Cooper, a UC San Diego composition researcher, and Beth Brenneman, a consultant for the State Department of Eduction (1990) wrote a report several years after California’s Assessment Program for writing emerged in 1986 to inform the State Department about its success. Cooper had been a thought leader for the project for several years. As we will see, this remarkable approach to systemwide assessment at the scale of California was capable of yielding robust, reasonable answers to questions about instruction, student learning, and writing. From two of the folks who devoted themselves to making CAP possible:
One doesn’t “write an essay.” One “writes in modes,” forms of being and thinking about and noticing possible worlds, tapping into representations of experiential reality, transforming thought into language, into a voice of fiction or fact depending on intentions and obligations. Listed among the “modes of discourse,” a pivotal phrase linking writing competence within action, interaction, and transaction in situated literacy events, are six modes (not text types, not rhetorical structures) of writing. Upon reflection, one concedes that assessing a “mode of writing” differs categorically from assessing a “product of writing” or even “the process of writing.” Modes suggest different products and differing processes for differing students. Modes increase the complexity of writing as a cultural practice and as an area of study.
Lee O’Dell (1981), a seminal researcher in the writing process research, grounded his perspective on writing, its teaching, and its learning, in his experience as a writer:
“Given our own experience as writers, it should come as no surprise that our students may benefit from being able to write under the same sort of circumstances we usually enjoy. Granted, we must ask students to work under some restrictions. They must meet certain deadlines, and, for various reasons, we may want them to do their writing during classtime. But even with these constraints, we can set up procedures that encourage students to explore their topic, consider their audience and purpose, and revise and edit their work” (Lee O’Dell, 1981).”
Around the late 1970s, stirrings were felt in California, a growing pressure to unleash the force of writing in the public schools. I began to sense it in the community colleges around 1981. California turned to Charles Cooper, a professor at UC San Diego working in composition theory, along with Irv Peckham in the 1980s to put together a self-referential model of writing which English teachers across the state already loved. Writing is what we say it is in the standards.
The significance of this moment must never be lost. In 1986 a writing assessment system authorized by law went live in California that had been percolating through the statewide network of teachers linked with writing projects across the state for several years. Teachers made this system. Teachers voices shaped the direction of the system from its earliest beginnings. Teachers selected from regions in the state based upon recommendations and interviews worked together for several years meeting monthly on weekends to write, field test, analyze, revise, write rubrics, find anchors, write commentary, report, educate the public.
The State Department of Education empowered teachers to write the Standards they would teach well before revising system assessments. Charles Cooper noted a sentence in standard 13 about a handful of “modes of writing” and fleshed out a map of an alternate universe of discourse. The State Department empowered a regional team of two dozen teacher thought leaders to write the test specs, to write prototypes for consideration by an Advisory Board. The spirit and ethos of O’Dell’s words, giving students the benefit of conditions we write under, meant that teachers working on the design understood their task to be finding a sweet spot between a teacher-centered and student-centered ideology.
*****
The California Assessment Program (CAP) when it was rolled out in 1986 riveted English department faculty in middle and high schools and focused their collective attention during department meetings on this revolutionary vision of writing across the universe. The cat was out of the bag: Assessment could drive instruction—and not straight off a cliff—if the test could be worth teaching to. What you test is what you get. Teachers were truly excited and were thinking about writing. The Area 3 Writing Project in the Sacramento region provided inservices in student-centered writing instruction throughout the surrounding districts. CAP set out to make a writing assessment system of assignments worth teaching to.
CAP was built from the ground up by California teachers who designed a functional performance-based assessment system, all of the DNA still intact in filing cabinets somewhere. Those teachers who wrote and field tested unique prompts aligned with a dozen different writing genres went on to become chief readers or table leaders at regional scoring sessions. This program was unapologetically “teach to the test” and ended up having perhaps an unimaginable impact on writing instruction and learning.
Murphy (2001) reported on the attitudes of teachers during CAP implementation and then after the return of multiple-choice tests. She wrote the following in her report:
The breadth, depth and clarity of the CAP message was received and implemented in classrooms by English teachers across the state, I think because it opened spaces for students’ personal experiences, observations, and voices to enter compositions—even rewarded aesthetic touches, experimentation, and nuanced styles. It explicitly spoke about building communities of writers, giving and getting feedback, and publishing student essays.
That teachers loved CAP is indisputable (cf: Murphy, 2001). By contrast, unfortunately, today’s vision for writing instruction is perhaps more narrow than it has ever been, perhaps even more troubling than the five paragraph essay. It’s as if the Common Core set out explicitly to close the door on writing across the universe:
To respond analytically to literature? Wow. I’ve not looked systematically for data to characterize teacher attitudes toward the Common Core ethos of writing, but my anecdotal sense is some high school English teachers are ambivalent at best, particularly the strongest writing teachers, some of them even feeling guilty about teaching the hallowed persuasive essay to the exclusion of joy and intrinsic motivation.
Any after glow from CAP dissipated quickly. The Core would not put writers in strict jackets until 2013, but the closing of the universe began by 2001. In this quotation, Murphy (2001) contrasts the high impact of CAP on writing teachers in 1988 vs. the diminished impact of the multiple-choice standardized test which replaced CAP even before NCLB wrought its destruction:
“…[A]lmost 50% of the high school teachers who responded to the 1988 survey indicated that CAP had a lot of influence on their teaching, but only 26% of the teachers who responded to the 2001 survey said that the SAT9/STAR had that amount of influence. Recall that most (96%) of the teachers who responded to the 1987 middle school survey about CAP thought that students in California would write more, and 94 % thought that the English curriculum in their school would be strengthened as a result of the test. Clearly, the data suggest that testing things that are relevant and worth teaching and involving teachers as professionals in the assessment process may be the more effective way to use assessment in educational reform” (Murphy, 2001, Journal of Writing Assessment).
*****
One California Middle School, which had been reconstituted, made its way out of the legal and financial hole it was in and took advantage of the CAP vision to rebuild the school. For the uninitiated, reconstitution takes place when the state declares that all of the faculty and administration who currently work at a school must be fired. They can apply for a new job, but they have no right to it.
The story of James Lick Middle School is fascinating in its own right, and I will discuss a little of it as it intertwines with the story of CAP. CAP contributed a collection of modes or stances or focuses a writer takes depending on the writer’s intentions. In other words, as the writer’s motive clarifies, the writing process from the writer’s perspective regulates the writing process in the same way that predictions get more narrow as more text is processed. The following screenshot from a paper published by Cotton (n.d.) lists the modes of writing CAP used to test eighth graders:
The subject of each pair of sentences following the heading in this list begins with “the writer”—not the teacher—doing something. These tasks engage the writer in thinking about the situation, reasonable actions to take to create a satisfying text, clarifying the cognitive style to invoke, how to present one’s self to a reader, how to gather details to report or describe, how to arrange the parts—everything the writer is going to need or face to do the work well.
These writing assignments have a built-in advantage for community building. The obvious factor, sharing personal experiences in a mentored classroom, can help learners develop bonds of trust and respect among peers as writers and as people. CAP thought leaders were scholarly experts and researchers in composition theory and understood writing as sociocultural activity best learned through participation in a writing community. Unlike the Common Core, which often emphasizes the single author in isolation involved in a solitary pursuit of textual evidence to support a claim, CAP fostered a peer-engagement strategy that took advantage of the power of local audience as an engine for publication.
James Lick teachers had access to intense and effective professional development through a Writing Project site. Linda Darling-Hammond recently published a paper concluding that the National Writing Project and Reading Recovery are the only two stellar examples of formal professional development for teachers that have stood the test of time with compelling evidence of efficacy. What happened at James Lick burnishes the credentials of the Writing Project model:
“During the summer of 1990, ten of James Lick's 40 certified staff participated in the six-week summer session of the U.C.-Berkeley-based Bay Area Writing Project (BAWP). In operation since 1972, the BAWP emphasizes an instructional approach that gives attention to all phases of the writing process--prewriting activities, drafting, revising, editing, and publication of written products. This way of teaching writing contrasts with the older, product- oriented approach that focused on the [end product].”
*****
How did John Lick Middle School do? Remember that the teachers were teaching to a test worth teaching to with high-quality professional development experience participating in the Writing Project. Here’s the result:
“The California Assessment Program (CAP) of the California Department of Education rates writing samples of eighth graders annually. Ratings are assigned for rhetorical effectiveness and use of writing conventions for each of the eight types of writing identified above. In 1989-90, the CAP scores of James Lick's eighth graders were a remarkable 56 points higher than those of the previous year's eighth graders--an increase which translates to one full point on the six-point holistic writing scale.”
That Was Then, This Is Now: The Impact of Changing Assessment Policies on Teachers and the Teaching of Writing in California
SANDRA MURPHY
University of California, Davis
The objective of this study was to investigate the impact of changes in assessment policies on teachers of writing and the teaching of writing in California. Surveys of middle school and high school English teachers conducted in California in 1988 and 2001 when two different accountability systems were in place provided the data for the study. Results indicate that both of California’s recent assessment systems have influenced what and how teachers teach, but in very different ways.