You got it. I’ve been working on this one for a couple of months. I jumped the gun a bit—a few slippery syntactic spots—but I wanted to finish it by New Years, I appreciate you, Matt.
Terry, it's good! You make an important connection between working memory, writing, and portfolio assessment. For writers who struggle with long term memory, relevance is important, but alone even that might not be enough. Portfolio assessment "holds things still", especially artifacts of the writing process + several pieces over time to aid the reflection process.
The portfolio framework implemented over a full school year that I studied in my dissertation extended to reading and writing in middle school English classes. I suspected that writing achievement would be significantly better in the portfolio classrooms than in the traditional classrooms, but I wasn’t so sure about reading achievement. In fact, the reverse turned out to be true. Students engaged in portfolio pedagogy posted significantly higher direct measures of reading at years’ end. There was no measured difference in writing achievement. I’ve done a lot of thinking about this finding over the past 20 some years. I think writing growth occurs more slowly than reading growth. I’m not seen any longitudinal research on portfolio pedagogy in either reading or writing. Research is needed to understand developmentally what happens when reflective analysis in the context of goal setting is taken up seriously as a learning outcome. Periodically “holding things still” over a long time period—I hypothesize that reciprocal influences across reading and writing experiences will be different as children’s cognition and memory grow. Motivational orientations will also become more complex with increases in intrinsic motivation gradually showing up—the reverse of what we now see.
That is interesting. You must have had to consider a lot of factors with this study, Terry. Writing as a more lagging measure makes sense. And what a nice surprise to see reading getting a boost!
When I examined our school's use of portfolios for only writing assessment, we saw almost double the growth in classrooms that were committed to the portfolio process vs. past efforts with portfolios.
These classrooms that found more growth:
- posted a student writing artifact every six weeks in a digital app that shared out to families
- had the student record audio of their structured reflection about the writing they chose to share
- positioned the teacher as a facilitator of reflection vs. a "giver" of feedback
You did a nice job with data collection in terms of breadth. Six traits is a good place to start. Think about structured ways to capture children’s reflective analysis on their reading. Also, consider examining questioning from the teacher-student perspective but also from the student-teacher perspective--questions going two ways. You could do a nice qualitative study of the videos of students engaged in reflection on their writing. Once you start planning your dissertation, I’m looking forward to giving you my feedback. The field needs more Matts willing to dig into research methodology and use it rigorously to lead a school in its day to day work.
See? There are likely important developmental differences. It may be that changes in reading competence and sophistication happen more slowly in the earlier grades while writing changes are more dramatic and visible. I wonder what happened with reading when you looked at portfolio pedagogy in your school? I know there is cognitive and linguistic spillover, a synergy between reading and writing... Part of the problem, too, is the sensitivity of measurement tools. For example, it’s always seemed problematic for me to use a standardized multiple choice measure of achievement to determine growth over periods as brief as a year.
Good points Terry. I continue to be startled from time to time when I connect with a 1st or 2nd grader around their reading or writing in the fall and then in the winter. "Are you the same kid?" :-)
And yes as they get older and growth seems hard to tease out in their literacy artifacts. Maybe more focus on the engagement and identity growth in the middle grades?
I think so. It’s reasonable to assume, for example, that younger kids thrive better with a mix of approval in the form of social rewards and clinical feedback from teachers. Middle schoolers based on what I found do not exert effort for social approval. Learning styles in regard to motivation are tied to developmental identity, and literacy learning is very personal. Can you imagine a school system organized around structured, orchestrated, differentiated, and resourced portfolio pedagogy? Large-scale mixed-methods external assessment for public accountability could be done annually—but in one grade level (third grade in one year, fourth grade the next, fifth grade the next…. That would disconnect teaching from the standardized treadmill, measure learning at the level of schools and districts, provide foundations for internal local inquiry, and make room for subjective assessment where such assessment can make a difference.
Your student interviews of readers at the start of the year and then at the end—and your ability to see clear differences—this is subjective assessment that can provide valuable insights to teachers. What a great use of administrative time. If you haven’t come across it yet, you’ll become acquainted with phenomenology as a method to uncover the essential elements of a phenomenon. Learning to read in first grade is a phenomenon, a defined experience large groups of people live through. Psychologists use the method to study questions like “What is it like to experience training in use of lethal force as a police officer?” Divorce, death, powerful phenomena that lots or all people experience are grist for phenomenological study. The phenomenon of learning in a portfolio culture of documenting and reflecting is wide open for analysis
"Problems are pre-identified, solutions are given, themes or captions are decided. Where is the writer?"
Where is the writer, indeed. Thanks Terry for this thoughtful post.
You got it. I’ve been working on this one for a couple of months. I jumped the gun a bit—a few slippery syntactic spots—but I wanted to finish it by New Years, I appreciate you, Matt.
Terry, it's good! You make an important connection between working memory, writing, and portfolio assessment. For writers who struggle with long term memory, relevance is important, but alone even that might not be enough. Portfolio assessment "holds things still", especially artifacts of the writing process + several pieces over time to aid the reflection process.
The portfolio framework implemented over a full school year that I studied in my dissertation extended to reading and writing in middle school English classes. I suspected that writing achievement would be significantly better in the portfolio classrooms than in the traditional classrooms, but I wasn’t so sure about reading achievement. In fact, the reverse turned out to be true. Students engaged in portfolio pedagogy posted significantly higher direct measures of reading at years’ end. There was no measured difference in writing achievement. I’ve done a lot of thinking about this finding over the past 20 some years. I think writing growth occurs more slowly than reading growth. I’m not seen any longitudinal research on portfolio pedagogy in either reading or writing. Research is needed to understand developmentally what happens when reflective analysis in the context of goal setting is taken up seriously as a learning outcome. Periodically “holding things still” over a long time period—I hypothesize that reciprocal influences across reading and writing experiences will be different as children’s cognition and memory grow. Motivational orientations will also become more complex with increases in intrinsic motivation gradually showing up—the reverse of what we now see.
That is interesting. You must have had to consider a lot of factors with this study, Terry. Writing as a more lagging measure makes sense. And what a nice surprise to see reading getting a boost!
When I examined our school's use of portfolios for only writing assessment, we saw almost double the growth in classrooms that were committed to the portfolio process vs. past efforts with portfolios.
These classrooms that found more growth:
- posted a student writing artifact every six weeks in a digital app that shared out to families
- had the student record audio of their structured reflection about the writing they chose to share
- positioned the teacher as a facilitator of reflection vs. a "giver" of feedback
A summary: https://readbyexample.substack.com/p/initial-findings-after-implementing-digital-student-portfolios-in-elementary-classrooms
I did acknowledge that I had no formal training with facilitating research like this, so I am confident there are errors in our methodology.
You did a nice job with data collection in terms of breadth. Six traits is a good place to start. Think about structured ways to capture children’s reflective analysis on their reading. Also, consider examining questioning from the teacher-student perspective but also from the student-teacher perspective--questions going two ways. You could do a nice qualitative study of the videos of students engaged in reflection on their writing. Once you start planning your dissertation, I’m looking forward to giving you my feedback. The field needs more Matts willing to dig into research methodology and use it rigorously to lead a school in its day to day work.
Thanks Terry. I appreciate this space to unpack past work and think creatively about the future, as well as your continued offer of support.
See? There are likely important developmental differences. It may be that changes in reading competence and sophistication happen more slowly in the earlier grades while writing changes are more dramatic and visible. I wonder what happened with reading when you looked at portfolio pedagogy in your school? I know there is cognitive and linguistic spillover, a synergy between reading and writing... Part of the problem, too, is the sensitivity of measurement tools. For example, it’s always seemed problematic for me to use a standardized multiple choice measure of achievement to determine growth over periods as brief as a year.
Good points Terry. I continue to be startled from time to time when I connect with a 1st or 2nd grader around their reading or writing in the fall and then in the winter. "Are you the same kid?" :-)
And yes as they get older and growth seems hard to tease out in their literacy artifacts. Maybe more focus on the engagement and identity growth in the middle grades?
I think so. It’s reasonable to assume, for example, that younger kids thrive better with a mix of approval in the form of social rewards and clinical feedback from teachers. Middle schoolers based on what I found do not exert effort for social approval. Learning styles in regard to motivation are tied to developmental identity, and literacy learning is very personal. Can you imagine a school system organized around structured, orchestrated, differentiated, and resourced portfolio pedagogy? Large-scale mixed-methods external assessment for public accountability could be done annually—but in one grade level (third grade in one year, fourth grade the next, fifth grade the next…. That would disconnect teaching from the standardized treadmill, measure learning at the level of schools and districts, provide foundations for internal local inquiry, and make room for subjective assessment where such assessment can make a difference.
Your student interviews of readers at the start of the year and then at the end—and your ability to see clear differences—this is subjective assessment that can provide valuable insights to teachers. What a great use of administrative time. If you haven’t come across it yet, you’ll become acquainted with phenomenology as a method to uncover the essential elements of a phenomenon. Learning to read in first grade is a phenomenon, a defined experience large groups of people live through. Psychologists use the method to study questions like “What is it like to experience training in use of lethal force as a police officer?” Divorce, death, powerful phenomena that lots or all people experience are grist for phenomenological study. The phenomenon of learning in a portfolio culture of documenting and reflecting is wide open for analysis