Command Performance: Applying AI as a Tool for Teacher Reflection

In a matter of a few short months, I’ve come to understand as a matter of practice the value conversational AI holds for academic learning task. I’ve benefited from using it for particular purposes, especially analytical tasks yielding an outline, a set of categories, and the like. It is a magnet for coherence once it establishes a stable context including the task, problem, or goal and topic the user has in mind. Note that the search for coherence across a conversation is a primary bot driver whether the context is dealing with fact, speculation, or fiction. A weak context is fertile ground for hallucinations or fabrication.

Here’s an example of a recent analytic task. Jeff provides his seminarians with questions to guide their individual reading assignments and to prepare them for class discussions. Jeff sent me a document with several weeks worth of questions he had used in 2022; he revised them for 2023 and included questions sets for his current, 2023 seminar. Each question set contained three questions for a total of roughly 30 question sets, 15 from 2022, 15 from 2023. The following excerpt from Jeff’s assignment sheet will give you a sample of what this document looks like (there were several future assignments mapped out beyond this sample):

9/1/22  Genesis, Chs. 1-9 and 11:1-9; Exodus 19-20

1)    Drawing from the list of “Main Themes and Issues” set forth on the first page of our syllabus, identify which themes or issues seem relevant for the assigned readings from Genesis and Exodus and briefly explain their relevance.

2)    Compare Genesis 1:1-2:4 with Genesis 2:4-3:24 with regard to the process and sequence of creation.

3)    In what ways does Exodus 19-20 present a continuation of the story and themes presented in Genesis 1-9 and 11:1-9?

Just under this 9/1/22 section of old questions was the revised question set for 2023. This document represents a type of evidence of practice I collected routinely when I was in the data collection phase of my dissertation. My task then was to code the data I found on teacher-prepared documents used instructionally, or partition the content, to look for patterns. In this case, 2022 represents evidence of old practice; 2023 represents evidence of new practice. Jeff revised questions to align with new reading outcomes he had committed to for the new semester. In 2022 he had not spelled out reading outcomes. Instead of painstakingly coding these questions to determine whether there is evidence of improvement, perhaps three hours of work for me, I read them for impressionistic and intuitive sensing, and then I pasted the document into the bot with my educated biases encoded as task requirements and used the bot as an analytical tool to save time. Three hours was reduced to perhaps ten minutes.

- robot: Hello, How can I assist you?

 - user: I’m going to provide you with a document containing study questions given to sophomores in college taking a course in ancient literature. The assignments focus student’s attention on the texts they read for homework and cue them to prepare for in-class discussions.

I want you to analyze and compare the qualities of the 2022 questions. Then do the same for the 2023 questions. List the qualities for each year separately.

Then I want you to provide me with specific examples that show improvement from 2022 to 2023. Include the specific words from the questions you select to show improvement.

One way they have improved I see is in helping students see personal relevance in the reading. Another is that they encourage more critical thinking. A third is that they engage students more by increasing demands on them.

Be certain to include at least six examples of explicit comparisons between 2022 and 2023 so that I can see evidence of improvement.

The list of qualities for each separate year was informative. There was overlap. For example, both years exhibit a focus on character analysis, directing students to compare characters across texts. In 2022, however, a greater emphasis was placed on tracking character development than in 2023. But several qualities identified in 2023 validated what Jeff was hoping to do, i.e., promote a reader centered perspective, encourage students to consider the reception of the text at time of production, and require students to relate the reading to contemporary life. Here is the bot’s output:

robot: Qualities of the 2022 questions:

1) Focus on character analysis: The questions prompt students to analyze the characters' actions and portrayals in the assigned passages.

2) Comparison with other texts: Students are asked to compare the characters and themes in Homer's Iliad with another literary work, such as the epic of Gilgamesh.

3) Evaluation of character development: The questions inquire about the changes and developments in characters' behaviors and portrayals throughout the assigned passages.

4) Investigation of the impact on the overall storyline: Students are prompted to explore how specific incidents or encounters affect the overall narrative of the texts.

5) Analysis of themes: The questions address important themes in the texts, such as friendship, mortality, heroism, justice, and the feminine versus masculine.

Qualities of the 2023 questions:

1) Emphasis on character analysis and comparison: The questions continue to focus on analyzing characters and their relationships, particularly in terms of friendship and heroism. They also encourage comparing characters across different texts.

2) Examination of the author's perspective: Students are asked to consider the author's viewpoint or agreement/disagreement with their evaluation of a character's heroism.

3) Connection to real-life issues: The questions now engage critically with the texts by relating them to contemporary issues like privilege and its impact on the afterlife.

4) Evaluation of plot structure: The questions explore the plot structure and its symmetrical nature, treating texts as mirror images with regard to plot and themes.

5) Oral discussion: Some questions now prompt oral discussions, providing an opportunity for students to engage in class discourse.

The bot also made quick work of finding illustrative examples:

Examples of improvement from 2022 to 2023:

1) Example 1:

2022 Question: "Compare Achilles' behavior in Book 22 and in Book 24. How does Homer present Achilles as a heroic character, and how does the portrayal in Book 24 augment or amend what we had come to know about from the other assigned passages?"

2023 Question: "Based on the assigned sections of the Iliad, to what extent do you consider Achilles to be a heroic character? How might Homer agree or disagree with you?"

Improvement: The 2023 question now encourages students to evaluate Achilles' heroism based on their interpretation, taking into account Homer's possible agreement or disagreement.

2) Example 2:

2022 Question: "Evaluate the relationship between Odysseus and Circe (Kirkē in Green's trans). What does it reveal about Odysseus' character?"

2023 Question: "Evaluate the relationship between Odysseus and Circe (Kirkē). What does it reveal about Odysseus' character? Analyze Odysseus' visit to the realm of Hades in light of the Wicked Problem of 'privilege.' To what extent does privilege affect the situation of souls in the Homeric afterlife, and how is privilege attained?"

Improvement: The 2023 question expands the analysis of Odysseus' character by incorporating the concept of privilege and examining its influence on the afterlife within the context of the characters and events.

3) Example 3:

2022 Question: "What do we learn about the backstory of Agamemnon - the significant events that precede the opening scene of the play - and how do we learn about it?"

2023 Question: "What do we learn in the Agamemnon of its plot's backstory - the significant events that precede the action of the opening scene of the play - and how do we learn it?"

Improvement: The 2023 question clarifies that the inquiry is about the plot's backstory, providing a more concise and focused question.

Before closing this conversation I asked the bot to summarize key insights into the nature of reading instruction in 2023 implied from its analysis of these question sets. I was able to share these findings with Jeffrey, who expressed the feeling of validation and relief that changes he was laboring to make could be discerned through logical analysis. Keep in mind I received these questions on late Tuesday evening, I think, and was ready with this analysis by 9:00 the following morning. Here are three points the bot selected.

1. Increased focus on textual analysis: The 2023 assignments may have a greater emphasis on close reading and analysis of the assigned texts. Students may be required to examine specific passages, identify literary devices, and analyze their impact on the overall meaning of the text. Example: "Identify and analyze the use of symbolism in the passage from 'The Epic of Gilgamesh'."

2. Integration of historical and cultural context: The 2023 assignments may encourage students to consider the historical and cultural context of the texts they are studying. Students may be asked to research and discuss how societal norms and beliefs of ancient civilizations influenced the themes and portrayal of characters in the literature. Example: "Explain how the portrayal of women in 'The Odyssey' reflects the societal expectations of ancient Greece."

3. Collaborative learning and discussion: The 2023 assignments may place a greater emphasis on in-class discussions and collaborative learning. Students may be expected to come prepared to discuss their interpretations and perspectives on the assigned readings, engaging in critical thinking and analysis together. Example: "Prepare a list of discussion questions based on the reading to facilitate a class discussion on the themes of fate and free will in 'Antigone'."