Beyond Cloze Tests: How I Evaluate Students in Story-Listening Classes

A teacher in our community recently asked me: “Are there sample cloze test PDFs that I could look at? What would be an easy way for me as a teacher to generate cloze tests for my students?”

I understand the question. A cloze test looks simple. You take a passage and remove some of the words. You can easily find information about cloze tests on the internet.

However, constructing a valid and reliable cloze test requires a thorough understanding of language testing, including the purpose of the test, text selection, students’ familiarity with the content, deletion procedures, scoring methods, and the interpretation of results.

Why Constructing a Cloze Test Is Not Easy

The cloze test I used with my students for many years required careful work at every step.

First, I had to choose a text that my students could relate to and that was at an appropriate reading level for the goal of the course. I wanted the test to include 100 items to improve its reliability, which required a much longer passage than those typically used in ordinary tests. I deleted every tenth word instead of every seventh word, as is often suggested.

I kept the first paragraph intact so that students could establish the context. I checked the reliability of the test using students at approximately the same proficiency level. I also had to decide how to score answers that differed from the original words but were still acceptable.

Once a valid and reliable test had been developed, I had to preserve copies carefully for future use rather than returning the tests to students. I numbered the test sheets and collected students’ smartphones before testing.

Constructing a cloze test that yields meaningful and dependable results is a demanding research task. I would recommend studying the research on cloze testing before attempting to develop your own test.

What I Use Most of the Time Instead

I do not recommend that teachers give frequent tests. Instead, I use a method that does more than simply produce a score.

To evaluate students’ comprehension of Story-Listening and observe the development of their listening comprehension over time, teachers can use summary writing in the students’ first language.

After listening to a story, my students write what they understood, from the beginning to the end, in their first language. Some students write a complete and detailed summary, while others write only a rough outline. Some do not reach the end of the story in their summaries. At the end, they also write what percentage of the whole story they think they understood. This percentage does not have to be exact.

These summaries serve several purposes.

1. Feedback for the teacher

Reading students’ summaries tells me what they understood and what they did not understand. This information helps me identify where my storytelling may not have been sufficiently comprehensible and how I can improve it next time.

In this way, summary writing is not only an evaluation of students. It is also feedback on the teacher’s storytelling skills.

2. Formative assessment

By collecting students’ summaries after each story and keeping them in individual folders, teachers can observe students’ development over time.

Rather than relying on a single test score, we can examine how their comprehension changes throughout the semester.

3. Evidence of progress

When teachers need to report students’ progress, the collected summaries provide useful evidence.

We can examine how much students write, how accurately they describe the content, and how precisely they recall the events of the story.

I also noticed something interesting in my own classes. As students began to understand more, their handwriting often became clearer.

These observations helped me notice changes that a conventional test score might not reveal, especially changes in students’ attitudes toward the class, which seemed to be reflected in their handwriting.

4. Using summary writing for final grades

At the beginning of the semester, I tell students that they will write a summary on the final day and that it will contribute to their final grade.

I explain that they should write as much as they can remember about the story.

Knowing this from the beginning encourages students to take the summary writing seriously throughout the semester.

5. Encouraging regular attendance and attentive listening

Students also understand why attending class regularly is important.

Each time they listen to a story and write a summary, they are gaining experience that prepares them for the final evaluation.

They do not need to memorize vocabulary lists or study grammar rules to prepare for the final test. They cannot prepare by memorizing a particular story because they do not know which story I will tell.

Their preparation consists of attending class, listening to stories, and understanding the messages.

This helps establish regular attendance and attentive listening as normal classroom behavior.

6. Summative assessment

On the final day of the semester, I tell a new story and ask students to write a summary.

As the semester progresses, the stories gradually become longer and more complex. By the end of the last week of the term (or semester), students will have been listening to stories that are considerably longer and more complex than those they heard during the first month.

For the final examination, I tell a new story that is no more difficult than the stories students were listening to during the last week of instruction. The content should be completely different, but the length and complexity should be comparable to what students have recently experienced.

This final summary serves as the summative assessment.

Students demonstrate how much they can understand and recall from a story they have not heard before.

7. Comparing the beginning and end of the semester

We can also compare students’ first summaries with their final summaries.

By examining the amount of information they understood, the accuracy of their descriptions, and the details they included, we can observe how their performance has changed over the semester.

This provides a useful picture of their development.

What Happens in the Second Year?

In the second year, my students listen to much longer stories, sometimes lasting about 40 minutes.

At this stage, I no longer ask them to write a summary after every story. I want them to spend their class time listening to and enjoying the stories.

However, I ask them to write a summary in the target language at the beginning of the semester and another at the end.

We can compare the number of words they write in the two summaries as one measure of their written production fluency. We can also examine the number of accurate phrases or clauses and compare the results.

We call this procedure the Listen and Write Test.

A similar procedure can be used with reading. Students read a short book that they can finish in 30 minutes and then write what they understood in the remaining 20 minutes of the class. I call this the Read and Write Test.

These procedures allow us to examine changes in students’ performance over time without requiring frequent tests or exercises during regular instruction.

The Purpose of Evaluation

Evaluation should help us understand how students are developing and how we can improve our teaching.

We do not need to test students constantly to find out whether they are making progress.

In Story-Listening classes, students acquire language by listening to comprehensible and compelling stories.

Summary writing, when used appropriately, can provide teachers with useful information about students’ comprehension and development without turning the storytelling itself into a testing activity.

The important thing is to protect the time students spend receiving comprehensible input.

— Beniko