Low stakes Quizzing and Retrieval Practice 5: Extended Quizzing

You can find the first four posts about retrieval practice here: one, two, three, four.

In the last post, I explored a number of principles that underpin effective retrieval practice as well as the relationship between the style and type of retrieval questions and the composition of the final assessment. This post will explore some additional retrieval methods.

A recent paper by Pooja Agarwal entitled ‘Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning?’ found that if a test item involved higher order processing, then retrieval practice activities should involve ‘higher order and mixed quizzes’ instead of just fact quizzing. But what do these terms mean? Using the 2001 revised Bloom’s taxonomy, the paper defines ‘higher order learning’ as being the top three layers of the Blooms pyramid: ‘apply, analyse, evaluate and create.’ The paper defines ‘mixed quizzes’ as retrieval practice that involves both factual and higher order questions and posits that ‘mixed complexity during initial learning may serve as an effective alternative to the prevalent ‘factual knowledge first’ standpoint.’ The paper demonstrates that fact quizzing will not, on its own, lead to good outcomes on higher order test items. Because almost all examination questions involve some level of higher order processing, this finding is of crucial importance to teachers.

The optimum retrieval approach to choose at a point in time will be heavily dependent upon the prior knowledge of the students in front of you. At the beginning of an instructional sequence with novice students, it may be appropriate to ask closed fact questions first so that students can acquire the concept or idea that you are teaching. This does not mean that you this is all you should do though. Teachers should continually be asking students to think hard about the concepts that they acquire so that their understanding gradually moves from being inflexible to flexible, helping them make links between different concepts and integrate them with their prior knowledge as well as asking them to apply, analyse, evaluate what is being taught.

So what might a mixed quiz look like?

For students to succeed in extended writing tasks at GCSE-whether these tasks are in History, English or any other subject that involves essay style responses-they need to select, combine and present often complex and abstract information, and then thread it all together into a coherent argument or explanation. To add to the challenge, they also need to do this from memory, retrieving and applying the information under pressure and in a limited time frame. Students need to have lightning quick recall as well as the cognitive dexterity to not only recognize which concepts are best deployed to a specific question, but also the ability to transfer their knowledge to questions that, whilst using novel terminology or phrasing, are essentially asking the same thing as those that they have practiced in class.

Here are some approaches that can help with this challenge:

Listing stuff

Instead of asking a series of unconnected factual questions, students can be asked to chunk information like this:

Teacher: Write down seven adjectives that we have learnt that describe Mr.Birling.

While asking for a list of facts is clearly more challenging that merely asking for one adjective, this still does not involve students doing anything more than remembering these words. The Agarwal Paper would point out that this disconnect between retrieval task and final test (in this case an essay) would mean that this quiz would be unlikely to help student performance in their Literature GCSE. However, the initial teacher question would then be followed by a number of other questions that would involve higher order processing, ensuring that the retrieval is mixed, an approach that should improve higher order test performance.

Here are some possible follow up retrieval questions that involve higher order questioning:

a) Because/But/So

Teacher: Copy and complete my sentences:

Mr.Birling is haughty because…

Mr.Birling is supercilious so…

b) Because/But/So with additional success criteria

Teacher: Copy and complete my sentences, making sure you include the additional components

Mr.Birling believes he is infallible, but

• dramatic Irony
• Birling symbolizes Upper Class
• 2 pieces of evidence

The regular inclusion of success criteria like these is a useful way to help students combine and integrate components into their responses. If I were to ask the question above, each of the bullet points would have already been taught and retrieved separately beforehand. Unless students have been taught the components here, it is likely that they will not be able to apply them, even in this restrictive activity. As an instructional sequence progresses and students become gradually more competent, two things will slowly change. Firstly, the context of these practice activities will slowly widen: students may start with sentence based practice, then move to paragraphs and then finally attempt full essays. Secondly, these prompts will slowly be removed as students no longer need to be made explicitly aware of the links between components or what is required by the question.

Analytical questioning

Teacher: Why is Mr. Birling haughty?

Evaluative questioning

Teacher: Who is more patronizing? Mrs. B or Mr.B

Extended Quizzing

Self-testing that uses a ‘free recall’ approach (also known as a ‘brain dump’) can be a really effective method and asking students to write down all the ideas that they know about a subject is clearly more challenging than splitting the information up into lots of different questions. However, if an instructional sequence begins with free recall as the method of retrieval, there are two potential problems. Firstly, students may find the task so difficult that they then become demotivated. Secondly, students may dump what they know onto the page in a haphazard and disorganized fashion, preventing them from thinking about and retaining the connections and links between individual ideas.

Similarly, if we merely ask a series of closed unconnected fact recall questions, this approach on its own will not enhance higher order learning as it will not allow students to see how concepts connect with each other, nor will it help them integrate ideas into schemas. For this to happen, teachers need to make these connections explicit to students and one way of doing this is through ‘extended quizzing’.

Let’s look at an example:

The screenshot above are the answers to a retrieval quiz about Macbeth. These were the questions that were asked in the order that they were posed:

• Complete the quotation ‘fair is foul and ……… is …………..’
• What technique is being used here? ‘chi………’
• Write down two words that describe the witches language

All these questions are connected and the intention here is to get student to start chunking this information: asking this sequence of questions makes the links between the individual concepts explicit, allowing students to organize and integrate them. Engelmann’s DI programmes contain similar chains of questions, the intention being to not only teach the specific concepts, but to also teach how they fit together in a larger knowledge system. If teachers do not teach these organizational structures and connections, then some students will not make these links. A core tenet of explicit instruction is that teachers teach everything that students will need and teaching the relationship between concepts is a vital part of this.

Here is a second example:

The screenshot above is from a retrieval quiz about Jekyll and Hyde, focusing upon Hyde. The red parts are the clues that I gave when I asked the questions (this was to a lower ability set and I expected them to struggle a little); the black parts are the answers that I filled in at the end.

These are the questions that were asked in the initial retrieval quiz in the order that they were posed:

• How does Enfield describe Hyde’s repulsiveness? (couldn’t specify the point)
• What two words describe his repulsiveness? (vague, ambiguous)
• What psychological concept does this relate to? (The Uncanny)
• Hyde is uncivilized and almost wild, what word can we use to describe this idea? (feral)
• Complete the quotation that demonstrates this attribute (ape-like fury)
• Circle the word that demonstrates that Hyde is animalistic (ape)
• What broader societal unease and fear does this word link to? (Victorian fears of evolution)
• Hyde’s violence is extreme, what word can we use to describe it? (brutal)
• Complete the quotation that demonstrates this (audible shattered)
• Hyde seems to enjoy inflicted pain on people, write down the word we can use for this pleasure (sadistic)
• Write down another word to describe Hyde’s attack (savage)
• What does his extreme behaviour suggest? (doesn’t conform to Victorian social mores)

While this example has plenty of clues and hints to help students successfully retrieve the information from memory, these are not always required. As an instructional sequence progresses, these hints and clues can be faded out so that students are retrieving the information with less support and higher attaining classes can dispense with this support at a much earlier point.

I would use extended quizzing as an intermediary approach in between classic short answer, closed questions and free recall. Earlier lessons may see extended quizzing; later lessons may see freer recall style retrieval. While extended quizzing sees the teacher creating the links and organizing the material into a logical structure, free recall shifts the responsibility to the student as they are asked to recall and organize the information themselves.

With essay writing as the final outcome, this table explores the benefits and detriments of different retrieval types:

Next Post: Insights from Learning As a Generative Activity