Low Stakes Quizzing and Retrieval Practice Part 3

In my last post, I gave an overview of retrieval practice and how it can be informed by whole class feedback. In this post, I will go over some different retrieval strategies, including how to approach quotation memorisation.

I began my teaching career as an EFL teacher, teaching English abroad. On my CELTA course, the entry level training qualification to teach English as a Second Language, we were taught to plan lessons that followed a PPP model (Presentation, Practice, Production), a framework that matches most people’s understanding of a deliberate practice approach to instruction. With regards to the practice element, student would be asked to complete controlled practice (essentially limited and restricted practice activities) before attempting freer practice. This is nothing revolutionary and is the same as ‘I do, we do, you do’, or any other approaches to the gradual fading of support and scaffolding when teaching. With regards to retrieval practice, it is useful to think in similar terms: I want students to experience quick, initial success with restricted and decontextualised recall, aiming for the ability to use the item in gradually freer and wider contexts. As Daniel Willingham points out, ‘Inflexible knowledge seems to be the unavoidable foundation of expertise’ (See this AFT article on Inflexible knowledge from Winter 2002).

I teach some of Engelmann’s Direct Instruction programmes and one of the key maxims behind DI schemes of work, an approach called ‘The Strand Curriculum’, is that concepts are taught to and used by students across a number of lessons and weeks, the intention being not just retention but also fluency and mastery of the material. A gradual progression from restrictive, closed practice activities-things like closed recap questions at the start of a lesson-to wider and application based practice is also something that is common across DI programmes that I have encountered, one of the aims being to slowly and methodically convert ‘inflexible’ into more ‘flexible’ knowledge.

Here is an example of a possible journey of a test item-in this case a new vocabulary term-from narrow to wider tasks and moving from massed to spaced practice:

1) The vocabulary word is initially encountered in a vocabulary table (see this post for more information). The teacher may have asked students questions about it, annotated it with further information and provided examples and non-examples to help define it.

2) Student then use the word when annotating an extract

3) They then see the word used in a wider context, perhaps within a model paragraph that they deconstruct and annotate.

4) They are then asked to use the word in a paragraph of their own.

4) Next lesson: The word is included in a low stakes quiz. The teacher probably gives the first few letters to jog their memories. It will probably be included in the next two or three lessons’ quizzes too, each time with less clues and ending up with no clues.

5) A few lessons later: Students are asked to complete a because, but, so task, as explained in this blog

6) Later still, a student could be asked to complete mini pieces of analytical writing where they use and apply the word in a freer context, perhaps combining the word with other items like quotations, contextual knowledge, analytical approaches, specific sentence structures and plot information.

Examples:

Jekyll and Hyde: How is Hyde Presented? Include:

• atavistic

• CONTEXT: Victorian fears about science the enlightenment

• volatile

• QUOTATION: ‘snarled aloud into a savage laugh’

• callous

• semi-colon sentence

• ANALYTICAL SKILL: 3 part explanation

 

Macbeth: How is Duncan presented? Include:

• Meek

• Benevolent

• oblivious

• CONTEXT: Flattering King James, validating Divine Right of Kings/Great Chain of Being

• QUOTATION: ‘there’s no art to find the mind’s construction in the face’

• appositive sentence (see here for an overview of teaching appositives)

 

Depending on the ability of the class and whether you think they have the relevant concepts stored in long term memory, you could either ask them to make a list of relevant points in response to the question before giving them things to include (retrieval practice) or you could explicitly tell them what to include.

The key premise here is the idea that concepts should be recycled across lessons and units, creating distributed practice activities that will help aid retention.

Memorising Quotations

Although I initially baulked at the idea of a closed book exam, I am now a convert to the idea that students should memorise quotations. Not only does it provide them with a solid knowledge base of a text, allowing them to anchor their thoughts when writing and providing sticking points for other concepts to attach themselves to, it also gives them confidence, something that is particularly evident with the weakest students. It also means that as teachers, we now need to know our texts that much better and, as a result of the last two years’ worth of closed book examinations, I know the texts that I teach far more comprehensively than I ever have before.

Here are some of the things that we did to address this new requirement:

1) Make lists of quotations, choosing lines that fit key characters and themes

This was a painstaking yet crucial first step, forcing us to think about the most salient and relevant parts of a text when seen through the lens of GCSE and A Level (we deliberately included some quotations that deal with themes and ideas that would push students beyond the requirements of KS4). The great works of literature are considered great for a reason and attempting to choose, prioritise and reduce them to a condensed list is always going to be difficult. Despite this difficulty, it needed doing: without a codified body of knowledge, retrieval practice and teaching for retention is all but impossible. For Macbeth, we listed quotations by scene, the intention being to aid understanding of the narrative arc and plotline by following it chronologically. For Jekyll and Hyde, we grouped them by character and theme. For Inspector Calls (arguably the most important one for students to learn parts of due to the lack of extract on the paper), we made 4 sets with no overarching organisational approach. The jury is still out in terms of which approach is best!

3) Upload them all in sets onto Quizlet

If you haven’t had a look at Quizlet, it is a free computer based memorisation programme. Some of our student use it regularly, finding it less tedious and more effective (it knows which items you perform badly on and prioritises retrieving those) than using traditional paper based self-quizzing.

2) We ask students to memorise them for homework

We explicitly taught and demonstrated how to approach self-quizzing so that students were able to take advantage of retrieval practice, taking care to explain that spreading their study over the week is a better and far less laborious approach than cramming the night before the test. Quotation learning homework is typically ‘lagged’ (the idea that students are studying one text in class, but learning quotations from a previous unit in order to better distribute practice). During the week that they are completing homework, I will include questions about their quotations in recap quizzes, again helping them understand whether they are on track and know the quotations (laser precise AfL!) as well as sometimes (particularly with weaker classes) giving them practice tests. We set the bar high, typically allowing students to fail two quotations out of the ten or so that they have to memorise. I explain that the pass mark is challenging because, if students use the strategies that we give to them and plan their sessions across the week, memorisation is achievable by almost everybody. This does not mean that all students are doing their homework-if only!

3) We test them

Initial tests are gap-fill, blocking out important words and usually providing the first letter to jog their memories.

4) We keep testing them, distributing practice across the unit, year or course.

As I mentioned earlier, the idea is to gradually move from restricted practice (gap fill test) to freer application, culminating in an essay response that uses the quotations. Here are some possible retrieval approaches and I have attempted to order them using this principle:

a) Start a lesson with a gap fill of the quotations

You could then annotate and then write using annotations (these could be teacher-led annotations or retrieval practice)

b) Write one quotation or gap fill on the board, then ask students to write for 5/10 mins about it

You could include explicit success criteria like mentioned in this post, setting high expectations as well as allowing you to give precise feedback

c) Start a lesson asking them to write down 3/5/7 quotes on a particular character/theme/idea

The idea here is to encourage students to make connections and deepen their knowledge of the items. You could provide the first few words for weaker students; again, this could be the beginning of a writing activity.

d) Model the planning of the essay response (paragraph titles) and then ask kids to match quotes to specific paragraphs

By providing them with an essay structure, we give them a framework to attach their quotations to, demonstrating how they fit into a freer more complex practice task.

e) Write essay question on the board and then ask kids to list quotes that are applicable:

You could make kids explicitly aware of parallel, equivalent questions that essentially ask the same thing:

E.G. Explore the idea of deception in Macbeth/Explore the theme of equivocation/Explore the theme of appearances vs reality (all essentially the same question!!!!)

E.G. Explore the theme of responsibility/How does An Inspector Calls comment on society?/ Explore the notion of community/ Explore the theme of inequality (all essentially the same question!!!)

This process of grouping problems according to their deep structure will hopefully help students deal with a range of potential exam questions, prompting them to apply the correct problem solving strategy. Experts intuitively see the deep structure of a problem, a skill that novice learner (our students) find difficult. Students can often get thrown by novel wording in a GCSE question and fail to realise that it is asking them about something that they know: making these links and drawing attention to the deep structure may help them to see past the surface structure (the wording of the question). If you want to find out more about the distinction between the surface structure and deep structure of problems, this AFT article is a fantastic introduction.

Next post: Insights from Direct Instruction: How can Engelmann’s theory and approach help with everyday teaching?

Low Stakes Quizzing and Retrieval Practice Part 2

In my last post, I outlined an approach to low stakes quizzing, focussing on vocabulary retrieval. Although vocabulary is important, there are a number of other things that I want my students to remember and apply. Like many other schools, we have begun developing knowledge organisers in an attempt to make clear what we believe to be the most important information in a particular unit. These ‘core knowledge’ documents contain crucial plot and character information, themes and facts about the socio-historical context of whatever text we are teaching. We have also begun to develop booklets, essentially home-made text books, so that knowledge is defined down to the word level. While knowledge organisers provide the core, the booklets allow us to define a wider associated domain, all of which is available for teachers to use in retrieval practice. As well as vocabulary, a typical low stakes quiz will ask questions about anything from the booklets. At present, we are approaching this in an ad hoc, spontaneous fashion, although I can see the potential for being more rigorous by aiming for optimum spacing between tests as well as thinking about the relative importance and utility of each concept that is being quizzed, prioritising the essential.

Here is a short sample quiz from a year 7 unit on Shakespearean rhetoric which exemplifies the range of questions that could be asked. Crucially, all the questions are about things that we have taught them from the booklets. (Although I have left them out here, as mentioned in the last post, students may be given clues and the answers may be followed up by further questions that deepen understanding or encourage students to make links):

1) What rhetorical technique involves a series of rhetorical questions? (key term from knowledge organiser)

2) In chronological order, write down the two monarchs who were on the throne during Shakespeare’s lifetime (information from a non-fiction text)

3) Who wrote Telephone Conversation and what country is he from? (question about previous unit)

4) Name two other famous playwrights who were Shakespeare’s contemporaries (information from a non-fiction text)

5) When did Queen Elizabeth ascend to the throne? (information from a non-fiction text)

6) Write a noun beginning with Emp….. that means the ability to understand and share the feelings of others (vocab from a vocab table…see this blog for more information about this resource)

7) Why was it difficult to be a Catholic in Elizabethan England? (information from a non-fiction text)

8) What is a pause in the middle of a line of poetry called? (question about a previous unit)

9) What adjective beginning with profi….. means competent and skilled? (vocab word highlighted in non-fiction text)

10) What was the system of racial segregation in South Africa called? (question from a previous unit)

 

Again, the purpose of these quizzes is to take advantage of the testing effect in order to promote retention. We don’t mark them or record scores. Quizzes should be cumulative, asking about things from yesterday, last week, last month and previous years.

Whole Class Feedback and Low Stakes Quizzing

Whole Class Feedback is a revelation, saving time for teachers and encouraging a focus on weaknesses, misconception and gaps in student understanding. If you want to know how to approach it, there have been plenty of influential and informative blogs that explain the process in detail: see this (https://readingallthebooks.com/2016/03/19/giving-feedback-the-michaela-way/)  from Jo Facer, one from Daisy Christodolou (https://thewingtoheaven.wordpress.com/2017/08/16/feedback-and-english-mocks/) and this from Mrthorntonteach (https://mrthorntonteach.com/2016/04/08/marking-crib-sheet/).

This approach links in perfectly with low stakes quizzing as you can compile a list of errors which can then be added to your recap quiz in order to help student make corrections. If you are worried about the fact that whole class feedback means that some students will be asked to practise or retrieve things that they were previously able to do, therefore wasting their time, the concept of overlearning seems to suggest that extra practice beyond the point of initial mastery aids long term retention. (see this blog for a useful overview of overlearning: http://www.learningspy.co.uk/featured/should-students-be-overlearning/ )

Here are some examples:

1) If a student spells antithetical wrong, include it in the quiz. Ask them to spell it, then write the correct spelling under the visualiser and they can then check and amend if necessary. Ask them to spell the noun and the adverb too!

2) If students have made a poor word choice and failed to use a more precise word, write the weak sentence and ask them to replace the word. Again, this is not vocabulary brainstorming-an activity that flatters yet fails to stretch the word-rich and disadvantages or ignores the word poor; I would ask them to replace the word with one that you have taught them, which is a far more equitable approach:

Weak sentence: Gerald was able to use Eva Smith because she was poor.

TASK: Replace the underlined words with more precise and sophisticated vocabulary.

Possible answer: Gerald was able to exploit Eva Smith because she was destitute.

Like other quiz questions, you can give them clues-the purpose is for them to successfully retrieve the word from memory. If they are successful, the memory’s durability will hopefully have been strengthened.

With this particular activity, you can ask them use write more complex sentences if they are able to:

As a result of her destitution, Gerald was able to exploit Eva and ‘install her’ as his mistress.

Destitute and vulnerable, Eva Smith was easily exploited by Gerald, a man of wealth, power and status who must have seemed like a ‘fairy prince’.

In a future post, I will write about how Whole Class Feedback can further inform the deliberate practice of sentence structures, ensuring that you teach, practice and recap areas of weakness (a precise approach to Assessment for Learning).

In the next post, I will explore how we help students succeed in closed book literature exams by asking them to memorise quotations, as well as looking at other retrieval practice strategies.

Explicit Vocabulary Teaching 3: Because, But, So

Explicit Vocabulary Teaching 3: Because, But, So

In my last post, I looked at how we present our approach to explicit vocabulary teaching. When we first began codifying the vocabulary that we wanted to teach, we had no real plan for getting students to practice using it, instead using vocabulary tables as a supporting resource and asking students to use the words in their written responses. Since reading The Writing Revolution I have begun to use one of the book’s better known techniques-Because, But, So.

Essentially, this approach asks students to complete sentences and requires them to engage in far more specific and focused thinking than just asking them to respond to an open-ended question (p.40). Here is an example from the book:

1) Seeds need light to grow because…..

2) Seeds need light to grow, but…..

3) Seeds need light to grow, so….

Each of these sentences ends with a conjunction and, due to the specific function of each term, forces students to think in precise and defined ways when completing the sentence: sentence 1 requires a student to give a reason; sentence 2 requires a contradictory idea and sentence three requires an effect or consequence. Not only does this allow students to practice using important conjunctions and create compound or complex sentences (areas where the weakest students at secondary still struggle), but it also encourages students to think more deeply about the content contained within the sentence as they have to produce developed responses, extending their thinking in the process.

Initially, I allow students to work with the support of their vocabulary tables-an approach that I explored in this blog-allowing a high chance of success in order to help encode these words in their memories. In later practice activities, I ask them to do these without any scaffolding, essentially combining the ‘retrieval effect’ with this approach.

Although The Writing Revolution restricts this practice activity to the three aforementioned conjunctions-and this is a great place to start, particularly with weaker students-the premise can be extended to include a wider range of sentence constructions, adding variety and increasing complexity to the practice. We have been experimenting with using phrase level analysis (appositive, participles and absolutes: see here for an overview) as well as ‘even though’ and ‘by’. This deliberate practice model is very similar to what Doug Lemov calls Art of the Sentence , an approach that asks students to regularly practice whatever sentence constructions that are deemed important in your subject.

With regards to teaching literature, let’s have a look at some examples that encompass the original framework:

1) Mr.Birling is pompous because…

2) Eric lives a privileged life so…

3) At the beginning of the play, Lady Macbeth seems dominant but…

As they are, these sentence stems pose a challenge and require students to think carefully about the meaning of the target vocabulary. To increase this challenge, you can add specific success criteria, detailing exactly what you would expect a student to include in their completed sentence or sentences. When used like this, these practice items can go beyond mere sentence completion, instead acting as targeted and purposive responses to whatever text you are studying. Sometimes these criteria can ask students to make links between the target vocabulary and other relevant bits of knowledge; they can also make clear your expectations as to technical sophistication or the full development or arguments and ideas. Also, they can be used to make links between different pieces of vocabulary that you have taught them.

Let’s have a look at some examples:

1) Mr.Birling thinks he is infallible because…

• Include pompous
• Include a quote
• Add a semi-colon and a further independent clause

Mr.Birling thinks he is infallible because he emphatically states that the ‘Titanic’ is ‘unsinkable’; alongside other bold and pompous pronouncements, this makes us doubt his intelligence and question the validity of his position as a man of influence at the top of society.

2) Eric lives a privileged life so…

• Include a quote
• Add a participle phrase

Eric lives a privileged life so he has few responsibilities and immense wealth, enabling him to drink ‘too much’.

3) At the beginning of the play, Lady Macbeth seems dominant but…

• Add an appositive
• Add ‘Out Damned spot!’
• Add a second sentence discussing Shakespeare’s intention
• Include nefarious, insurrection, plight and admonishment

At the beginning of the play, Lady Macbeth seems dominant but her nefarious plan eventually causes her to become mentally unstable as she chants ‘Out Damned spot!’, evidence of her psychological instability. Perhaps Shakespeare wanted her plight to serve as an admonishment to those considering insurrection or regicide, potentially flattering King James, who had just survived an assassination attempt.

An additional benefit to these restrictive practice exercises is that they allow the teacher to circulate and give precise feedback to students as they write. You need quotation marks around your evidence. Has your appositive got a noun in it? Check your spelling of ‘assassination’. Tell me what ‘but’ means…does you sentence make sense here? Instead of taking in books, marking and then handing them back-a laborious and lengthy process-corrections can be made instantly and quickly. Because the practice is so defined and restricted, your feedback can be completely purposive and precise. Instead of vague, abstracted comments like you need to develop that further or you should extend that point or you need to use better vocabulary, you can refer to the success criteria for each practice item. I tend to do these under a visualiser: modelling several by writing them ‘live’ is crucial, especially for the ones that contain specific success criteria. Sometimes I take a student’s book and we explore their work under the visualiser; sometimes I ask students to read out one that they are proud of (making sure they SAY the punctuation marks so that I can check if they are accurate).

In a future blog, I will look at how we stagger and space out this practice across units to take full advantage of the benefit of distributed as opposed to massed practice. Regularly recapping and engaging in retrieval practice is essential if we are serious about students retaining the information that we teach.

Next post: How to use low-stakes quizzing and retrieval practice