Insights from Learning as a Generative Activity: What is Learning? Part 2

In the last post, I explored different definitions of learning and how the ability to transfer or achieve generalized understanding will often be the end goal of an instructional sequence. This table provides a helpful summary:

kirchner vs fiorella and Mayer

What is Generative Learning?

generative

Fiorella and Mayer’s book explores a number of effective learning techniques, each of which are intended to produce meaningful learning. They express concern about the ‘passive experience’ of reading books, watching videos and attending lectures, all of which may result in ‘suboptimal learning’. The medium in which the content is being presented is not the issue here, instead the concern seems aimed at a failure to combine the medium with a sense making process of some kind where the content is manipulated and processed effectively by the learner.

Asking a student to read a text will not necessarily result in learning: it is how they process the content that matters.

Let’s take reading a book as an example: text is often the most efficient way of presenting complex, abstract information and a school that asks students to read frequently and widely is clearly doing the right thing. However, reading in order to learn is qualitatively different from reading for pleasure. When we read for pleasure, reading is about escapism, joy, inspiration and imagination and while we may learn much from the experience, often learning is not the primary goal of the activity.

other minds

As an example, I recently read Other Minds: The Octopus, The Sea and The Deep Origins of Consciousness by Peter Godfrey Smith, a fascinating book that explores the evolution and intelligence of cephalopods, and while I enjoyed it immensely and thought that I was learning a lot whilst reading it (I knew next to nothing about cephalopods before picking the book up), I’m sure that the experience was a suboptimal learning experience.

Even though I would have employed many of the strategies that are seen as the hallmark of skilled reading-and I’m sure that these would have helped me to understand the content whilst I was reading it-I didn’t deliberately accompany my reading with any of the effective learning techniques that Fiorella and Mayer explore. In this sense, my experience of reading it could be described as ‘passive’.

When a text is used as focal point of a learning situation (and for many subjects this should be the norm), the actual reading part should be accompanied by specific tasks as directed by the teacher. These tasks will ask students to engage in sense making and will involve all three stages of cognitive processing that Learning as a Generative Activity explores:

  1. Selecting relevant information to attend to
  2. Organising the material into a coherent cognitive structure in working memory
  3. Integrating it with relevant prior knowledge activated from long term memory

All three stages of cognitive processing are essential to generative learning and the absence of a particular stage will almost certainly preclude engagement with another. At one level, the stages can be seen as hierarchical as each depends upon the other in a sequence: the process of learning may begin with ‘selecting’ and proceed towards ‘integrating’ the new content with long term memory. Expressed using synonymous terms, learning begins with encoding, then becomes consolidated and is then stored.

However, this does not mean that the three stages occur in a strict, fixed sequence as each process will inevitably influence another. For example, your prior knowledge-a concept mentioned in the ‘integrating’ stage-will directly influence what you pay attention to when dealing with new information, thereby having an influence over the ‘selecting’ stage. If, as an expert, you are reading an extract from Macbeth, you are far more likely to notice and select the salient information than someone who has never studied Shakespeare before.

Let’s look at each stage in turn:

  1. Selecting relevant information to attend to

Ensuring that students pay attention to what is being taught is a fundamental aspect of teaching and learning: if students don’t pay attention, then no learning will happen at all.

As Samuel Johnson pointed out: ‘The true art of memory is the art of attention. If the mind is employed on the past or the future, the book will be held before the eyes in vain.’

The idea of attention is wide ranging. At one level, it is used to refer to general motivation and behaviour and whether a student is engaging-in a general sense-with the lesson. At another, it refers to the precise interaction between teacher, content and learner and whether the learner is focusing on the exact things that the teacher intends.

What happens if students fail to ‘select’ the relevant information?

If students fail to ‘select’ the relevant information, then this will effectively preclude them from engaging in the second two stages of cognitive processing: you cannot organize information into a coherent cognitive structure in working memory or integrate it with prior knowledge if ‘it’ is absent.

If students select the wrong information and use this information within a sense making process, then this will potentially be even worse: unlearning or unpicking misconceptions will often take far longer than learning things for the first time.

What can we do to ensure that students select the relevant information?

If we accept that the first stage of Fiorella and Mayer’s conception of learning is crucial and that failure to select the correct information will hinder or completely prevent learning from happening, then teachers need to ensure that all students-not just the high attainers-are able to do so.

In Direct Instruction programmes, all details of instruction are controlled  in order to minimise student misinterpretation and to maximise learning. This ensures that students always select the relevant information to attend to. DI programmes are based upon ‘faultless communication’, a state that is achieved through a number of techniques, approaches and processes. Firstly, lessons are scripted in order to prevent instructional and conceptual confusion; secondly, examples and non-examples  are carefully sequenced and thirdly, learning tasks are atomised and overtised so that learners pretty much cannot fail to select and practice what is relevant.

Although most normal lessons do not use DI schemes, if the principles mentioned above are considered and adhered to when planning sequences of learning, then students are far more likely to ‘select’ what is relevant and pay attention to what is important.

The optimal instructional procedure for ensuring that students will select the relevant information will also depend upon the prior knowledge of the student.  This is why the three stages of learning should probably not be seen as strictly sequential: they influence and depend upon each other. Students with less prior knowledge will benefit from a more directed and scaffolded approach to their more knowledgeable peers.

Let’s look at an example:

Imagine you are reading a two page non-fiction article about nostalgia as a preparation for reading the poem Émigrée. Each student has a copy of the article, the teacher has a visualizer and projector and everyone follows when someone is reading. At the end of the article there are two text-dependent questions that you want students to answer:

1) From the first page, list five psychological benefits of nostalgia

2) From the second page, explain the effects of nostalgia. Your answer should contain at least 3 effects

Assuming that motivation and behaviour are good and that your class contains students with sufficient prior knowledge, you could merely ask them to read it in silence and then answer the questions, safe in the knowledge that the breadth of their vocabulary and reading proficiency will mean that they will (hopefully) select the information required to answer the questions.

With a class of students who might struggle, you could do something like this:

  1. Read the whole text out loud to them, adding annotations (all students do the same on their copy) to explain vocabulary and asking lots of questions to check for understanding.
  2. Read the text again, stopping just past where the first psychological benefit can be found.
  3. Ask a student to read the question: ‘From the first page, list five psychological benefits of nostalgia.
  4. Check that everyone understands the question: add annotations to ‘psychology/benefit/nostalgia’
  5. Put red brackets around the section where the first benefit can be found
  6. Ask all students to reread the bracketed section in silence and underline the benefit
  7. Ask a student for feedback and underline your text under the visualizer.
  8. Ask lots of questions about the answer and give further annotations and elaborations about what is being underlined
  9. Ask all students to check that they have the same underlined answer and annotations as you.
  10. Repeat steps 2-9 for the next benefit.

By following this step by step approach, it is far more likely that all students will ‘select’ the relevant information than if you had adopted the same approach as the class with higher prior knowledge.

Successfully ‘selecting’ information is also dependent on being able to discriminate between ‘what is important from what is not.’ When learning a new concept, a student needs to understand its breadth and boundaries in order to fully understand it and this can be achieved through the careful sequencing and manipulation of examples and non-examples.

The intention of both the step by step approach and the manipulation of examples is to explicitly teach students what they need to do in order to successfully select information. Once this has been taught, the instructional support should be faded so that students eventually have to determine what is important and what is not without assistance.

What happens if students fail to ‘organise the material into a coherent cognitive structure in working memory’?

If students fail to organise the material in their working memory, then they will be unable to make any sense of it as they will quickly become confused or give up. When this happens, the incoming information will not be processed properly, effectively preventing it from being integrated with the relevant prior knowledge activated from long-term memory.

As teachers, we often forget what it is like to be confused by complex content but being overwhelmed by too much information is a common experience for many students.

Here is an example of something confusing, try reading it:

In mathematical terms, Derrida’s observation relates to the invariance of the Einstein field equation under nonlinear space-time diffeomorphisms (self-mappings of the space-time manifold which are infinitely differentiable but not necessarily analytic). The key point is that this invariance group “acts transitively”: this means that any space-time point, if it exists at all, can be transformed into any other. In this way the infinite-dimensional invariance group erodes the distinction between observer and observed; the Pi of Euclid and the G of Newton, formerly thought to be constant and universal, are now perceived in their ineluctable historicity; and the putative observer becomes fatally de-centered, disconnected from any epistemic link to a space-time point that can no longer be defined by geometry alone.

When reading this, I become deeply confused before the end of the first sentence as there are multiple concepts that I do not understand. Because I cannot retrieve these concepts from long term memory, I quickly reach the limits of my working memory and become cognitively overloaded.

What can we do to ensure that students organise the material?

Effective teachers will take a number of steps to ensure that students are not overwhelmed by new material. Content should be broken down into manageable chunks and introduced at a rate that does not cause cognitive overload. Teachers should use worked examples as ‘schema substitutes’ so that novices are better able to solve problems. A teaching sequence should progress through a process of backwards fading where initial learning is modelled and scaffolded by the teacher, allowing the student to focus on one or fewer elements of a problem or task and preventing them from becoming confused by a task that has high element interactivity.

What happens if students fail to integrate information with relevant prior knowledge activated from long term memory?

If the new material is not integrated properly into long term memory, then it is unlikely to be retained by a student. If we accept that ‘learning is a change in long-term memory’, then a failure to integrate equates to a failure to learn.

What can we do to ensure that students integrate the material?

In order for new information to be successfully integrated into long term memory, a student needs to already possess the relevant prior knowledge to connect it to. To understand complex and abstract ideas, a learner often needs to understand the concepts that underpin them: if these subordinate concepts are not understood, it is likely that the higher order concept will not be fully understood.

feynman-blackboard

There is a good example of this problem in this video clip. Upon being asked what it is that we feel when we hold two magnets that repel each other, Richard Feynman tells the reporter that he cannot explain the answer any further as his explanation would involve nothing at all that connects to the reporter’s prior knowledge: it would be at a level of complexity and abstraction that would be incomprehensible.

As teachers, we should be spending much of our time building student background knowledge and teaching high-utility vocabulary, content and approaches to writing. If prior knowledge is lacking, then we need to teach it!

When teaching abstract ideas, teachers should ensure that they are exemplified through concrete examples.

Let’s look at an example:

Imagine you are teaching the concept of ‘ignominy’, a word that we teach in year 9 when students study the jingoistic poem ‘Fall In’ by Harold Begbie. In order for this to be successful, you will need to be able explain the term using words that the student already understands, thereby beginning the integration with relevant prior knowledge. It can be helpful to start with a definition and example which can then be questioned and annotated.

ignominy

Assuming that students understand the definition, the teacher could then ask questions like:

  1. What does shame and humiliation mean?
  2. Can you give me an example of something humiliating?
  3. What is the difference between public and private?
  4. ‘Funked’ means avoiding something because you’re scared. What do the onlookers think the character is scared of?
  5. Why is this seen as ignominious?
  6. Why does Begbie ask ‘where will you look.’?
  7. Orally complete these sentences from the start: Avoiding going to war would be seen as ignominious because…The poem suggests that cowardice will result in ignominy because…
  8. Complete these sentences in your books:  When Begbie says ‘where will you look’, he is hinting at ignominy because… The poem explores the ignominious consequences of cowardice so…

After this sequence, later lessons could then involve wider questions and practice to broaden student understanding above and beyond the initial context of the poem:

  1. Complete these sentences in your books: Liverpool’s defeat at Watford was ignominious because…Her mistakes were ignominiously announced to the whole class, so…

Understanding here refers to creating more meaningful connections between the new item (ignominy) and the student’s prior knowledge. The more connections that are made, the greater the depth of understanding.

If we want all students to succeed, then we should be explicitly teaching them the connections between different concepts and this can be done through extended quizzing, annotation  or elaboration

In the next post, I will look at Learning by Summarising, the first strategy that Fiorella and Mayer explore.

Jekyll and Hyde Resources

Trying to remotely teach Victorian literature is difficult: the complexity of the texts means that many students will find them inaccessible without teacher support.

Here are two resources that might help:

Prose Summaries and questions of all chapters J and H

Jekyll and Hyde Knowledge Organiser

 

 

Insights from Learning as a Generative Activity Part 1: What is Learning?

Over the last few years, countless blogs, articles and books have been written, each one attempting to distill and describe the essence of good teaching. There has been a major shift in people’s opinions as explicit instruction has been lifted out of the shadows and placed at the centre of many school’s pedagogical frameworks. This shift is partly down to the massively increased interest in Rosenshine’s Principles of Instruction, a framework that has become almost ubiquitous now, partly thanks to the work of Tom Sherrington and Oliver Caviglioli.

Rosenshine’s work is based upon three broad areas: research into cognitive science; research on the classroom practices of expert teachers and also research on cognitive supports like scaffolding and using models to help students learn complex tasks. This AFT article from Spring 2012 is a good starting point if you want an overview of Rosenshine’s ideas and research.

generative

While Rosenshine’s work can be seen as research-informed overview of the kind of approaches and techniques that effective teachers use, Learning as a Generative Activity outlines eight different ways to foster ‘generative learning’ in students. Much like Strengthening the Student Toolbox, Learning as a Generative Activity explores strategies that students can use in order to learn effectively. Mayer’s work should be seen as complementary to Rosenshine’s as the instructional choices of teachers will determine the approaches that students adopt. Effective instructional choices will be based upon a sound understanding of how students learn and effective teachers will direct students to interact with content in ways that help students ‘make sense of the material’.

What is Learning?

The authors of Learning as a Generative Activity, Mayer and Fiorelli, define ‘generative learning’ as ‘helping learners to actively make sense of the material so they can build meaningful learning activities to transfer what they have learned to solving new problems.’ This definition is far more detailed than Kirschner’s proposition that ‘learning is a change in long term memory’ and the additional requirement of transfer means that many would reject it  as being too exacting. As Dylan Wiliam has pointed out, if transfer is a necessary condition of learning, then this would imply that memorising your social security number is not an example of learning.

Transfer is notoriously hard to achieve, even within the same domain and at best, we should be attempting to induce near transfer in our students so that their understanding moves from being inflexible and shallow to flexible and deep. David Didau gives a useful overview of the concept of transfer in this blog.

The tracks within DI schemes are planned in order to promote this increase in flexibility and the purpose of teaching to the general case-one of the main aims of DI-is to ensure that students can apply their knowledge to new but similar problems or applications. This can be achieved by manipulating sequences of examples or by making students aware of the similarities and differences between different contexts-essentially helping them to perceive the deep structure of a problem. A well planned instructional sequence will purposively facilitate this transition from inflexible to flexible knowledge as students move from restricted drills to freer application. The table at the end of this post demonstrates this transition. Initial tasks may look much like rote memorization where students merely respond to a cue-perhaps asking student to recall a specific date or fact; later tasks may ask students to develop their understanding-perhaps using elaborative interrogation; later still, tasks may require students to apply these component parts within a wider application-perhaps using the date or fact as part of a chain historical argument. 

So if transfer or flexible understanding are important aspects of learning, why has Kirschner’s definition become so widely accepted? I would argue that his definition has two main strengths. Firstly, its succinct nature encourages teachers to concentrate on the vital role of memory and retention in learning and, in an age where attainment is often judged via close books exams, this can be no bad thing. Secondly, its breadth allows it to encompass the entire continuum from inflexible to flexible knowledge. Both ends of this continuum have important roles to play in learning, and, while ‘flexible knowledge’ is the end goal of almost all instructional sequences, ‘inflexible knowledge’ almost always forms the foundations of understanding. If a definition of learning effectively excludes ‘inflexible knowledge’ by adding the additional requirement of transfer, then this may have adverse effects: teachers may come to believe that rote learning is undesirable or ineffective; equally, teachers might focus entirely on novel problem solving in the belief that the best way to improve this ability is to practice problem solving itself when, counterintuitively, it may be more effective to begin by building student background knowledge which will initially be inflexible. Under Kirschner’s definition, both the rote memorization of your social security number and the ability to solve novel algebraic problems would be defined as learning.

So if Kirschner’s definition has clear strengths, what else can we learn from Fiorella and Mayer’s? Their definition of ‘generative learning’ is ‘helping learners to actively make sense of the material so they can build meaningful learning activities to transfer what they have learned to solving new problems.’ This definition of learning seems aimed at the end goal of most instructional sequences: flexible knowledge and understanding in which students can achieve near transfer. This seems entirely uncontroversial: we want students to be able to flexibly apply their knowledge. When we first learn a concept, our understanding is shallow, especially if it is a concept unrelated to our existing background knowledge. As we begin to make sense of what we have learned, we make connections between the newly learnt concept and our existing knowledge.

In an attempt to further hone their conception of learning, the authors critique a number of others:

Conception 1: Learning works by engaging in hands-on activity, so it is better for you to learn by doing rather than by being told.

Mayer and Fiorelli dismiss this as being overly concerned with behavioural activity and the nature of the learning tasks that student attempt. They point out that this conception fails to focus ‘enough on cognitive activity’ as students could be busily engaged in an activity without purposively thinking about the content in a way that helps them make sense of it. This opinion is analogous to Rob Coe’s assertion that ‘learning happens when you think hard.’ When teachers focus on the activity rather than what students are thinking about, this often results in classrooms that seem busy and purposive but lack any real cognitive processing. A good piece of advice when planning lessons is to always think about what the students will be thinking about. Activities or tasks should be chosen so students can spend as much time as possible thinking as deeply as possible about the content. If seen through the lens of Cognitive Load Theory, tasks should be easy to explain to students with no irrelevant procedures or activities that can either cause confusion or distract students from thinking about the content that they need to ‘actively make sense of.’

Conception 2: Learning works by building association, so you should practice giving the right response over and over.

The authors dismiss this conception of learning as being too prohibitive and only really applying to situations where students are expected to ‘give the right response for a given stimulus’. They concede that this conception is not wrong, but that it is ‘just too limited’ as it in no way ‘deals with understanding’ which they define as when people are able to ‘take what they have learned and apply it in new situations.’ While there are many instances of learning that are described by associative learning (Dylan Wiliam’s social security example fits this well), if this conception were to be a catch all definition, it would fail to describe what is the end goal of most instructional sequences: generalized, flexible understanding.

As an aside, the very term ‘understanding’ seems to be contested: when Fiorellia and Mayer use it, they intend it to include near transfer; Willingham, however, gives a slightly more restrictive definition, defining it as ‘remembering in disguise…What do cognitive psychologists know about how students understand things? The answer is that they understand new ideas (things they don’t know) by relating them to old ideas (things they do know)’ Interestingly, this definition fits exactly with the three stages of cognitive processing that form the theoretical basis of ‘sense making’ and underpin all of the learning strategies that Learning as A Generative Activity explores:

  1. Selecting relevant information to attend to
  2. Organising the material into a coherent cognitive structure in working memory
  3. Integrating it with relevant prior knowledge activated from long term memory

Conception 3: Learning works by adding information to your memory, so you should work hard to find and memorise new information

The authors dismiss this conception because ‘humans do not work like computers’ and ‘we do not simply take in what was presented and put it in our memory’. They also posit that learning involves ‘changing what is presented from information (which is objective) into knowledge (which is subjective). I would argue that it is precisely because we are not ‘like computers’ that we need to adopt proven strategies like distributed practice and the testing effect in order to aid us with encoding and retaining information in our long term memories. While a computer reliably stores information without error as soon as it is asked to, our messy and fallible cognitive processes require the adoption of deliberate and specific learning strategies if we are to retain information. Cognitive Load Theory highlights the absolute importance of background knowledge, making the important point that ‘novices use thinking skills, experts use knowledge’ when attempting problems. Background knowledge also has a huge role in reading comprehension. Memorising or automatizing information can give a huge advantage to students. Times tables are a good example here: even simple algebraic problems are made impossible if students do not have reliable and rapid recall of multiplication facts. No-one, however, would argue that memorisation encompasses learning as a whole.

Conception 4: Learning is a social activity, so it is better for you to learn with others in a group than to learn alone.

Although the authors concede that ‘you can interact with others during the learning process’, and that social activities that promote meaningful learning can be effective, they dismiss this conception because group work is hard to manage and ‘research on group learning tends to show that all group interactions are not equally helpful’. Although group work can be useful if there are both group goals and individual accountability, it can result in social loafing and off-task behaviour. I tend to avoid group work for these very reasons although this does not mean that group work is a poor instructional strategy per se.

In the next post, I will continue to explore Fiorella and Mayer’s conception of learning.

 

Low stakes Quizzing and Retrieval Practice 5: Extended Quizzing

You can find the first four posts about retrieval practice here: one, two, three, four.

In the last post, I explored a number of principles that underpin effective retrieval practice as well as the relationship between the style and type of retrieval questions and the composition of the final assessment. This post will explore some additional retrieval methods.

A recent paper by Pooja Agarwal entitled ‘Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning?’ found that if a test item involved higher order processing, then retrieval practice activities should involve ‘higher order and mixed quizzes’ instead of just fact quizzing. But what do these terms mean? Using the 2001 revised Bloom’s taxonomy, the paper defines ‘higher order learning’ as being the top three layers of the Blooms pyramid: ‘apply, analyse, evaluate and create.’ The paper defines ‘mixed quizzes’ as retrieval practice that involves both factual and higher order questions and posits that ‘mixed complexity during initial learning may serve as an effective alternative to the prevalent ‘factual knowledge first’ standpoint.’ The paper demonstrates that fact quizzing will not, on its own, lead to good outcomes on higher order test items. Because almost all examination questions involve some level of higher order processing, this finding is of crucial importance to teachers.

The optimum retrieval approach to choose at a point in time will be heavily dependent upon the prior knowledge of the students in front of you. At the beginning of an instructional sequence with novice students, it may be appropriate to ask closed fact questions first so that students can acquire the concept or idea that you are teaching. This does not mean that you this is all you should do though. Teachers should continually be asking students to think hard about the concepts that they acquire so that their understanding gradually moves from being inflexible to flexible, helping them make links between different concepts and integrate them with their prior knowledge as well as asking them to apply, analyse, evaluate what is being taught.

So what might a mixed quiz look like?

For students to succeed in extended writing tasks at GCSE-whether these tasks are in History, English or any other subject that involves essay style responses-they need to select, combine and present often complex and abstract information, and then thread it all together into a coherent argument or explanation. To add to the challenge, they also need to do this from memory, retrieving and applying the information under pressure and in a limited time frame. Students need to have lightning quick recall as well as the cognitive dexterity to not only recognize which concepts are best deployed to a specific question, but also the ability to transfer their knowledge to questions that, whilst using novel terminology or phrasing, are essentially asking the same thing as those that they have practiced in class.

Here are some approaches that can help with this challenge:

Listing stuff

Instead of asking a series of unconnected factual questions, students can be asked to chunk information like this:

Teacher: Write down seven adjectives that we have learnt that describe Mr.Birling.

birling 7 adjectives

While asking for a list of facts is clearly more challenging that merely asking for one adjective, this still does not involve students doing anything more than remembering these words. The Agarwal Paper would point out that this disconnect between retrieval task and final test (in this case an essay) would mean that this quiz would be unlikely to help student performance in their Literature GCSE. However, the initial teacher question would then be followed by a number of other questions that would involve higher order processing, ensuring that the retrieval is mixed, an approach that should improve higher order test performance.

Here are some possible follow up retrieval questions that involve higher order questioning:

a) Because/But/So

Teacher: Copy and complete my sentences:

Mr.Birling is haughty because…

Mr.Birling is supercilious so…

b) Because/But/So with additional success criteria

Teacher: Copy and complete my sentences, making sure you include the additional components

Mr.Birling believes he is infallible, but

  • dramatic Irony
  • Birling symbolizes Upper Class
  • 2 pieces of evidence

The regular inclusion of success criteria like these is a useful way to help students combine and integrate components into their responses. If I were to ask the question above, each of the bullet points would have already been taught and retrieved separately beforehand. Unless students have been taught the components here, it is likely that they will not be able to apply them, even in this restrictive activity. As an instructional sequence progresses and students become gradually more competent, two things will slowly change. Firstly, the context of these practice activities will slowly widen: students may start with sentence based practice, then move to paragraphs and then finally attempt full essays. Secondly, these prompts will slowly be removed as students no longer need to be made explicitly aware of the links between components or what is required by the question.

Analytical questioning

Teacher: Why is Mr. Birling haughty?

Evaluative questioning

Teacher: Who is more patronizing? Mrs. B or Mr.B

Extended Quizzing

Self-testing that uses a ‘free recall’ approach (also known as a ‘brain dump’) can be a really effective method and asking students to write down all the ideas that they know about a subject is clearly more challenging than splitting the information up into lots of different questions. However, if an instructional sequence begins with free recall as the method of retrieval, there are two potential problems. Firstly, students may find the task so difficult that they then become demotivated. Secondly, students may dump what they know onto the page in a haphazard and disorganized fashion, preventing them from thinking about and retaining the connections and links between individual ideas.

Similarly, if we merely ask a series of closed unconnected fact recall questions, this approach on its own will not enhance higher order learning as it will not allow students to see how concepts connect with each other, nor will it help them integrate ideas into schemas. For this to happen, teachers need to make these connections explicit to students and one way of doing this is through ‘extended quizzing’.

Let’s look at an example:

extended quiz 1

The screenshot above are the answers to a retrieval quiz about Macbeth. These were the questions that were asked in the order that they were posed:

  • Complete the quotation ‘fair is foul and ……… is …………..’
  • What technique is being used here? ‘chi………’
  • Write down two words that describe the witches language

All these questions are connected and the intention here is to get student to start chunking this information: asking this sequence of questions makes the links between the individual concepts explicit, allowing students to organize and integrate them. Engelmann’s DI programmes contain similar chains of questions, the intention being to not only teach the specific concepts, but to also teach how they fit together in a larger knowledge system. If teachers do not teach these organizational structures and connections, then some students will not make these links. A core tenet of explicit instruction is that teachers teach everything that students will need and teaching the relationship between concepts is a vital part of this.

Here is a second example:

extended quiz 2

The screenshot above is from a retrieval quiz about Jekyll and Hyde, focusing upon Hyde. The red parts are the clues that I gave when I asked the questions (this was to a lower ability set and I expected them to struggle a little); the black parts are the answers that I filled in at the end.

These are the questions that were asked in the initial retrieval quiz in the order that they were posed:

  • How does Enfield describe Hyde’s repulsiveness? (couldn’t specify the point)
  • What two words describe his repulsiveness? (vague, ambiguous)
  • What psychological concept does this relate to? (The Uncanny)
  • Hyde is uncivilized and almost wild, what word can we use to describe this idea? (feral)
  • Complete the quotation that demonstrates this attribute (ape-like fury)
  • Circle the word that demonstrates that Hyde is animalistic (ape)
  • What broader societal unease and fear does this word link to? (Victorian fears of evolution)
  • Hyde’s violence is extreme, what word can we use to describe it? (brutal)
  • Complete the quotation that demonstrates this (audible shattered)
  • Hyde seems to enjoy inflicted pain on people, write down the word we can use for this pleasure (sadistic)
  • Write down another word to describe Hyde’s attack (savage)
  • What does his extreme behaviour suggest? (doesn’t conform to Victorian social mores)

While this example has plenty of clues and hints to help students successfully retrieve the information from memory, these are not always required. As an instructional sequence progresses, these hints and clues can be faded out so that students are retrieving the information with less support and higher attaining classes can dispense with this support at a much earlier point.

I would use extended quizzing as an intermediary approach in between classic short answer, closed questions and free recall. Earlier lessons may see extended quizzing; later lessons may see freer recall style retrieval. While extended quizzing sees the teacher creating the links and organizing the material into a logical structure, free recall shifts the responsibility to the student as they are asked to recall and organize the information themselves.

With essay writing as the final outcome, this table explores the benefits and detriments of different retrieval types:

extended quizzing table

Next Post: Insights from Learning As a Generative Activity

Low stakes Quizzing and Retrieval Practice 4

You can find the first three posts about retrieval practice here: one, two, three.

Everyone seems to be doing retrieval practice now and there is an abundance of research  in support of the effectiveness of self-testing as a learning strategy, particularly with regards to increasing long term retention. Ever since retrieval practice has become popular amongst teachers, there has been a notable concern about how it is being approached and whether or not it really is as effective as its proponents would claim. One line of criticism is that the questions-often closed, recall questions-are nothing like the final performance that students encounter when they take an exam. Merely asking students something along the lines of ‘What word means excessive pride or ambition?’  is, on its own, not going to help students with their understanding of Macbeth. However, understanding the meaning of ‘hubris’ (even in this most restrictive question-answer example) may well be the necessary, inflexible beginning of their journey towards knowing how Macbeth’s hubris is his harmartia. It is the job of teachers to skillfully transform this inflexible, rote knowledge into flexible understanding.

generative

In Learning as a Generative Activity by Fiorella and Mayer, a book that explores 8 learning strategies that promote understanding, self-testing is explained as being an effective study strategy, mirroring the findings of Dunlosky in this AFT paper. One of the strengths of Learning as a Generative Activity is that the authors are careful to outline the boundary conditions under which a strategy is most effective. In the minds of many teachers, retrieval practice has reached the status of ‘universally a good thing’ and this is potentially a problem. Like all pedagogical approaches, the decision when and how to apply it requires thought and judgment. If a strategy reaches the status of ‘100% effective’ then the nuance and theory that supports it will be lost as teachers pursue the surface features, unaware that the deep structure of the approach requires more than the mere robotic delivery of a quiz every single lesson.

Fiorelli and Mayer point out that, for retrieval practice to most effective, there are a number of important things that need to be considered:

  1. Learners need to receive corrective feedback following practice testing

This can act as a laser precise form of AfL as, when corrections are provided, students are able to plug tiny gaps in their knowledge. With instant corrective feedback, students can also benefit from the hyper-correction effect. This is the idea that the more confident students are that their answer is correct, the more likely they are to not repeat the error if they are corrected.

2. Self-testing is often more effective when questions are free-recall or short answer.

Free-recall is also known as a ‘brain dump’ and involves students writing down everything that they know regarding a specific topic.

Here are some examples:

a) Write down everything you know about Hyde

b) Spend 5 minutes writing as much as you can about Hitler’s rise to power.

3.Tests should be taken repeatedly

Distributed practice can massively help with long term retention. If we want students to retain information, then spacing out retrieval practice is crucial. Engelmann highlights it as one of the important shifts of task design-beginning with massed practice and moving to distributed practice. Damien Benney writes in detail about attempting to optimize the spacing gap here.

4. There should be a close match between practice test items and the final test.

Opponents of retrieval practice would point to the disconnect between quizzing and final performance. This is most apparent in subjects where the final assessment is extended writing as there is a stark difference between closed recall questions and essays. Being able to recall that a word beginning with ‘At…’ means relating to characterized by reversion to something ancient or ancestral is in no way going to help a student with writing an essay response that explores how the boys in Lord of the Flies descend into barbarism and savagery. However, proponents of low stakes quizzing would point out that if retrieval practice is being used appropriately, the closed question about ‘atavism’ would not exist on its own, instead being the beginning of a series of questions or being part of a wider recall activity that allows students to make the necessary links between vocabulary, character and theme. If the retrieval practice is effective, the concept of ‘atavism’ would not be retrieved in isolation or seen as an end itself. The teacher would carefully situate it within a wider body of knowledge, asking questions and discussing it in terms of the final test outcome: extended critical interpretation.

A recent paper by Pooja Agarwal entitled ‘Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning?’ explored the efficacy of different forms of retrieval practice and came to similar conclusions to those in Learning as A Generative Activity.

Here are some of Agarwal’s key findings with some commentary:

  1. Closed, unconnected ‘fact’ quizzing will not help students perform well in higher order tasks

If retrieval practice means merely asking a series of closed recall questions, then this activity will probably not lead to successful performance on higher order tasks like extended writing. Mirroring the findings in Learning as a Generative Activity, the paper again stresses the importance of matching the practice test to the final test.

2. Higher order quizzing helps students with higher order testing

We should ask students retrieval questions that span the higher strata in blooms taxonomy. While there may be some contention regarding the strict hierarchical nature of the taxonomy, it is a good idea to ask questions that involve a deeper level of processing than mere factual recall. As an example, the distributed practice of analytical introductions  is a good method of synoptic recall which involves higher order thinking.

I often have an open recall question on the board at the start of a lesson. Because students tend to trickle into the class over a number of minutes, this means that those who arrive the earliest can begin working instantly instead of waiting for all students to get there before we begin a quiz. Also, the tasks are deliberately open ended-I often give them a 5 minute limit- so that low and high attainers can attempt them successfully, the differentiation here being by the depth and complexity of the outcome. I will often follow these open ended tasks with something that looks more like a quiz.

Here are some examples of higher order, open recall questions:

  • Why is Gerald the most sinister character in An Inspector Calls?
  • Think back to London, how do you know that Blake was a Romantic poet from the content of the poem?
  • What kind of woman is The Landlady in Telephone Conversation?
  • What is the connection between The Blackmailer’s Charter and Jekyll and Hyde?

All of these questions are asking for higher order cognitive processing, ensuring that there is a close link between the practice and final test. However, if students are to produce high quality answers to these questions, it is often important to have previously asked them more restrictive retrieval questions on the required components, initially in isolation, then later asking students to make links between the individual items thereby facilitating the integration of the individual concepts. This process reflects the journey from inflexible to flexible knowledge: well planned and carefully sequenced retrieval tasks can help students move along this continuum. While initial quizzing may be factual and restrictive, later retrieval tasks will look far more like what is expected (extended writing). If I were to skip straight to asking open ended retrieval tasks then students may not be able to retrieve and therefore apply the relevant components, precluding them from producing a high quality response.

Let’s look at an example:

  • Why is Gerald the most sinister character in An Inspector Calls?

Assuming a student has attended the lessons where Gerald’s character has been taught, then they will be able to answer this question at some level. If, however, this was the first retrieval question that they were asked, then their answer may lack some of the specific components that the question requires. Teaching students the components and ensuring that they can retrieve and apply these before they are asked to attempt a more complex task may well be a more efficient approach to mastering the content than beginning with a higher order complex retrieval task. If students skip straight to the open ended retrieval task then their poor performance will necessitate complex and detailed feedback in order to close the gap. Not only will this be time consuming, but it may also be very difficult or even impossible for students to take on board the feedback because of the myriad omissions and errors that they made. It may be far more efficient to insist that students retrieve and master each component before attempting to integrate them into a complex task.

Here is a list of some of the components that I want students to be able to include in their answer:

Vocabulary

  • exploitative
  • objectify
  • infidelity
  • unscrupulous
  • disparity
  • benevolence
  • supercilious

Textual References

  • ‘I suppose it was inevitable’
  • ‘young and fresh and charming’
  • ‘made the people find food for her’
  • ‘I didn’t feel about her as she felt about me’
  • ‘I didn’t install her there so that I could make love to her’

 Initial retrieval questions that focus on these components may look like classic closed questions, things like:

  • Which word beginning with EX means the take advantage of someone?
  • Complete the quotation: ‘I suppose it was inev………’

When feeding back with these initial retrieval questions, the teacher should ask a number of follow up questions to ensure that students begin, even at this early stage, to engage in higher order thinking. Although the initial retrieval question may be closed, these follow up questions will have mixed formats.

Here is the original closed single component retrieval question:

1) Which word beginning with EX means the take advantage of someone?

Here are some possible follow up questions:

  1. How does Gerald exploit Eva?
  2. What is it about Gerald that makes his actions so exploitative?
  3. What is the most sinister part of his exploitative behaviour?
  4. Who else exploits Eva?
  5. How were the lower classes exploited in Edwardian society?

In Direct Instruction programmes, individual components- perhaps vocabulary or sentence structures-are ‘firmed’ before students are asked to use them in wider applications. The term ‘firmed’ here refers to accuracy, fluency and retention and students are often expected to demonstrate these stages of learning by applying a concept in a restricted context before they are asked to integrate a concept or skill into something more broad or complex. DI schemes use track planning where many different concepts are being ‘firmed’, each concept moving along a continuum from inflexible to flexible knowledge and slowly being combined and integrated with others. The idea here is that the atomization of content allows students to experience consistently high success rates which can be really motivating, particularly for low attaining students. Equally, it allows the teacher to give instant, precise and effective feedback on each of the components. In the initial stages of learning, instant feedback is really important and if used in conjunction with some form of atomization where components are taught and practiced initially in isolation, then this can help prevent cumulative dysfluency. If students are asked to skip straight to the higher order retrieval question (Why is Gerald the most sinister character in An Inspector Calls) then the danger is that they may make so many errors and omissions that effective feedback becomes impossible.

Effective and efficient instructional sequences will depend upon two important variables. Firstly, the context and type of retrieval activities should begin as restrictive tasks and move slowly towards wider application. Secondly, retrieval will be distributed over time in order to ensure long term retention.

This graph shows the relationship between the two variables and how specific retrieval tasks may be more appropriate at the start or the end of an instructional sequence:

graph

 

At the start of an instructional sequence, a ‘quiz’ of restrictive closed questions may be most appropriate; at the end of a sequence and closer to the final test, wider retrieval tasks like paragraph and essay writing may be more suitable. As time progresses, the retrieval tasks should become wider, eventually mirroring the final test: extended writing.

With essay writing as the final outcome, this table explores the benefits and detriments of different question types:

retrieval practice table.png

Next Post: Retrieval Practice 5: further findings and extended quizzing

Analytical Introductions

Teaching students to write consistently well-structured essays is a vital part of our job as English teachers. Successful analytical writing will be made up of high quality components-precise vocabulary, sophisticated sentence structures, judicious use of evidence and perceptive interpretation-but the one thing that often signals truly exceptional performance is a level of crafting at the whole text level. The best analytical essays will be stuffed full of high quality components but, crucially, they will be organized in a logical and coherent way with a strong line of developing argument that threads through them. Not only that, but the writing will be pitched at a conceptual level, dealing with abstract notions and nominalized ideas. Instead of commenting on a hypocritical character, it will delve into the hypocrisy of an archetype; instead of exploring the unfair treatment of a girl, it will delve into the exploitation of the working class as a whole. Characters become constructs; language becomes symbolic and the tenor of the essay will be pitched far in excess of a mere analytical commentary where students move from quotation to quotation.

Exceptional essays do not begin with fine grained language analysis. Exceptional essays do not dive straight into the actions of a character. Exceptional essays begin with analytical introductions.

Here is an example:

Question: How does the novella explore the ideas of secrecy and the unknown?

Analytical Introductions example 1 Jekyll.png

Analytical Introductions will sketch out the big ideas within a text, often remaining at the level of the conceptual, the abstract and the thematic. They will often touch upon authorial intent and will contain a few succinct, well-chosen quotations to demonstrate that even at this level of abstraction, the interpretations are still based upon a close reading of the text. Appositive sentences lend themselves well to these introductions, allowing students to hit the examiner with thematic commentary from the very beginning.

Asking students to begin essays like this has a number of benefits. Firstly, it ensures that students are instantly writing about conceptual and thematic ideas, preventing them from slipping into the formulaic and prosaic repetition of PEEL paragraphs where a sequence of quotations are dissected and the word ‘connotations’ is lavishly slathered all over the writing as the true mark of critical interpretation. We’ve all read these essays before: they are repetitive and boring and the pages are filled with monotonous and relentless chains of language analysis. If students are to get top marks, they need to write in far more depth and with an appreciation of the big ideas that the text is commenting upon. Secondly, crafting an analytical introduction provides students with a plan. In the example above, each of the underlined words or phrases is not only a potential paragraph, but also the nascent beginnings of a topic sentence for that paragraph. Having the plan contained within the introduction can help students avoid getting carried away with one particular part of their essay. It can also help prevent students from frantically and randomly writing about stuff that they remember in a vain effort to fill the page with relevant content. Instead of students rushing to write about the first quote they remembered, followed by the next quote they remembered (from a different part of the text and about something entirely different), they will be led by the big ideas in the introduction. Thirdly, regular practice of these is a fantastic synoptic retrieval practice exercise.

Analytical introductions work equally well for exam questions with extracts and those without. If students are attempting a question with an extract, I will ask them to begin with the introduction, then deal with the extract, then revert back to the big ideas that they have touched upon in their introduction.

How to Teach Analytical Introductions?

We usually teach these in year 9 and they build upon The Six Skills. Although you can teach far simpler versions of these, for the example above, students would need to be secure in embedding evidence, writing appositives and using ‘not only…but’. Like many other things, initial teaching of these should span a minimum of two lessons with distributed practice spanning many more lessons. Initial lessons should involve the teacher writing and labelling a model, making their thought process explicit throughout by narrating WHY they have written it in the way that they have. In the first lesson, it can be useful to get kids to transform and rearrange a model into their own writing. This will inevitably involve a degree of mimicry but inflexible knowledge is almost always the start point in a sequence of learning.

Lesson 1 and 2

Analytical Introductions example 1 Jekyll

After writing the model, the teach underlines the big ideas, making it clear that these are the potential headings and inchoate topic sentences of conceptual paragraphs. The teacher then asks student to help them make a list of the ideas under the introduction. This list will be a paragraph plan and it is important to make this link clear to students as one of the functions of writing like this is to create a plan to follow. Asking students to use different words in the plan-essentially paraphrasing the introduction-is a useful check for understanding regarding vocabulary and the meaning of these conceptual phrases. The plan might look like this:

BIG IDEAS:

  1. Problems with super strict society
  2. Fixated on manners and how they behave
  3. We are all good and evil
  4. Hyde as a construct
  5. Denial of pleasure
  6. Gothic setting
  7. Paranoia/secrecy

Asking students to paraphrase the ideas also means that when you ask them to write their own, they are more likely to depart from the one above. Once they have compiled their list of big ideas, you can then ask them to reassemble this skeletal plan into an introduction of their own, perhaps changing the order in which they cover the themes and perhaps using different vocabulary or quotations.  Before they being writing, it is worth reminding them of what they should not include:

  • No language analysis
  • No explanations, elaboration or justification
  • No longer than the example

When they have finished writing their own, they can then label it in the same way that the teacher initially did and begin to think about which big idea should be dealt with first. Often, at this stage, it becomes apparent that the order that they appear in the introduction is not the most logical order for the essay. The order will sometimes be dictated by plot chronology; other times it will be because there is a natural link where one idea feeds into another: for instance, in the example above, there is a natural and obvious connection between paranoia/secrecy and problems with a super strict society as well as many other clear links.

Later Lessons

Example-problem pairs work particularly well with teaching this approach. A teacher could present an example like this:

Analytical Introductions example 2 macbeth.png

It can be really useful to group questions for students so that they begin to see the deep structure of what is being asked. Novices often fail to see similarities between questions, instead seeing a task as being unconnected to others. Demonstrating the similarity between tasks makes it far more likely that students will succeed: exam question words often confuse students and this approach can help to mitigate this problem.

The teacher could then ask students to generate the ideas-admittedly the example above could be improved by changing ‘Lady Macbeth’ to something more thematic and conceptual-perhaps manipulation or duplicity. This idea generation is a really useful synoptic retrieval task. The teacher can then write a model answer live:

Analytical Introductions example 3 macbeth.png

Students can then be given a really similar question to attempt, allowing them to use this model as an analogy. The question will be different enough that they cannot just copy this model: this approach is something that is threaded throughout our booklets and is explained in this post

For students to really master this, it will need to be taught across multiple lessons and across the full range of texts that they are expected to respond to. An effective instructional sequence will probably move through the the six shifts of task design and will involve both the alternation strategy as well as backwards fading.

In the latter stages of examination preparation, giving students examination questions and asking them to create these at speed (once they have demonstrated an accurate and reasonably flexible understanding of them that is) can be a really useful way of practicing planning as well as being a useful synoptic retrieval task.

Next Post: Retrieval Practice 4: Extended Quizzing.

The 6 Skills Part 5: Evidence in Explanation

6 skills

Analytical Progression Model

  1. Tentative Language
  2. 3 Part Explanation
  3. Zoom In/Technique
  4. Multiple Interpretations
  5. Evidence in Explanation
  6. Link Across Text

When writing analytically, proficient writers substantiate their analysis by referring directly to the text. While indirect references have their place, the use of actual quotations is crucial. When students arrive at secondary school, one of the most important things that they learn in year 7 is how to embed evidence fluently in their writing so that they are equipped to respond to the texts that they encounter. The ability to use embedded evidence is a threshold concept, an idea that once mastered, changes a student’s relationship to the texts that they encounter. Before they cross this threshold, students are able to talk about evidence and textual references; after the threshold has been crossed, this ability is transformed into written output.

Initially, students should be taught to embed single quotations which they can then build arguments around, adding tentative interpretations, zooming in on words and commenting on techniques.

While this is an important start point, more proficient analysis involves multiple pieces of evidence where arguments are sustained and developed through the addition of further textual references. Sometimes this development can take the form of several references in support of one idea; sometimes it can involve a number of sequential references that contribute to and develop a line of argument; alternatively, subsequent pieces of evidence can be judiciously added in order to subtly demonstrate that the ideas within the argument are closely matched to the text itself.

Here are some examples with commentary:

  1. Multiple bits of evidence that contribute to one point:

Priestley uses dramatic irony to undermine the Birling’s close minded views on the world through phrases such as ‘the Germans don’t want war’ and referring to the titanic as ‘unsinkable.’

In this example, the use of two quotations potentially provides a more substantiated argument. It also demonstrates that the writer is able to notice patterns-in this case both Birling’s habit of pompous pontification as well as Priestley’s frequent usage of dramatic irony. The ability to collate and combine evidence is dependent on two variables: the amount of text that you are using as your evidence base as well as the depth of knowledge that you have of the text. If you set aside the relative complexity of the language contained within a text-Shakespeare is almost always going to be harder than Priestley for students- it is often harder to collate evidence from across an entire text than it is from two sentences, one speech or one page. The former requires a knowledge of far more content. The narrower the source, the easier it is to see patterns; the wider the source, the harder it is to do so.

2) Multiple bits of evidence that sustain an argument-each piece complements the one that precedes it, building an argument and developing a point across a paragraph.

Tempted by the witches prophecy, Macbeth seems torn and conflicted by their words, exclaiming ‘cannot be ill, cannot be good.’ It is as if he desires the throne but recognises that to achieve it, he would need to commit the worst crime imaginable: regicide. When he says ‘whose murder yet is but fantastical’, the word ‘yet’ foreshadows the upcoming crime. Macbeth has very quickly moved from being intrigued by the witches’ language to a position of certainty where Duncan’s murder seems inevitable. Perhaps Shakespeare is exploring the power of greed and how hubris can quickly dominate a character’s judgment.  

In this example, the writer is using pieces of evidence sequentially, each building upon the previous and each contributing to one chain of argument or a wider idea. In this case, the writer is analysing Macbeth’s initial reactions to the witches equivocal language and commenting upon the malign influence of hubris.

What to teach?

3) Evidence in explanation:

Birling is an Upper Class man who revels at the idea of ‘lower costs and higher prices’, a selfish notion that conveys Priestley’s argument that the rich exploited the lower classes and treated them as ‘cheap labour.’

He has passed the point of turning back and is in ‘blood stepped in so far’ that his reign causes Scotland to suffer ‘under a hand accursed’

Enfield, like all the upper class men, is singularly focused on his own reputation; he avoids discussion and does not pry into the lives of others because ‘asking a question is like starting a stone’, a statement that conveys his fear of gossip and his desire to avoid ‘disgrace.’

Why Teach it?

In these three models, the final underlined quotation is an example of ‘evidence in explanation’. This approach to using evidence is slightly different as, instead of helping to substantiate or develop and argument, it is being used to justify an argument. When using evidence in this way, the quotation will often come at the end of a line of reasoning, demonstrating that the ideas and interpretations are closely matched to the text itself. If a line of reasoning ends with a quotation-or has one embedded within its final conjecture-then it reads as if the ideas are defensible, lending them a certain credence and making them easier for a reader to accept.

evidence table2

Successful analytical writing treads a narrow line between the need for perceptive and interesting interpretation-essentially the input of the reader-and the need to ensure that these interpretations are based upon textual evidence. If it sways too far from the text, then it can become baseless, absurd or even just plain wrong; if it is swamped with excessive quotations, then analysis and interpretation find no room to breath. A successful essay will achieve this fine balance. It will-to use the language of mark schemes-contain the ‘judicious use of precise references to support interpretation’ and hopefully will avoid the repetitive chaining of PEEL paragraphs where the monotony of the restrictive and clunky structure diminishes the overall quality of the writing. In the absence of a PEEL type structure, students are free to use evidence more flexibly and creatively. While I have touched upon three possible types above, there are undoubtedly many more that could be modelled and practised.

Each category mentioned above is not clear cut and ‘evidence in explanation’ can often be conflated with the sequential use of quotations-this is partly because the process of substantiation is minimally different to justification. While substantiation means to verify through evidence and justify means to explain, in reality the lines between both are blurred. You can explain something by using evidence: a quotation can form an integral part of an explanation. Equally, you can substantiate something by providing further explanation. This ambiguity is one of the reasons why proficient analytical writing can take myriad forms: there are innumerable ways of demonstrating analytical skill and each successful essay will combine a variety of approaches to using evidence. Because of this, helping students to recognise, critique and practice different ways of using evidence can be really helpful in developing their analytical writing.

Next Post: Analytical Introductions-How to ensure students write conceptually.