Learning by Spaced Repetition

For me, the medical learning process goes something like this: First I’ll come across something that motivates me to extend my knowledge — whether that’s something I’ve seen at work, something I’ve read, or something I’ve heard about from a colleague or podcast. Then I’ll go away and swot up. I use various sources including journals (via databases like pubmed or MDConsult), books, blogs, podcasts or online videos that cover the subject of interest. Once I’ve collated the necessary information, long-term retention seems to come from relearning the information after I’ve forgotten it and had to find it again (doh!), actively discussing the issues with my colleagues, teachers and students and — for the past few years — writing about it here on LITFL.

When it comes to exams, I confess that I’ve always been a crammer in the build up to test day. Cramming certainly has merits. In the short term it is probably is the most effective way of stuffing a large number of facts into your head. The downside, is that information learned by cramming (or ‘mass presentation’) is poorly retained in the long-term. In fact, the decline in retention over time is exponential! This inevitable decay in our ability to recall memorised information was discovered by a chap called Hermann Ebbinghaus in the late 1800s… Hence the ‘Ebbinghaus Forgetting Curve’:

For medical students and doctors (and their patients!), this is important. We should not be learning information for a one-off test. There are many facts that we need to carry in our memories to function as experts in our profession. Also, more prosaically, over the course of our careers we are subjected to repeated examinations that require us to recall facts previously learned and to build on previous knowledge.

So, what is the answer?

In addition to actively discussing, using and teaching the information we learn from clinical practice, I think learning by spaced repetition is a useful strategy. I have been experimenting with this over the past year or so.

What is learning by spaced repetition?

Spaced repetition utilises the spacing effect, which comes from an observation made by Ebbinghaus over a hundred years ago:

 …with any considerable number of repetitions a suitable distribution of them over a space of time is decidedly more advantageous than the massing of them at a single time.

Hermann Ebbinghaus, 1885; 89

Gwern Branwen provides an excellent description of how the spacing effect works:

 The spacing effect essentially says that if you have a question (“What is the fifth letter in this random sequence you learned?”), and you can only study it, say, 5 times, then your memory of the answer (‘e’) will be strongest if you spread your 5 tries out over a long period of time – days, weeks, and months. One of the worst things you can do is blow your 5 tries within a day or two. You can think of the ‘forgetting curve’ as being like a chart of radioactive half-lives: each review bumps your memory up in strength 50% of the chart, say, but review doesn’t do very much in the early days because the memory simply hasn’t decayed very much!

www.gwern.net

In other words, if you are a busy person and need to commit something to memory, you should space out the repetitions to improve retention. This is shown graphically below:

How spaced repetition improves recall. It is also clear, that active recall is far more effective than passively rereading information. This means that we need to test ourselves. Indeed, psychologists call this the ‘testing effect‘, and it was well described by Francis Bacon in the Novum Organum centuries ago:

 “If you read a piece of text through twenty times, you will not learn it by heart so easily as if you read it ten times while attempting to recite from time to time and consulting the text when your memory fails.

Francis Bacon, Novum Organum. 1620; 171

How effective is spaced repetition?

This seems to be difficult to quantify. A meta-analysis in 1999 suggested that those who learn information by spaced repetition will outperform 67% of those who learn by mass presentation given the same number of practice episodes. Of course, this will vary according to the “nature of the task being practiced, the inter-trial time interval, and the interaction between these two variables”. Also, the studies varied in methodological rigour.

•  Donovan JJ, Radosevich DJ. A meta-analytic review of the distribution of practice effect: Now you see it, now you don’t. Journal of Applied Psychology, 1999; 84(5), 795-805.

How can spaced repetition be put into practice?

The simple answer is: use computer software.

There are a number of programs available. I have used Mnemosyne and Anki and found them to be comparable. Both are free. However, when I decided to get serious about putting spaced repetition software into practice I committed to Anki and paid the additional ~$US25 for the iphone app.

Why did I do this?

Anki uses an algorithm to predict when you need to be re-exposed to a flashcard for optimal retention. When you have tested yourself you can grade the difficulty you had in remembering the answer with a touch of a button, and Anki will put the card back into the virtual pack in the correct place. The algorithm uses an ‘expanding spacing’ approach. This means that the more times you review a card the greater the time interval until the next review (unless you couldn’t remember it, then it moves back to the front of the cue!). There appears to be an ongoing controversy about how best to space trials for longterm recall, but Anki’s algorithm is probably as effective and as efficient as any other.

There is no way I could realistically do spaced repetition without a computer program. Like most people I naturally tend to review information that I find easiest or most interesting. Also I would no doubt lose my cards if they were physical rather than virtual, and I’d certainly have trouble keeping them in sequence. Anki not only helps space the cards appropriately (though I’m sure the algorithm is not perfect) it helps the learner stay disciplined so that you focus on your weaknesses and learn the cards you need to learn.

For me, the iphone app — though somewhat costly for an app — was well worth the price. It enables reviews of cards during the wasted moments of the day — whether that be waiting for a train or standing in a queue, or getting in 60 seconds of revision between patients. Furthermore, the iphone app syncs with ‘the cloud’ and my personal computer, so that it remembers what cards I have reviewed wherever, and however, I want to review them.

How to create flashcards for spaced repetition?

It is easy to create flashcards using software like Anki. You can insert or cut-and-paste text, images and even audio files. However, it is important that you create your cards wisely.

Spaced repetition seems to work best for the memorisation of conceptually simple facts and lists. The greater the complexity, the less useful spaced repetition will be. Thus it is most effective for simple facts (e.g. What is the NNT for post-resuscitation therapeutic hypothermia?), OK for word lists (e.g. What are the causes of splenomegaly?)  and much less so for complex, skill-based tasks.

For example, becoming proficient in ECG interpretation is a complex skill-based task. Learners needs to constantly expose themselves to ECG interpretation to become proficient (get over to the ECG library now!). Constant deliberate practice is needed to become an ECG expert (think of Ericsson’s 10,000 hour rule for becoming an expert, as popularised by Malcolm Gladwell). Yet certain facts about ECGs can be learned using spaced repetition (e.g. What is the normal QT interval?, or what is the differential diagnosis of ST elevation?). Hence even the learning of complex, skill-based tasks can be enhanced by the appropriate use of spaced repetition for fact retention.

Flashcards for spaced repetition must use the ‘test effect’. They should be designed to trigger active recall, rather than simple recognition. A question and answer format (a la the case-based Q&As) works well for this. Furthermore the questions and answers should be clear and unambiguous. I also suggest avoiding the use of multi-choice questions in a spaced repetition learning system. MCQs are potentially dangerous for learning as the exposure to false answers runs the risks of retaining incorrect information. When practicing MCQs it is essential that you have access to the answers, or go to the trouble to find out the correct answers soon after testing yourself.

Finally, spaced repetition improves recall, but only once you’ve already learnt something! You need to go to the trouble of really nutting things out and learning the information in the first place, for spaced repetition to help you retain it. Remember to be selective when choosing what facts you need to memorise… Only fill your brain up with facts that you can’t look up in real time, need to know for a test or that will impress your friends and colleagues!

 References

• Donovan JJ, Radosevich DJ. A meta-analytic review of the distribution of practice effect: Now you see it, now you don’t. Journal of Applied Psychology, 1999; 84(5), 795-805.
• Spaced repetition (www.gwern.net) [highly recommended review of spaced repetition]
• Stahl SM et al. Play it Again: The Master Psychopharmacology Program as an Example of Interval Learning in Bite-Sized Portions. CNS Spectr. 2010 Aug;15(8):491-504
• Ebbinghaus H. Über das Gedächtnis. Untersuchungen zur experimentellen Psychologie. 1885
• Ebbinghaus H. Memory; a contribution to experimental psychology. 1913 [English translation HA Ruger]
• Bacon F. The New Organon: or True Directions Concerning the Interpretation of Nature. 1620 [PDF translation 2000]
• Wikipedia — active recall, cramming, forgetting curve, spaced repetition, spacing effect, testing effect.
• Wolf, G. Want to Remember Everything You’ll Ever Learn? Surrender to This Algorithm. Wired 16.05 [Article about the inventor of Supermemo, Piotr Wozniak]
• Wiesbauer F. Teaching Masterclass: The Psychology of Learning. Medmastery