This is the new m*th! 2024 edition (researchEd Belfast 28-9-24)

These are the slides for the researchEd Belfast session on 28 September 2024. Timing was slightly off; had to rush a few things...but you get the idea! :-)

1. This is the New M* th ! Christian Bokhove University of Southamptom 28 September 2024 2024 edition!

2. Purpose and disclaimers Combatting myths is a worthwhile endeavour… But….we run the risk of creating other myths in the process I want to frame myths and mechanisms first …and then give some examples I encountered in more than a decade of research use Some greatest hits Being critical does not mean I don’t value the work I don’t reduce text to almost nothing on slides, nor will be silent so you can read all of it – narrative (transience trumps modality) Ironically, a presentation like this simplifies, which is a risk in communicating concepts and ideas

3. What can we say about myths

4. https://en.oxforddictionaries.com/definition/myth

5. Howard-Jones (2014) “In 2002, the Brain and Learning project of the UK’s Organization of Economic Co‑operation and Development (OECD) drew attention to the many misconceptions about the mind and brain that arise outside of the medical and scientific communities.” (Howard-Jones, 2014, p. 817)

6. Myths begin “examples of cases in which entrepreneurs have knowingly set out to mislead educators are difficult to find.” (Howard-Jones, 2014, p. 817) “more likely that such interventions originate from uninformed interpretations of genuine scientific facts and are promoted by victims of their own wishful thinking who hold a “sincere but deluded fixation on some eccentric theory that the holder is absolutely sure will revolutionize science and society” (Howard-Jones, 2014, p. 817)

7. Perpetuated Cultural conditions e.g. differences in terminology and language Also check Lilienfeld et al. (2015, 2017) for lists with psychological terms to avoid and pairs of confusion Counter-evidence difficult to access Untestable Biases Complex

8. Rekdal (2014) Case example of urban legend spinach and iron Not significantly more iron Not first food if iron deficient The truth is too simple

9. The truth is too simple But in fact, Larsson cited Hamblin (1981) Treasure hunt Can add more references but sometimes back to one source

10. Hamblin (1981) But the decimal point claim has no reference.

11. Irony Frontline of the fight against bad science and academic carelessness. Sutton (2010) argued other possible causes. Pointed to another person, Bender etc etc

12. Hamblin to Sutton on his website

13. “Follies and fallacies in medicine”

14. https://www.psychologytoday.com/blog/ulterior-motives/201709/the-pitfalls-popularizing-new-science?amp

15. “At the same time, I think that there are real dangers in popularizing science.  One danger comes from oversimplifying core concepts where people may come to believe they understand a key concept better than they actually do.  A potentially bigger danger comes from overhyping new science .” “When scientists want to make recommendations for how people might live their lives differently based on studies, then, we ought to wait about 15 years before giving those recommendations .  Otherwise, we run the risk of giving bad advice that we have to walk back later.  Having to take back our advice can undermine the public’s faith in the science.” https://www.psychologytoday.com/blog/ulterior-motives/201709/the-pitfalls-popularizing-new-science?amp

16. SOME EXAMPLES

17. A set mind

18. Instruction and Einstellung 80s work Sweller (Cognitive Load Theory) is fantastic. But interestingly, not all its features communicated – not even by Sweller himself. I therefore now want to dive deeper into a 1982 article. Like often with older psychology studies, it has a small sample size. According to Sweller (2016) himself these problems are at the basis of his theory. Sweller , J. (2016). Story of a research program.  Education Review ,  23 .

19. Problems 2a Start 60 81 34 35 35 31 Goal 111 453 21 156 315 3 Solution x3 -69 x3 Actions -69 The a in 2a denotes ‘alternating’ x3 -69 x3 -69 x3 -69 x3 -69 x3 -69 x3 -69 x3 -69 x3 x3 x3 -69 x3 -69 x3 -69 x3 -69 x3 -69 6a 10a 2c 4a (1) 4a (2)

20. The history-cued (H) group was reminded of previous solutions. The other group (M) was just given the problem. For example, after solving 6a, they would be told: The solution of the first problem was x – The solution of the second problem was x – x – The solution of the third problem was x – x – The solution of the fourth problem was x – x – x –

21. Results from the paper M takes much longer for 10a! But H does much longer over 2c! 35 315 x3 x3 2c

22. Einstellung “Often called a problem solving set, Einstellung refers to a person's predisposition to solve a given problem in a specific manner even though better or more appropriate methods of solving the problem exist.” Algebra Chess So, need to balance ‘just telling’ and preventing a set mind. https://www.sciencedirect.com/science/article/pii/S0010027708001133

23. Teaching approaches

24. Rosenshine Originally International Bureau of Education, UNESCO, 2010 Popularised by American Educator article called ‘Principles of Instruction’ in 2012 Referenced in the ITT& ECT Framework (2024) https://www.aft.org/sites/default/files/periodicals/Rosenshine.pdf

25. Rosenshine References a bit dated. But… Can’t go much wrong with the recommendations Correlational though ‘I do, we do, you do’ Age of reference     Reference type   1950s 1   journal article 13 1960s   book chapter 3 1970s 7   book chapter 7 1980s 7   report 3 1990s 8      

26. Risk survivorship bias “the logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility.” Example WW2 (Wald) Successful teachers Successful schools Column

27. Following up references Challenges with Rosenshine : Does not confirm to normal conventions in the scientific literature, namely that you reference a claim where the claim is made. Principles are only accompanied by two or three references at the end of the sections Many principles also are rather underspecified, for example feedback.

28. Obtain a high success rate : It is important for students to achieve a high success rate during classroom instruction . The research also suggests that the optimal success rate for fostering student achievement appears to be about 80 percent. A success rate of 80 percent shows that students are learning the material , and it also shows that the students are challenged .

29. Top and bottom 9 teachers of a total sample of 100 teachers. 83% for the most successful teachers, 76% for the least successful teachers? Grade 4 maths teachers I haven't been able to find support for it that really distinguishes effective from ineffective teachers, but as Rosenshine's referencing is unclear, we can always believe there is a magical source somewhere...

30. Project Follow Through *Not* an experiment, but more a social experiment Planned Variation The Direct Instruction Model *did* do well on most of the outcomes Balance *was* an issue Variability *was* great These statements can all be true

31. Statistics and more Only if we have the time…

32. Woo you with statistics Labaree in 2011 in “ THE LURE OF STATISTICS FOR EDUCATIONAL RESEARCHERS” History of ‘statistics’ From German Statistik , from New Latin statisticum (“of the state”) and Italian statista (“statesman, politician”). Statistik introduced by Gottfried Achenwall (1749), originally designated the analysis of data about the state. Aim was to improve credibility and stature, policy influence.

33. Complex diagram or table (Muijs & Bokhove, 2017)

34. I thought about adding… Achievement – Motivation PISA to ‘prove’ the superiority of teaching approaches Generic v domain-specific knowledge Experts v novices - relative expert is the term Advantages of slides with the words you say Fast reading …but that’s for another day…

35. conclusions

36. Conclusions #1 – how do we prevent this? Follow-up sources But… it can be very time-consuming Read, read and read (and sometimes refrain from a position until read up a bit) Beware of over-simplifications Note that fact some over-complicate things, does not mean ‘simple is best’. Nuance is ok, and not always ‘evading debate’. Assume every discussion has been had in the past already

37. Conclusions #2 – how do we prevent this? 15yr rule might be a bit too much but add scope and disclaimers to claims We love a medical analogy: provide a ‘prescription’ why it works, how to use, side effects Educate, e.g. “These results suggest that further training in science may help people to better understand what makes something a good explanation, possibly mitigating the reductive allure effect.” (Hopkins et al., 2016, p. 75)

38. Questions How can we best retain such a level of criticality without: Needing to study for years first? Without antagonising each other? Calling everything a myth beforehand? Have I perpuated certain myths? @ cbokhove