Synthesis of OER Efficacy and Perceptions Research: 2015-2016

This presentation reviews nine studies of the efficacy and perceptions of OER that were published in 2016.

1. A Synthesis of OER Efficacy and Perceptions Research: 2015-2016 John Hilton III, PhD Brigham Young University @ johnhiltoniii By John Hilton III . This work is licensed under the Creative Commons Attribution 4.0 International License , except where noted.

2. In 2015 and 2016 there were… Four peer-reviewed studies published that focused on comparing how students performed when using OER versus traditional resources Three peer-reviewed studies (plus two white papers) published that focused on comparing user perceptions of OER with traditional resources

3. Why Should We Care About OER Research?

4. The Babson 2014 survey found that college professors rate “proven efficacy” and “trusted quality” as the number 1 and number 2 most important criteria for selecting teaching resources. -One key way to establish efficacy an quality is through research.

5. Image by Petr Kratochvil  ( Website )

6. "Sacramento City College has a large percentage of its student body who are below the poverty line. When I require students to spend $78 on a book, many of them simply cannot. Having a free textbook is a matter of equity, it is allowing low-income students to have an equal shot at earning a good grade! I had a young woman in my office today, crying that she had to choose between feeding her children and buying books for school! Because of [an OER], she doesn't have to pay for a book in my class!” Jessica Coppola (Sacramento City College) Public Domain Image Courtesy of George Hodan

7. If you didn't have to pay for textbooks, how would you use the money you saved?

8. If you didn't have to pay for textbooks, how would you use the money you saved? “Fresh fruits and vegetables. Money is extremely tight now so I am eating mostly rice and beans with some cheese. I miss eating fresh food .”

9. Additional Comments on Textbook Prices Comment Category Number of Responses Sample Comment Positive 12 (2%) “I appreciate keeping textbooks when it was a good, thought out, well written choice.”

10. Additional Comments on Textbook Prices Comment Category Number of Responses Sample Comment Positive 12 (2%) “I appreciate keeping textbooks when it was a good, thought out, well written choice.” Neutral 49 (8%) “I…like it when teachers use free online articles instead.”

11. Additional Comments on Textbook Prices Comment Category Number of Responses Sample Comment Positive 12 (2%) “I appreciate keeping textbooks when it was a good, thought out, well written choice.” Neutral 49 (8%) “I…like it when teachers use free online articles instead.” Negative 339 (56%) “New editions don't usually change enough to be worth the cost.”

12. Additional Comments on Textbook Prices Comment Category Number of Responses Sample Comment Positive 12 (2%) “I appreciate keeping textbooks when it was a good, thought out, well written choice.” Neutral 49 (8%) “I…like it when teachers use free online articles instead.” Negative 339 (56%) “New editions don't usually change enough to be worth the cost.” Extremely Negative 223 (37%) “Textbooks are the biggest scam targeting the poorest demographic. Requiring specific editions is a gross abuse of power for monetary gain.”

13. Let Me Tell You Some Stories About Efficacy You can see http://openedgroup.org/review if you’d like more details on these research studies.

14. Assessing the Impact and Efficacy of the Open-Access ChemWiki Textbook Project. Allen, G., Guzman-Alvarez, A., Molinaro , M., Larsen, D. (2015) . Educause Learning Initiative Brief, January 2015.

15. Same semester Same teacher Same lecture Same homework Same TAs Same exams Back to back class periods 478 students 448 students FREE 290.05 Total cost > 130,000 Total cost = 0.00

16. A Multi -Institutional Study of the Impact of Open Textbook Adoption o n the Learning Outcomes of Post-secondary Students Fischer, Hilton, Robinson, and Wiley Journal of Computing in Higher Education (2015)

17. OER Users took 15% More Credits

18. 1 control class did better than treatment 5 treatment classes did better than control 9 classes = no sig. difference

19. “The Tidewater Z-Degree and the INTRO Model for Sustaining OER Adoption.” Wiley, Williams, DeMarte , Hilton Education Policy Analysis Archives , 24(41), pp.1-12.

20. When a student drops, it.. Slows down their graduation Costs the institution tuition dollars (refunds)

21. If all business courses at TCC had the lower drop rate, TCC would retain $ 101,042 in additional income each year.

22. Improving Course Throughput Rates and Open Educational Resources: Results from the Z Degree Program at Tidewater Community College Hilton, Fischer, Wiley, and Williams Accepted in the International Review of Research in Open and Distance Learning

23. Course Throughput Rate IRRODL (in press) Drop Deadline Withdraw Deadline Final Grade Students

24. Commercial vs OER 2.3% | 1.8 % 9.9% | 8.1 % 68% | 74% (Face to Face) 59.8% | 66.4% Drop Withdraw C or Better CTR IRRODL (in press)

25. Commercial vs OER 4.0% | 1.4% 13.7% | 13.1% 66% | 70% (Online) Drop Withdraw C or Better IRRODL (in press) 54.2% | 59.8 % CTR

26. We’ve Just Heard Four Stories About… A dozen institutions Thousands of students Image Credit: Tulane Public Relations

27. When using OER students Dropped fewer courses Were less likely to withdraw We more likely to pass their classes Took more classes Saved millions of dollars

28. Let Me Tell You Some Stories About Perceptions of Quality You can see http://openedgroup.org/review if you’d like more details on these research studies.

29. Exploring faculty use of open educational resources at British Columbia post-secondary institutions. Jhangiani, R. S., Pitt, R., Hendricks, C., Key, J., & Lalonde , C. (2016). BC Campus Research Report. Victoria, BC.

30. Educators who had adopted OER rated the quality of OER as significantly higher than those who had not adopted OER [1.88 vs. 3.28 on a 5-point Likert scale].

31. Mainstreaming Open Textbooks: Educator Perspectives on the Impact of OpenStax College open textbooks. Pitt , R. The International Review of Research in Open And Distributed Learning , 16 (4).

32. 79 respondents (~ 2/3 of the sample) strongly agreed or agreed that use of OER such as OpenStax increased learners’ satisfaction with the learning experience.

33. University students and faculty have positive perceptions of open/alternative resources and their utilization in a textbook replacement initiative. Delimont , N., Turtle, E. C., Bennett, A., Adhikari , K., & Lindshield, B. L. (2016) . Research in Learning Technology ,  24 .

34. Researchers surveyed 524 students in thirteen different courses at Kansas State University regarding their use of OER. They found: Students used the OER more than commercial textbooks. Students rated the OER as “good quality.” Students preferred using OER to buying textbooks.

35. Thirteen instructors were interviewed; all but one said they “preferred teaching their course with [OER] instead of a traditional textbook .”

36. Examining Student Perceptions of an Open Statistics Book Illowsky, B. S., Hilton III, J., Whiting, J., & Ackerman, J. D ., Open Praxis , 8(3): 265-277.

37. Across two years, 325 students at one college were surveyed regarding thei r use on an Open Statistics Textbook

38. OER Adoption Study: Using Open Educational Resources in the College Classroom The California OER Council (2016). OER Adoption Study: Using Open Educational Resources in the College Classroom.

39. 16 faculty shared their perceptions about their use of OER Seven faculty of sixteen felt that the OER textbook was superior to the traditional textbook for the course. Five faculty rated the OER as equivalent to the traditional textbook. Faculty were not as positive about the support materials (PowerPoints, Test banks) available with the OER textbooks. Half of the faculty felt that the support materials lacked quality. 25% of faculty felt that implementing the support materials took a significant amount of time. In their comments, the biggest comment made by faculty was about the need for support materials or the amount of time they spent in developing them for this adoption.

40. 351 students completed a survey about their use of OER.  How does OER Compare to Commercial Textbooks?

41. Of the 351 students, 100% wanted to use OER textbooks in the future and would recommend the use of OER to friends. 100%

42. Across multiple recent perceptions studies, researchers found… Students prefer OER Faculty prefer OER OER helps students

43. 14 Peer Reviewed Studies of Perceptions of OER Quality http:// openedgroup.org /

44. 6,050 Professors and Students http:// openedgroup.org /

45. 35% Better 15% Worse http:// openedgroup.org /

46. 13 Peer Reviewed Studies of Efficacy http://openedgroup.org/

47. 119,720 Students http:// openedgroup.org /

48. 95% Same or Better Outcomes http:// openedgroup.org /

49. Overall Takeaways From the dawn of time until today there have been 24 efficacy and/or perception studies that compare OER with commercial products. Nine of these have been done in the last two years…not bad!

50. Overall Takeaways From the dawn of time until today there have been 24 efficacy and/or perception studies that compare OER with commercial products. Ten of these have been done in the last two years…not bad! But we need more – and increasingly rigorous – studies.

51. How can we do better research? http://openedgroup.org/toolkit Control for teacher effect Interrupted Time Series Same teacher using both resources in the same semester

52. How can we do better research? http://openedgroup.org/toolkit Control for student differences Statistically control for differences Pre-post test to determine initial differences

53. How can we do better research? http://openedgroup.org/toolkit When doing perception studies, specifically ask for comparisons between open and commercial resources. Consider asking questions about “use” in perceptions studies.

54. Take Away Message We need more research and also more stories of impact. Please gather those stories and distribute them however you can. Help us see the human side of the incredible benefits that come from replacing commercial resources with open ones.

55. A Synthesis of OER Efficacy and Perceptions Research : 2015-2016 John Hilton III, PhD Brigham Young University @ johnhiltoniii By John Hilton III . This work is licensed under the Creative Commons Attribution 4.0 International License , except where noted.