A comparative analysis of i-Ready, IXL, and Prodigy learning software programs

A comparative analysis of i-Ready, IXL, and Prodigy learning software programs

1. A comparative analysis of i-Ready, IXL, and Prodigy learning software programs Ernest J-A. Conerly, Christopher L. Jarrett, and Cristina M. Ryter Biscayne College of Liberal Arts, Social Science and Education, St. Thomas University EDU 650: Practicum in Instructional Design & Technology Dr. Timothy Stafford April 25, 2021

2. Abstract Teachers are required to differentiate instructions in order to meet the individual needs of students. Math learning platforms are marketed to educators claiming to be self-paced, data-driven, and researched-based theoretical frameworks that meet the needs of each student. Research examines three well-known math learning platforms, i-Ready, IXL, and Prodigy, with learning theories that support the success of each program.

3. Abstract Cont’d All three programs offer individual student differentiation and academic improvements. No conclusive data on which program produces the greatest result in student growth. T his study will make a recommendation for which program demonstrates higher student growth.

4. Introduction The average National Assessment of Educational Progress’ (NAEP) math assessment score in 2012 for 17-year-olds was not significantly different from the score in 1973 (National Center for Education Statistics, 2013, p.29). Academic learning gaps are not closing as quickly as NCLB and ESSA planned. i-Ready, IXL, and Prodigy have gained popularity because of online learning which promise to achieve learning gains. This lends itself to research question, What is the causal-comparative effect of math-based e-learning platforms, i-Ready, IXL, or Prodigy based on student performance on the math portions of the NAEP?

5. Background COVID-19 has accelerated the pace at which schools have moved towards digital resources. Districts are under pressure to integrate technology. Increased mandates for differentiating instruction, given rise to educational software resources marketed as “evidence-based” and “data-driven” to prove their effectiveness for increasing student achievement. Research evaluating the effectiveness of these e-learning academic programs has proven difficult to achieve.

6. Background Cont’d Difficult to complete a true controlled research study due to school reluctance. Important to gather as much field tested data as possible to ensure that districts are providing students with the most effective resources available.

7. Problem Needing to be Solved There is a lack of academic progress and achievement gains in math from American students compared to European and Asian peers. Results from 2015 Programme for International Student Assessment (PISA) stated, the U.S. placed an unimpressive 38th out of 71 countries in math. T his problem is due to the large number of unmastered skills that contribute to a lack of proficiency on standardized assessments.

8. Problems Needing to be Solved Cont’d High-stakes testing has deepened the lack of academic progress by forcing chronically low achievers to reach unattainable growth standards without providing necessary resources to close learning gaps and reach proficiency to succeed on math portions of mandated tests. Data from testing fueled an over-testing explosion but student growth did not improve. Which led to teaching to the test, spending more time and attention on test preparation, creating a never-ending cycle of breeding more tests.

9. Purpose of the Research To analyze the three math educational software platforms, i-Ready, IXL, and Prodigy improve student achievement on the NAEP. Math is often the hardest of the 4 core subjects to differentiate, however, math learning platforms are marketed to educators claiming to be self-paced, data-driven, researched-based theoretical frameworks that meet the needs of each student.

10. Purpose of Research Cont’d The research will d etermine which platform will have the greater impact on a student's overall growth and performance on standardized assessments, Which will provide a scope of focus to determine whether that particular type of instructional technology should be used within classrooms.

11. Research Question What is the causal-comparative effect of math-based elearning platforms, i-Ready, IXL, or Prodigy on student performance on the math portion of the National Assessment of Education Progress?

12. Hypothesis H 1 1: All three e-learning platforms, i-Ready, IXL, and Prodigy will have a greater effect than the control group. H : The control group will do better than the three e-learning platforms, i-Ready, IXL, and Prodigy.

13. Hypothesis Cont’d H 2 : Prodigy uniquely applies the most learning theories to its platform, intertwining cognitivism and behaviorism which will lead to self efficacy and have a greater impact than i-Ready, IXL, and the control group. H : All three e-learning platforms are essentially the same and none are better than the others.

14. Literature Review This literature review examines how standardized testing gave rise to the e-learning industry because of the need to differentiate instruction, fill learning gaps, and prepare students for standardized assessments given in each state and nationally every four years. The study looks to identify whether or not e-learning platforms, specifically math-based, are meeting the expectations that brought them into existence. Are students making substantial academic gains on standardized assessments after learning, playing, or practicing on these platforms? If so, how are these platforms accomplishing these improvements?

15. Review of the Historical Literature Programme for International Student Assessment (PISA) results from 2015, the U.S. placed an unimpressive 38th out of 71 countries in math. Among the 35 members of the Organization for Economic Cooperation and Development, which sponsors the PISA initiative, the U.S. ranked 30th in math. (Desilver, 2017). Congress requiring states to bring all students to the “proficient level” on state tests by the 2013-14 school year, although each state was able to decide, individually, what “proficiency” would look like(Klein, 2015).. A new never-ending cycle began where tests bred more tests, including practice tests and test preparation. States and school districts start conducting more tests to use as test preparation and predictors to determine a student's scoring ability on the mandated testing (Duncan, 2013).

16. Review of the Current Literature According to former Education S ecretary Arne Duncan, the need for data-driven instruction and differentiation, coupled with increased teacher scrutiny because of the accountability measure being required of school districts receiving federal funding, created a huge need in e-learning resources to truly differentiate instruction(Duncan, 2013). Micheal Petrilli (2011) wrote: The greatest challenge facing America’s schools today isn’t the budget crisis, or standardized testing, or “teacher quality.” It’s the enormous variation in the academic level of students coming into any given classroom.

17. Review of the Current Literature Mathias Decuypere, Emiliano Grimaldi & Paolo Landri (2021) Several factors over the last decade, specifically COVID-19, have seen the development of platforms tailored to primary and secondary schools specifically constructed for the field of higher education; from digital environments designed to manage pupils’ learning to environments focused on the monitoring of their behavior; and from digital spaces bundling a variety of functionalities to interfaces with a more singular function: no matter the focus, there seems to exist a corresponding digital platform used within the educational field.

18. IXL IXL is based on the drill and practice function of instructional technology. Drill and practice is rooted in the theory of behaviorism, focusing on repetition of stimulus-response practice and the concept of reinforcement (Lim et al., 2012).

19. i-Ready According to Hughes (2018) i-Ready is an adaptive diagnostic assessment and personalized instructional tutorial tool that places each student in a personalized learning path through online lessons. i-Ready’s tutorial program is rooted in explicit instruction; clear and concise tracking of growth to determine which students need additional help or interventions which lends itself to the cognitivist theory, learning focusing on the internal process and connections that take place during learning

20. Prodigy Prodigy is a practice website and app for grades K–8 that addresses standards-based skills and testing- based concepts in Math for both home and school. Prodigy is based on the gamification-theory, which is the use of game elements in non-game contexts, and is increasingly being implemented in both student and organizational learning initiatives (Kampen, 2019).

21. Review of this Study In Light of the Reviewed Literature (the gap) The central argument to all three learning platform programs are that they are supported by research-based educational theories that lead to increased academic progress. Each platform has the ability to differentiate instruction through personalized learning plans based on diagnostic examinations that students take

22. The Gap cont’d Where all the programs overlap is a combination between two learning theories of cognitive and behavioral learning. According to Kendra Cherry (2019) The combination is called Social Learning theory or Social Cognitive Learning theory. the theory considers how environmental, behaviorism and cognitive factors interact to influence human learning and behavior.

23. The Gap cont’d Alberts Bandura's, the author of Social Cognitive theory While the behavioral theory of learning suggested that all learning was the result of associations formed by conditioning, reinforcement, and punishment, Bandura's social learning theory proposed that learning can also occur simply by observing the actions of others. His theory added a social element, arguing that people can learn new information and behaviors by watching other people. Known as observational learning,

24. The Gap cont’d Bandura explanation of observational and modeling processes Four steps: attention, retention, reproduction and motivation,

25. The Gap cont’d At the core of all three is platform retention, the ability to store information. IXL and i-Ready overlap at step reproduction, or the ability to perform the behavior you observed. Each of these programs have explicit instruction, tutoring and practice built into their programs, so that after a while the students are able to reproduce what they have learned.

26. The Gap cont’d IXL and Prodigy overlap in motivation or the stimulus that reinforces behaviors. Both programs have built in extrinsic motivational interfaces that are designed to help the students focus and want to get better at math. With Prodigy’s unique gamification style, it has the greatest potential to keep student’s attention or the ability to focus on the content, which Bandura states it has to be interesting.

27. Methodology We intend to analyze records and artifacts including district data from the previous NAEP administration among districts that implemented one of the three math programs in the year following the next NAEP assessment in 2021. The 2025 NAEP administration’s data will be compared to the district data for 2021, the year prior to the treatment.

28. Type of Research This research will be quantitative, measuring data at the ordinal level.

29. Research Design Causal-comparative Groups are chosen through pre-existing data from the NAEP assessment. Treatment groups are exposed to learning platforms and compared to groups that are not using the e-learning platforms (Key Elements of a Research Proposal Quantitative Design, n.d.).

30. Data Gathering Procedures Identify districts that are frequent participants in the NAEP assessment, having taken the last three tests. Select districts that have adopted either of the independent variables (i-Ready, IXL, or Prodigy) district-wide for the first time in 2022, the year after the last NAEP assessment and establish a data-sharing agreement with those districts.

31. Data Gathering Procedures Classify each district as performing in the lowest quartile, 2nd quartile, 3rd quartile, or upper quartile. Randomly select twelve treatment districts - one adoption district for each of the three programs and each of the four performance quartiles. A homogeneous subgroup comparison will be completed for each quartile to strengthen the research sample. (Salkind, 2010)

32. Data Gathering Procedures Randomly select four control districts - one from each performance quartile that did not adopt either of the treatment programs. After the next NAEP assessment in 2025, district personnel will provide NAEP data to the research team for an Analysis of Covariance (ANCOVA). The second NAEP district average will serve as the dependent variable.

33. Determine the 2022 average NAEP percentile among fourth and eighth graders for the students in each district to serve as the baseline. Determine the average NAEP percentile for the students in each district after the 2025 administration. Data analysis Procedures

34. Data analysis Procedures Run a regression between: the independent variables - the programs implemented, or the control with no program used; and the dependent variables - the average NAEP percentile for the district. Complete an Analysis of Covariance (ANCOVA) to examine the differences in the mean values of the dependent variables ( Analysis of covariance (ANCOVA) 2020).

35. Data Analysis Procedures Use the ANCOVA analysis within each grade level and each of the four quartiles to determine the program that yields districts with the greatest positive change in each category.

36. Final Summary of the Proposal With the research conducted and data collected, we expect that all the three e-learning platforms will have a greater effect on student data than student groups that are not receiving e-learning supplemental resources. We further anticipate that Prodigy will have a greater impact than the other treatment programs. The goal is to highlight each e-learning platform for its specific capacities to determine the most effective program for supporting achievement and closing learning gaps.

37. References Abuhassna, H., Al-Rahmi, W.M., Yahya, N. et al. Development of a new model on utilizing online learning platforms to improve students’ academic achievements and satisfaction. Int J Educ Technol High Educ 17, 38 (2020). https://doi.org/10.1186/s41239-020-00216-z Analysis of covariance (ANCOVA). Statistics Solutions. (2020, June 30). https://www.statisticssolutions.com/analysis-of-covariance-ancova/#:~:text=Analysis%20of%20covariance%20(ANCOVA)%20is,of%20the%20uncontrolled%20independent%20variables . Berwick, C. (2019, October 25). What Does the Research Say About Testing? Edutopia. https://www.edutopia.org/article/what-does-research-say-about-testing . Cherry, K. (2019, December 1). How Does Observational Learning Actually Work? Verywell Mind. https://www.verywellmind.com/social-learning-theory-2795074 Cozad, L. E., & Riccomini, Paul. J. (2016, November 30). Effects of digital-based math Fluency interventions on learners with Math Difficulties: A review of the literature. https://eric.ed.gov/?id=EJ1127743.

38. References Cont’d Curriculum Associates. (2015, January). i-Ready® Research Base for Instruction . https://www.curriculumassociates.com/-/media/mainsite/files/i-ready/Iready-research-base-for-instruction-2015.pdf. https://www.curriculumassociates.com/-/media/mainsite/files/i-ready/iready-research-base-for-instruction-2015.pdf. DeSilver, D. (2020, August 21). U.S. academic achievement lags that of many other countries. Pew Research Center. https://www.pewresearch.org/fact-tank/2017/02/15/u-s-students-internationally-math-science/. Duncan, A. (2013, August 1). Arne Duncan: How Technology Will Revolutionize Testing and Learning . Scientific American. https://www.scientificamerican.com/article/arne- duncan- how-technology-will-revolutionize-testing-learning/. Hollands, F. & Pan, Y. (2018). Evaluating Digital Math Tools in the Field. Middle Grades Review: Vol. 4 : Iss. 1 , Article 8. Evaluating Digital Math Tools in the Field Hughes, M.D.R.J. E. (2018). Integrating Educational Technology into Teaching (Subscription). [VitalSource Bookshelf]. Retrieved fro m https://bookshelf.vitalsource.com/#/books/9780134746296/

39. Reference Cont’d i-Ready. (2021). Research Base for i-Ready Personalized Instruction for Mathematics Executive Summary . i-Ready Research Foundations Mathematics Executive Summary 2020. https://www.curriculumassociates.com/-/media/mainsite/files /i-ready/iready-research-foundations-mathematics-executive_summary-2020.pdf Klein, A. (2015, April 10). No Child Left Behind Overview: Definitions, Requirements, Criticisms, and More. Retrieved April 21, 2019, from https://www.edweek.org/ew/section/multimedia/no-child-left-behind-overview-definition-summary.html Key Elements of a Research Proposal Quantitative Design. (n.d.). https://www.wssu.edu/about/offices-and-departments/office-of-sponsored- programs/pre-award/_Files/documents/develop-quantitative.pdf Landers, R. (2015, July 05). Psychological theory, learning gamification. Retrieved April 25, 2021, from https://neoacademic.com/2015/01/15/psychological-theory-gamification-learning/ Lim, C. S., Tang, K. N., & Kor, L. K. (2012). Drill and Practice in Learning (and Beyond). Encyclopedia of the Sciences of Learning , 1040–1042. https://doi.org/10.1007/978-1-4419-1428-6_706

40. Reference Cont’d Lisciandrello, J. (2021, April 4). The Top 4 Math Sites for Adaptive Learning . Room to Discover. https://www.roomtodiscover.com/adaptive-learning/. Mcleod, S. (2017, February 5). Behaviorist Approach . Behaviorism | Simply Psychology. https://www.simplypsychology.org/behaviorism.html#:~:text=Behaviorism%2C%20also%20known%20as%20behavioral,a%20response%20to%20environmental%20stimuli. Market Insider. (2020, January 7). 2019 Global Edtech Investments Reach a Staggering $18.66 Billion . Business Insider. Retrieved from https://markets.businessinsider.com/news/stocks/2019-global-edtech-investments-reach-a-staggering-18-66-billion-1028800669 Mathias Decuypere , Emiliano Grimaldi & Paolo Landri (2021) Introduction: Critical studies of digital education platforms , Critical Studies in Education, 62:1, 1-16, DOI: 10.1080/17508487.2020.1866050 To link to this article: https://doi.org/10.1080/1 McCormick, C. (2018, December 27). Three ways technology can help TEACHERS differentiate student learning - Edsurge News. Retrieved April 25, 2021, from https://www.edsurge.com/news/2018-10-22-three-ways-differentiation-can-move- the-needle-on-your-students-learning

41. References Cont’d National Center for Education Statistics. (2013, June). Trends in Academic Progress . The Nation's Report Card: Trends in Academic Progress 2012. https://nces.ed.gov/nationsreportcard/subject/publications/main2012/pdf/2013456.pdf Petrilli, M. J. (2011). All Together Now? Education Next. https://www.educationnext.org/all-together-now/ . Petrilli, M. (2018, October 4). NAEP 2017: America's "lost decade" of educational progress. Retrieved April 25, 2021, from https://fordhaminstitute.org/national/commentary/naep-2017-americas-lost- decade-educational-progress Rhalmi, M. (2020, November 18). Learning Theories: Cognitivism . My English Pages. https://www.myenglishpages.com/blog/description-of-cognitivism/#:~:text=Cognitivism%20is%20a%20learning%20theory,that%20take%20place%20during%20learning. Roblyer M. D., & Hughes J. E. (2018). Integrating Educational Technology into Teaching (Subscription). [VitalSource Bookshelf]. Retrieved from https://bookshelf.vitalsource.com/#/books/9780134746296/

42. References Cont’d Salkind, N (2010). Encyclopedia of Research Design. Volume 1. Retrieved from http://web.utk.edu/~ewbrewer/pdf/encylopedia/Encyclopedia%20of%20Research%20Design_Volume%201.pdf Scalise, Kathleen. (2007). Differentiated e-learning: five approaches through instructional technology. IJLT. 3. 169-182.10.1504/IJLT.2007.014843. Stern, J. (2015, March 30). Enhancing Learning Through Differentiated Technology . Edutopia. https://www.edutopia.org/blog/enhanced-learning-through-differentiated-technology-juliestern#:~:text=Three%20tech%20tools%20%2D%2D%20SAS,and%20offering% 20appropriate%20formative%20assessment.&text=Each%20day%2C%20teachers%20face%20the,growth%2C%20and%20master%20the%20curriculum.l. Stiggins, R., & Chappuis, J. (2010, June 24). Using Student-Involved Classroom Assessment to Close Achievement Gaps . Theory Into Practice. https://www.tandfonline.com/doi/pdf/10.1207/s15430421tip4401_3?needAccess=true . Teaching and Education. (2021, March 23). What Is The Behavioral Learning Theory? Western Governors University. https://www.wgu.edu/blog/what-behavioral-learning-theory2005.html#close. Tino, S. (2020, April 17). How Prodigy's Logic Model Improves Student Outcomes . Prodigy Education. https://www.prodigygame.com/main-en/blog/prodigy-logic-model/ . Tino, S. (2020, April 9). Is Prodigy Math Game Adaptive? Our Algorithm, Explained . Prodigy Education. https://www.prodigygame.com/main-en/blog/is-prodigy-math-adaptive/ .