DBR_Presentation_SPKarunanayaka_08.05.2024.pdf

Presentation made at a session conducted for the "ODL Practitioner Research Training and Mentorship Programme" of Commonwealth of Learning, on 08.05.2024.

1. A Design-based Approach
to Research
ShironicaP. Karunanayaka
Senior Professor in Educational Technology
Faculty of Education
The Open University of Sri Lanka

2. Session Plan
08 May 2024 Shironica P. Karunanayaka, OUSL 2
•DBR: What, Why, and How ?
•The Role of Researcher in DBR
•DBR in Practice
•Challenges in DBR

3. Design-based Research (DBR)
08 May 2024 Shironica P. Karunanayaka, OUSL 3
What?
•“Design” is the key element of DBR.
•DBR is an “Interventionist” approach.
•It involves studying flexible iterations of designed
interventions in a naturalistic context.
•Yet, it extends beyond mere designing of interventions
and testing them.
•The goal of DBR is solving current real-world problems by
designing and implementing interventions, while
extending theories and refining design principles, rather
than testing theory.

4. Types of
“Design”
Research
08 May 2024 Shironica P. Karunanayaka, OUSLS 4
Source: https://craigdennishoward.wordpress.com/2013/08/08/the-international-journal-of-designs-for-learning-and-the-differences-among-design-research-design-based-research-and-design-science/

5. Design Research is not defined by its methods but by the goals of
those who pursue it. Design research is constituted within
communities of practice that have certain characteristics of
innovativeness, responsiveness to evidence, connectivity to basic
science, and dedication to continual improvement.
(Bereiter, 2002, p.31 cited in McKenney & Reeves, 2013, p.3)
08 May 2024 Shironica P. Karunanayaka, OUSL 5

6. Terminology
08 May 2024 Shironica P. Karunanayaka, OUSL 6
➢The term Design Experiment was introduced by Ann Brown (1992) and Allan Collins
(1992) as an innovative approach to educational research, which took place in the
“real world”, in contrast to the traditionally laboratory experimental approach.
➢By 2004, the term Design-Based Research (DBR) was introduced to avoid confusion
with the phrase “design experimentation”, which implies controlled experimentation
that does not capture the breadth of the approach (DBRC, 2004).
➢The term Educational Design Research (EDR) has been adopted by van den Akkar et
al., (2006); Plomp & Nieveen (2009); and McKenney & Reeves (2013), by adding the
term “educational” to avoid confusion with design research used in other fields.
➢Design-based Implementation Research (DBIR) is another term used in initiatives
that involve scaling DBR to support change in larger systems (Penuel et al., 2011).

7. Design-based Research (DBR)
08 May 2024 Shironica P. Karunanayaka, OUSL 7
DBR is a systematic, but flexible methodology aimed to improve
educational practices through an iterative process of analysis,
design, development, and implementation based on
collaboration among researchers and practitioners in real-world
settings and leading to contextually-sensitive design principles
and theories (Wang & Hannafin, 2005, p.6).

8. Design-based Research (DBR) - Stages
8
Analysis
of Practical
Problems by
Researchers and
Practitioners in
Collaboration
Development of
Solutions Informed
by Existing Design
Principles and
Technological
Innovations
Iterative Cycles
of Testing and
Refinement of
Solutions in
Practice
Reflection to
Produce ‘Design
Principles’ and
Enhance Solutions
Implementation
(Adapted from Reeves, 2006, p.59)
Refinement of Problems, Solutions, Methods, and Design Principles
Shironica P. Karunanayaka, OUSL08 May 2024

9. Key Principles of DBR
•Addressing complex problems in real contexts in collaboration
with practitioners;
•Integrating known and hypothetical design principles with
technological advances to render plausible solutions to these
complex problems;
•Conducting rigorous and reflective inquiry to test and refine
innovative learning environments as well as to define new
design principles.
(Reeves, 2006, p. 58)
08 May 2024 Shironica P. Karunanayaka, OUSL 9

10. Characteristics of DBR
•Pragmatic: The goals of DBR are solving current real-world problems by designing and
enacting interventions, as well as extending theories and refining design principles. It
generates usable knowledge and usable solutions to problems in practice.
•Grounded: DBR is grounded in both theory and the real-world context. Theory is
both the foundation and the outcome of DBR.
•Interactive, Iterative and Flexible: DBR requires interactive collaboration among
researchers and practitioners. The ongoing recursive nature of the design process
allows greater flexibility.
•Integrative: In DBR, researchers need to integrate a variety of research methods from
both qualitative and quantitative research paradigms.
•Contextual: Results of DBR are connected with both the design process through
which the results are generated and the setting where the research is conducted.
(Barab & Squire, 2004; DBRC, 2003; McKenney & Reeves, 2013; Van den Akkar, 1999; Wang & Hannafin, 2005)
08 May 2024 Shironica P. Karunanayaka, OUSL 10

11. Predictive vs. Design-based Research
08 May 2024 11Shironica P. Karunanayaka, OUSL
(Reeves, 2006)
Comparing DBR with qualitative and experimental traditions.

12. DBR vs. Action Research (AR)
08 May 2024 12Shironica P. Karunanayaka, OUSL
•AR is a form of self-reflective enquiry undertaken by
practitioners in social situations, to improve their own practices.

13. DBR vs. AR
•Similarities:
•Both AR and DBR identify real world problems accompanied by subsequent
actions to improve the status quo. Both are interventionist research.
•Practitioners are highly involved in the research process in both AR and DBR.
•Differences:
The major goal:
•AR aims to simultaneously investigate and solve a selected issue.
•The goal of DBR is not only to address immediate local issues but also to
contribute to theory by developing and refining design principles. i.e.
Generating theory to solve authentic problems.
The role of the researcher in the research process:
•In DBR, researchers take the initiative as both researchers and designers.
•In AR, the practitioners initiate the research; The practitioner is both researcher
and teacher.
08 May 2024 Shironica P. Karunanayaka, OUSL 13

14. Nine Principles of DBR
•Support design with research from the outset
•Set practical goals for theory development and develop an initial plan
•Conduct research in representative real-world settings
•Collaborate closely with participants
•Implement research methods systematically and purposefully
•Analyze data immediately, continuously and retrospectively
•Refine designs continually
•Document contextual influences with design principles
•Validate the generalizability of the design (Wang & Hannafin, 2005)
08 May 2024 14Shironica P. Karunanayaka, OUSL

15. The Quality of a DBR Study is defined by:
✓Being situated in a real educational context
✓Focusing on the design and testing of a significant intervention
✓Use of mixed methods
✓Involving multiple iterations
✓Involving a collaborative partnership between research(s) and practitioner(s)
✓Evolution of design-principles
✓Comparison to Action Research
✓Practical impact on practice (Anderson & Shattuck, 2012)
08 May 2024 Shironica P. Karunanayaka, OUSL 15

16. Design-based Research (DBR)
08 May 2024 Shironica P. Karunanayaka, OUSL 16
Why?
DBR was developed to address issues central to the study of
learning:
•The need to address theoretical questions about the nature of
learning in context.
•The need for approaches to the study of learning phenomena in
the real world rather than the laboratory or controlled settings.
•The need to go beyond narrow measures of learning.
•The need to derive research findings from formative evaluation.
(Collins et al., 2004)

17. DBR: Significance
➢The ultimate goal of DBR is to build a stronger connection between
educational research and real-world problems.
➢Proponents of DBR believe that conducting research in context, rather
than in a controlled laboratory setting, and iteratively designing
interventions yield authentic and useful knowledge.
➢Educational research that is detached from practice may not account
for the influence of contexts, the emergent and complex nature of
outcomes, and the incompleteness of knowledge about which factors
are relevant for prediction.
08 May 2024 17Shironica P. Karunanayaka, OUSL

18. DBR: Significance
➢Being grounded in real-life context and conducted in
collaboration with the practitioners, the DBR approach have
more potential in enacting desired changes of authentic
educational practices.
➢The outcomes of design-based research are a set of design
principles or guidelines derived empirically and richly described,
which can be implemented by others interested in studying
similar settings and concerns.
08 May 2024 18Shironica P. Karunanayaka, OUSL

19. Design-based Research (DBR)
08 May 2024 Shironica P. Karunanayaka, OUSL 19
How?
The DBR process comprises four phases:
✓analysis of existing levels of practice;
✓designing, developing, and implementing solutions as
appropriate;
✓testing and refining solutions in practice; and
✓reflection on action to produce design principles and
enhance solution implementation
(Reeves, 2006)

20. DBR - Steps
•Begin with a meaningful real-life problem
•Collaborate with practitioners
•Integrate robust theory about learning and teaching
•Conduct literature review, needs analysis…etc. to generate research
questions
•Design an educational intervention
•Develop, implement, and revise the design intervention
•Evaluate the impact of the intervention
•Iterate the process
•Report DBR
(Collins et al., 2004; Reeves et al., 2005)
08 May 2024 Shironica P. Karunanayaka, OUSL 20

21. 08 May 2024 Shironica P. Karunanayaka, OUSL 21
Design Research Process (Plomp, 2013, p.17)
(Source: http://international.slo.nl/publications/edr/)

22. DBR – Prototyping (McKenney, 2001)
08 May 2024 Shironica P. Karunanayaka, OUSL 22
Source: Plomp, 2013, p.17
http://international.slo.nl/
publications/edr/)

23. DBR Process (Sayer, 2023)
08 May 2024 Shironica P. Karunanayaka, OUSL 23
(Source: https://edtechbooks.org/studentguide/design-based_research)

24. A Process Model for DBR (Hoadley & Campos, 2022)
08 May 2024 Shironica P. Karunanayaka, OUSL 24
Source: https://www.tandfonline.com/doi/full/10.1080/00461520.2022.2079128

25. 08 May 2024 Shironica P. Karunanayaka, OUSL 25
A Generic Model for Conducting Educational
Design Research (EDR) (McKenney & Reeves, 2012)
(McKenney,& Reeves, 2012)

26. •The Intervention: Could be a learning activity, a type of assessment, the
introduction of an administrative activity, or a technological intervention.
•Design Artifacts: Products developed through design
Eg: Digital learning platforms; Educational software applications;
Authentic assessments, Learning experiences; Instructional strategies, A
flipped classroom model
•Design Principles: Guidelines derived empirically and richly described,
which can be implemented by others interested in studying similar
settings and concerns.
Eg: Theory; Framework; Model; Set of Guidelines;

27. Researcher’s Role in DBR
•The duality of roles for DBR researchers as “designer” and “researcher”
contributes to a greater sense of responsibility and accountability.
•In DBR, researchers view the participants (e.g., students, teachers,
practitioners) as “co-participants” and “co-investigators”.
•DBR begins with the negotiation of research goals between practitioners
and researchers. The practitioner is seen as a valuable partner in
establishing research questions and identifying problems that merit
investigation.
(Barab & Squire, 2004; Collins, 1990)
08 May 2024 Shironica P. Karunanayaka, OUSL 27

28. 08 May 2024 28Shironica P. Karunanayaka, OUSL
•The selection and creation of the intervention is a collaborative task of both
researcher(s) and practitioner(s).
•Design-based researchers see themselves as “agents of change” and as
accountable for the work they do.
•The design-based researcher is humble in approaching research by
recognizing the complexity of interactions that occur in real-world
environments and the contextual limitations of proposed designs.
•“[…] DBR researchers are not simply observing interactions but are actually
“causing” the very same interactions they are making the claims about […]”
(Barab and Squire, 2004, p. 9).
Researcher’s Role in DBR (contd.)

29. DBR Studies (2000-2010) (Anderson & Shattuck, 2012)
08 May 2024 Shironica P. Karunanayaka, OUSL 29
0
50
100
150
200
250
300
350
400
2000 01 02 03 04 05 06 07 08 09 10
Number of DB Scholarly
Articles Published
Computers
7%
Cross-Curricula
13%
English Language
5%
Literacy
4%
Mathematics
9%
Science
51%
Teacher Training
9%
Vocational
2%
Subject/program in which DBR Studies were
undertaken (n=45)
DBR in Practice

30. DBR in
Practice
(Examples)
08 May 2024 Shironica P. Karunanayaka, OUSL 30
(Source: McKenney,& Reeves, 2013)

31. DBR in Practice (Examples)
08 May 2024 Shironica P. Karunanayaka, OUSL 31
•Karunanayaka, S. P., & Naidu, S. (2017). A design-based approach to support and
nurture open educational practices. AAOU Journal, 12(1), 1-20.
https://doi.org/10.1108/AAOUJ-01-2017-0010
•https://www.slideshare.net/shka/karunanayakanaiduaaou2016-67793666
•Karunanayaka, S. P., & Naidu, S. (2018) Designing Capacity Building of Educators in Open
Educational Resources Integration Leads to Transformational Change. Distance
Education, 39(1), 87-109. http://dx.doi.org/10.1080/01587919.2017.1413933
•Sandanayake, T.C., Karunanayaka, S.P., & Madurapperuma, A.P. (2021). A framework to
design open educational resources-integrated online courses for undergraduate
learning: A design-based research approach. Education and Information Technologies,
26, 3135–3154 (2021), https://doi.org/10.1007/s10639-020-10393-z
•Peramunugamage, A., Rathnayake, U., & Karunanayaka, S.P. (2020). Design of a
framework to foster collaborative learning through a Moodle mobile plugin. [Paper
presentation]. PhD Symposium – EDEN, Lisbon, 10 October 2020. https://www.eden-
online.org/2020_lisbon/phd/

32. DBR in Practice (Doctoral Thesis)
08 May 2024 Shironica P. Karunanayaka, OUSL 32
•Nieveen, N. (1997). Computer support for curriculum developers. Doctoral thesis. Enschede (The Netherlands):
University of Twente.
•McKenney, S. (2001). Computer-based support for science education materials developers in Africa: exploring
potentials. Doctoral thesis. Enschede (The Netherlands): University of Twente.
•Nihuka, K. (2011). Collaborative course design to support implementation of e-learning by instructors. Doctoral
thesis. Enschede (The Netherlands): University of Twente. http://doc.utwente.nl/78096/1/thesis_K_Nihuka.pdf
•Palalas, A. (2012). Design guidelines for a Mobile-Enabled Language Learning system supporting the
development of ESP listening skills. Doctoral Dissertation. Athabasca University.
https://dt.athabascau.ca/jspui/handle/10791/17
•Ostashewski, N. (2013) Networked Teacher Professional Development: Assessing K-12 Teacher Professional
Development within a social networking framework. Doctoral dissertation. Athabasca University.
https://dt.athabascau.ca/jspui/handle/10791/26
•Shattuck, J. (2013) Training higher education adjunct faculty to teach online: A design-based research study.
Doctoral dissertation. Athabasca University. https://dt.athabascau.ca/jspui/handle/10791/27
•Porcaro, D. (2011). Omani undergraduate student reactions to collaborative knowledge building: A design
research study. Doctoral dissertation. University of Georgia.
https://getd.libs.uga.edu/pdfs/porcaro_david_s_201105_phd.pdf

33. DBR: Overcoming Challenges
•Information richness and efficiency: Seeking a productive balance
•Optimizing processes: Stacking smaller studies together
•Measuring Impact: Powerful examples needed
•Generalizability: Toward uptake and use of new knowledge
(McKenney & Reeves, 2013)
08 May 2024 Shironica P. Karunanayaka, OUSL 33
•Credibility and trustworthiness of assertions: Sound methodology;
Triangulation with multiple sources of data; Thick descriptions
•Bounding the temporal scope of a DBR project
•Complexities of real-world situations
•Dual roles of the researcher as designer and researcher
(Barab & Squire, 2004; DBRC, 2003)

34. References
Anderson, T., & Shattuck, J. (2012). Design-Based Research: A Decade of Progress in Education Research? Educational Researcher,
41(Jan/Feb.), 16-25
Amiel, T., & Reeves, T. C. (2008). Design-Based Research and Educational Technology: Rethinking Technology and the Research
Agenda. Educational Technology & Society, 11 (4), 29–40.
Barab, S. and Squire, K. (2004). Design-based research: putting a stake in the ground. The Journal of the Learning Sciences, 13(3), 1-
14.
Hoadley, C., & Campos, F. C. (2022). Design-based research: What it is and why it matters to studying online learning. Educational
Psychologist, 57(3), 207–220. https://doi.org/10.1080/00461520.2022.2079128
Kali, Y. & Hoadley, C. (2020). Design-Based Research Methods in CSCL: Calibrating our Epistemologies and Ontologies. In U. Cress, C.
Rosé, A. Wise, and J. Oshima (Eds.), International Handbook of computer-supported collaborative learning. Springer.
https://doi.org/10.13140/RG.2.2.31503.20642
McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. Routledge.
Reeves, T.C. (2006), Design research from a technology perspective. In van den Akker, J., Gravemeijer, K., McKenney, S. & Nieveen, N.
(Eds), Educational Design Research, Routledge. pp. 86-109
Sayre, E. C. (2023). “Design-Based Research.” In Research: A Practical Handbook. https://handbook.zaposa.com/articles/design-
based-research.
The Design-Based Research (DBR) Collective. (2003). Design-based research: an emerging paradigm for educational inquiry”,
Educational Researcher, 32 (1) 5-8. www.designbasedresearch.org/reppubs/DBRC2003.pdf
Wang, F. & Hannafin, M.J. (2005). Design-based research and technology-enhanced learning Environments. Educational Technology
Research and Development, 53 (4), 5-23.
08 May 2024 Shironica P. Karunanayaka, OUSL 34

35. 08 May 2024 Shironica P. Karunanayaka, OUSL 35