Accreditation-Outcome-Based-Approach.pptx

outcome based education

1. Accreditation & Outcome Based Approach Prof Megat Johari Megat Mohd Noor UTM QRIM@KL & MJIIT Lahore/Faisalabad/ Jamshoro , Pakistan September 2016

2. Introduction

3. We are joining the Mutual Recognition Train (MRT) ! 3

4. WASHINGTON ACCORD SYDNEY ACCORD DUBLIN ACCORD 4 YEARS 3 YEARS 2 YEARS IPEA International Professional Engineers Agreement (ENGINEERS MOBILITY FORUM) APEC ENGINEER FEANI / EUR-ACE / ENAEE (EUROPE) 3 + 2 YEARS NABEEA (ASIA) EDUCATION ACCORDS PRACTICE AGREEMENTS IETA International Engineering Technologist Agreement ( ENGINEERING TECHNOLOGISTS MOBILITY FORUM) INTERNATIONAL ENGINEERING ALLIANCE (IEA) (INTERNATIONAL ENGINEERING MEETING, IEM ) 4 AIET Agreement of International Engineering Technician

5. Development of International Engineering Alliance IEA Established in 2007 5

6. WASHINGTON ACCORD FULL SIGNATORY Australia - Engineers Australia ( 1989 ) New Zealand - Institution of Professional Engineers NZ ( 1989 ) Canada - Engineers Canada ( 1989 ) United States - Accreditation Board for Engineering and Technology ( 1989 ) United Kingdom - Engineering Council UK ( 1989 ) Ireland - Engineers Ireland ( 1989 ) Hong Kong China - The Hong Kong Institution of Engineers ( 1995 ) South Africa - Engineering Council of South Africa ( 1999 ) Japan - Japan Accreditation Board for Engineering Education ( 2005 ) Singapore - Institution of Engineers Singapore ( 2006 ) Chinese Taipei - Institute of Engineering Education Taiwan ( 2007 ) Korea - Accreditation Board for Engineering Education of Korea ( 2007 ) Malaysia - Board of Engineers Malaysia (2009) Turkey - MUDEK (2011) Russia - Association for Engineering Education of Russia (2012) India - National Board of Accreditation (2014) Sri Lanka - Institution of Engineers Sri Lanka (2014) China - CAST (2016) Provisional Status Bangladesh Pakistan Phillippines Peru Costa Rica Mexico Potential Applicants Thailand Indonesia 6

7. Accreditation 7

8. Importance of Accreditation to Institutions of Higher Learning Recognises institutional missions and goals Involves faculty/staff in evaluation and planning Assists institutions in determining the acceptability of transfer credits Promotes “best practices” in education Increases visibility and reputation of the institution Aids engineering schools to identify required operational resources to institution management 8

9. Pakistan Washington Accord Route (2011 – 2016) Nominator ( EME, GIKI ) Mentor ( Islamabad, Topi , Risalpur , Faisalabad , Peshawar, Karachi, Lahore ) 1a Reviewer ( Universities? ) 1b Reviewer ( ADM ) Prof Abang (MAL) Prof Megat (MAL) Prof Lock (SIN) Kim (Korea) Collins (UK) Basil ( NZ) Nominator 9 Mentor Prof Megat (MAL) Ir Azlan (MAL) Prof Lock (SIN) Reviewer?

10. DECIDE ENGINEERING ACCREDITATION BOARD (EAB) Representatives of Professional Societies Representatives of Universities Representatives of Industries Constituents in Accreditation PAKISTAN ENGINEERING COUNCIL (PEC) ACCREDITATION DEPARTMENT (AD) EAB EVALUATION PANEL ENDORSE FACILITATE RECOMMEND Representatives of Government Major Stakeholders 10

11. EAB Manual From input based to outcome based 20.. 2014 11

12. Continual Quality I mprovement 12

13. Programme Evaluators (PEVs) Chair (Criteria of appointment) Two members (Criteria of appointment) one member with extensive academic experience and one member with extensive industrial experience - knowledgeable - trained - independent 13

14. Challenges Paradigm Shift – Outcome & Quality Maintain Fundamentals while Encourage Inclusion of Latest Technology Advancement in the Curriculum Allow Academic Innovation and Creativity Avoid Side-tracked Variety of Modes of Delivery

15. Complex Problems Broadly Defined Problems Well defined Problems S olved using limited theoretical knowledge, but normally requires extensive practical knowledge K nowledge of principles and applied procedures or methodologies I n -depth knowledge that allows a fundamentals-based first principles analytical approach Depth of Knowledge Required 15

16. Engineering & Technology Domain Engineers Technologists Research & Design Supervision & Maintenance Strong in Mathematics, Engineering Sciences, Professional courses (Theoretical) Appropriate Mathematics, Engineering Sciences, Professional courses (Practical) Education Work Engineering Breadth & Depth of Curricula Technology Breadth & Depth of Curricula 16

17. Programme EO / O Development/ Review Internal Stakeholders Teachers Students University External Stakeholders Potential Employers / Industry Alumni Regulatory Body Course O / Content Development / Review 1, 2, 3 …… Course Implementation 1, 2, 3 …… Course Assessment 1, 2, 3 …… Teacher – Knowledge, Skills, Affective Students – Teaching Teacher – Descriptive Self Assessment on Cohort’s Achievement Programme Evaluation Summative - direct Exit Survey - indirect Industry Survey - indirect Alumni Survey - indirect External – direct Accreditation - direct Educational Process & Stakeholders Pull factor Internal Stakeholders Teachers Technicians Students Internal Stakeholders Teachers Students External Stakeholders Potential Employers / Industry Alumni Regulatory Body External Assessor Summative Formative / Summative Internal Stakeholders Teachers Specification 17

18. (8) Leadership, governance and administration (6) Educational resources (1) Vision,mission and learning outcomes (5) Academic Staff (2a) Curriculum Design (2b) Curriculum Delivery INPUT (STUDENTS) (4a) Selection of Students (4b) Supporting Services (7) Programme Monitoring and Review (3) Student Assessments (9) Total Continual Quality Improvements OUTPUT (GRADUATES) STAKEHOLDER NEEDS AND INSTITUTIONAL MISSION FEEDBACKS FROM STAKEHOLDERS STAKEHOLDER ’ S SATISFACTION Academic IQA Practices in Perspective 18

19. OBE Programme Objective (after 3-5 Years) Programme Outcome (at Exit) Knowledge Skill Attitude Assessment Course/Unit/Learning Outcome (Abilities & Intentional) Student Centered Stakeholders Performance Indicator Measurable Evaluation Appropriate Delivery (Case, PBL …) Directed & Coherent Curriculum Graduate Relevant to Industry Continual Quality Improvement (CQI) Accountable 19

20. Characteristics of OBE curricula Have programme objectives, programme outcomes, course outcomes and performance indicators . Stated objectives and outcomes can be assessed and evaluated . Centered around the needs of the students and the stakeholders . 20

21. Characteristics of OBE curricula Learning outcomes are intentional and assessed using suitable performance indicators. Programme objectives address the graduates attainment in their career within 3-5 years after their graduation. Programme outcomes ( abilities attained by students before they graduate) are formulated based on the programme objectives – TOP DOWN . 21

22. Characteristics of OBE curricula Programme outcomes address Knowledge, Skills and Attitudes to be attained by students. Course outcomes must satisfy the stated programme outcomes. There is no need for ANY (individual) course to address all programme outcomes. Teaching/ Learning method may have to be integrated to include different delivery methods to complement the traditional Lecturing method. 22

23. OBE in a nut shell What do you want the students to have or able to do? How can you best help students achieve it? How will you know what they have achieved it? How do you close the loop Knowledge, Skill, Affective PDCA Student Centred Delivery Assessment 23

24. Plan, Do, Check & Act (PDCA) 24

25. Strategy of OBE Top down curricula design Appropriate Teaching & Learning Methods Appropriate Assessment & Evaluation Methods 25

26. Different Levels of Outcomes Programme Educational Objectives Programme Outcomes Course/subject Outcomes Weekly/Topic Outcomes Upon graduation Upon subject completion Upon weekly/topic completion Few years after Graduation – 3 to 5 years 26

27. Accreditation

28. Preparation for Accreditation Comprehend the EAB Manual Prepare the SAR Address previous accreditation report Arrange the evidence Complete the Checklist Assign key persons according to accreditation schedule 28

29. SAR & Evidence 29

30. Azlan or a man sitting down in a garden and a lady passing by? 30

31. Issues from Approval Report First Year Visit Report External Examiner’s Report Stakeholders’ Recommendations Recent Accreditation Visit EVIDENCE BASED Documents & Records Interviews Observations CQI

32. Self Assessment Report Prepare a checklist of questions based on the following criteria in preparing a SAR: Programme Objectives Programme Outcomes Curriculum and Learning Process Students Faculty Support staff Industry stakeholders Alumni Facilities and Infrastructure Institutional and Financial Support Continual Quality Improvement Industry Linkage Exercise

33. Industrial Linkage Using the Clause 9 of the PEC Manual, evaluate the relevant part of the submitted SAR. Exercise

34. 3.2.9 Criterion 9: Industrial Linkages Students are expected to undertake assignments from industry to provide solutions to complex engineering problems. Students and faculty should be encouraged to establish collaboration for R&D and product development related projects , with due regard to environmental and societal impact. Feedback from the industry and employers is crucial and an essential part of curriculum review process used to evaluate attainment of the program objectives.

35. 4.1.9 Industrial Linkages 4.1.9.1 Discuss the involvement of industry in discussions and forums, professional practice exposure, and collaborative projects / research for the solutions to engineering problems.

36. List down HEI’s representatives (parties) that will be involved in an accreditation visit Top Management, Dean Head of Department ( HoD ) Academic staff Support staff Stakeholders: Industry & Alumni Students Exercise 36

37. Item/Criteria List down the involved parties Accreditation Planning Opening Meeting Programme Educational Objectives (……) Programme Outcomes (…….) Curriculum (…….) Students (……) Academic & Support Staff (……) Facilities (……) Quality Management System (……) Exit Meeting Exercise Management, HoD , Academic Staff Management, HoD Stakeholders, Management, HoD , All Staff , Students Stakeholders, Management, HoD , All Staff , Students Management, HoD , Students, All staff Management, HoD , Students, All staff Management, HoD , Students, All Staff Management, HoD , Students, All Staff Management, HoD , Students, All Staff Management, HoD 37

38. Sensible questioning Check records Observing processes Analyse inputs and outputs Organised using tables, matrices, flowcharts and checklists PROGRAMME EVALUATOR’S APPROACHES 38

39. What are the six typical words that Programme Evaluators (PEVs) would usually begin with, when questioning? What, Why, Where, Who, When & How Quiz Show Me (the evidence) Programme Evaluators’ (PEVs’) Best friend – 39

40. What are the methods/techniques employed by Programme Evaluators (PEVs) when conducting an accreditation exercise? Check documents, Interview and Observe Quiz 40

41. Cause for concerns at Accreditation Decision Meeting Phases of OBE Planning Implementation Effectiveness CQI List of concerns Breadth & depth (taxonomy & complex problem) Staffing Industrial Training Commitment to change System failure Stagnant (no improvement) Repeat offender Safety 41

42. Rubrics for New Programme , New Cycle & Continuing 42

43. What WA will be observing? Adherence to EAB document EAB PEV’s aplomb and decorum Probing questions (not interrogative) Discussion level Clarity of reports Graduate outcomes Health & safety at HEIs Equivalency of practice 43

44. Opening Meeting You are the Dean/ HoD with three programmes to be evaluated; Mechanical Engineering, Civil Engineering and Electrical Engineering, for the third cycle. Prepare a list of talk points to address the Programme Evaluators (PEVs) at the Opening Meeting. Exercise 44

45. Welcome Evaluation Team Introduce team members Corrective & Preventive Actions from previous accreditation Short presentation on Faculty/ Dept / Prog strengths F ill up with the latest (within a specified timeframe) if any OPENING MEETING 10 minutes 45

46. PEC Manual 2014 Programme Educational Objectives 46

47. Exercise List down potential stakeholders Major Minor

48. Programme Educational Objectives Broad statements : What graduates are expected to achieve ( BE ) a few years after graduation. Linked to programme outcomes Include feedback from employers, alumni, academics and other stakeholders CQI 48

49. PROGRAM EDUCATIONAL OBJECTIVE (PEO) Limit number of statements (manageable) No restatement of outcomes Forward looking and challenging Distinctive /unique features/having own niche S pecific , M easurable, A chievable, R esult oriented, and having a T ime frame ( SMART ) Programme Educational Objectives 49

50. Write down your evaluation on the following PEO statements To provide graduates with sufficient knowledge in engineering. To produce graduates who are sensitive and responsible towards the environment. To prepare graduates for design and innovation. Exercise 50

51. Write down your evaluation on the following PEO statements To provide graduates with sufficient knowledge in engineering and possess the necessary skills for work in the industry. To produce graduates who are sensitive and responsible towards the society, culture and environment. To prepare graduates for work in advanced design and innovation at international level. Exercise 51

52. PEC Manual 2012 Programme Outcomes 52

53. Programme Outcomes (PO) What the graduates are expected to know and able to perform or attain by the time of graduation (skills, knowledge and behaviour /attitude) Need to distribute the outcomes throughout the programme , and not one/two courses only addressing a particular outcome There must be a clear linkage between Programme Objective and Outcomes 53

54. Programme Outcomes Discuss on HEI’s possible approaches or methods to demonstrate implementation of the 12 programme outcomes Discuss on the possible models to show attainment of the 12 programme outcomes Exercise 54

55. 4 YEARS WA 1 ENGINEERING KNOWLEDGE WA 2 PROBLEM ANALYSIS WA3 DESIGN WA5 MODERN TOOLS WA6 ENGR & SOC WA7 ENV & SUST WA8 ETHICS WA4 INVESTIGATION WA9 IND & TEAM WA10 COMMUNICAT-ION WA11 PROJ MGMT & FINANCE WA12 LIFE LONG PEO WHAT YOU WANT YOUR GRADUATES TO BE IN 3 - 5 YEARS EXTRA-CURRICULAR UNIVERSITY EXPERIENCE WA = PO

56. Problem Organised Project Work or POPBL (Project Oriented Problem Based Learning) Problem Analysis Problem Solving Report Literature Lectures Group Studies Tutorials Field Work Experiment 57

57. POPBL Requirements High degree of supervision Office space Lectures to be constantly changing or renewed Flexibility in the distribution of resources 58

58. Instructors/Supervisors Pedagogical skills Scientific skills Time management Project based on staff research 59

59. Graduates’ Strength AALBORG UNIV Strong in problem solving Communication Cooperation General technical knowledge DENMARK TECHNICAL UNIV Specialist knowledge Technical methodology 60

60. Typical questions on PEO/PO What are the PEOs/POs? Who were involved in the development of the PEOs/POs? How were they developed/improved? To what extent the stakeholders were involved? How their attainment were determined? What were the improvements introduced? 61

61. Curriculum Discuss on the possible relationship between taxonomy levels and the different knowledge profile with consideration to the 12 programme outcomes What are typical probing questions in ascertaining that student’s POs have been attained? What are typical probing questions in ascertaining that student’s COs have been attained? Exercise 62

62. ASSESSMENT: Processes that identify, collect, use and prepare data for evaluation of achievement of programme outcomes or educational objectives. EVALUATION: Processes for interpretation of data and evidence from assessment practices that determine the programme outcomes are achieved or result in actions to improve programme . 63

63. Outcome-Based Assessment Implementation Strategy Assessment Strategy Data Sources/Assessment instruments Industrial project Improve student competence in communication, teamwork, and project management Exams, interview, survey, observe, assess skill level, monitor development of skills Reports, interview schedule, survey, observation records, grades of exams and projects, exit skill checklist Design course Address industry needs Assessment by industry, and lecturers List of assessment criteria, observation, reports, interview, students evaluation, exams, exit skill checklist 64

64. Assessment drives learning (necessary evil!) is formative or/and summative; to demonstrate student’s competence in demonstrating a specific outcome is the process that identify, collect, use and prepare data that can be used to evaluate attainment. 65

65. Assessment Do not assess those that have not been taught 66

66. What Assessment? Assessing Student/Cohort (Course Outcome) Assessing Student/Cohort & Faculty (Programme Outcome) 67

67. Assessment Process Anecdotal vs. measured results Reliance on course grades only Over-reliance on indirect assessment (survey) 68

68. University Assessment & Evaluation Student, Alumni Perception Employer, Industry Perception MEASURE & EVALUATE 69

69. How will you know what they have achieved it? Formative Assessment Summative Assessment Course Assessment Program Assessment Assessment Tools Direct and Indirect Assessment 70

70. How do you close the loop ? Assessment Plan Who is doing what and when Stakeholder participation CQI in place 71

71. Programme Outcome Assessment Matrix Outcome indicators & core courses PO 1 PO 2 Project Report A B Course 1 B B Course 2 C B A: slightly, B: moderately, C:substantively - base on a review of course materials (syllabus, learning objectives, tests, other assessment…..) Outcome 1: ability to ….. Outcome 2: ability to ….. 72

72. Course Assessment Matrix Outcome-related learning objectives PO 1 PO 2 Explain A C Perform calculation B B Identify B B Solve B C A: slightly, B: moderately, C : substantively Outcome 1: ability to ….. Outcome 2: ability to ….. 73

73. Rubric 4 – Exceeds Criteria 3 – Meets Criteria 2 - Progressing to Criteria 1 - Below Expectations Content Provides ample supporting detail to support solution/ argument Provides adequate supporting detail to support solution/ argument. Some details but may include extraneous or loosely related material. Inconsistent or few details that may interfere with the meaning of the text. Organization Organizational pattern is logical & conveys completeness & wholeness. Organizational pattern is logical & conveys completeness & wholeness with few lapses. Little completeness & wholeness, though organization attempted. Little evidence of organization or any sense of wholeness & completeness. Style Uses effective language; makes engaging, appropriate word choices for audience & purpose. Uses effective language & appropriate word choices for intended audience & purpose. Limited & predictable vocabulary, perhaps not appropriate for intended audience & purpose. Limited or inappropriate vocabulary for the intended audience & purpose. Consistently follows the rules of standard English. Generally follows the rules for standard English. Generally does not follow the rules of standard English. Does not follow the rules of standard English. Adopted from G.Rogers 74

74. Performance Criteria/ Indicators - Good Teamwork Students are able to demonstrate 1. Positive contribution to the team project (minutes of meeting) 2. Well prepared and participate in discussion (observation) 3. Volunteer to take responsibility 4. Prompt and sufficient attendance 5. Aplomb and decorum 75

75. Quiz Courses PO1 PO2 PO9 PO10 C1 3 2 1 1 C2 2 1 2 2 C3 3 3 2 C4 2 1 3 1 Discuss on the potential problems, if any, where 3, 2, 1, and 0 refer to High, Moderate, Low, and No emphasis, respectively. C1..4 refer to the courses, whereas POs 1,2,9 and 10 refer to Programme Outcomes. How would cohort POs attainment be obtained? 76

76. Quiz PO1 PO2 PO9 PO10 C1- CO 1 + + C1- CO 2 + + + C1- CO 3 + + + C1- CO 4 + + How would you design the assessment for the above matrix? CO : Course Outcomes + : There is assessment 77

77. Quiz Table 1 PO 1 Q1 CO1 + Q2 CO2 + Q3 CO3 + Q4 CO4 + Table 2 PO 1 PO9 Q1 CO1 + CO2 + Q2 CO2 + CO3 + Q3 CO3 + CO4 + Q4 CO4 + Discuss on the attainment of COs and POs for both Tables 1&2, where Qs are questions set to address the COs 78

78. Quiz Delivery Assessment Lecture Laboratory PBL Case Method Project Based Identify suitable assessment techniques for the different delivery modes. 79

79. Lessons learnt from accreditation activities related to assessment Do not know the teaching plan Done without referring to the plan Do not know how to translate plan into assessment Assessing at low-medium level (not challenging) No feedback to students except at end of semester Do not know how to relate assessment to expected outcomes Repetition Bulk marking Traditional assessments 80

80. Big Picture MODEL ? Attainment PHILOSOPHY ? Design Programme or Student Improvement ? Selective Culminating Hybrid Taxonomy Level (Average, From, Up To) Assessment – Constructive Alignment 81

81. Curricula Models Yr. 1 Yr. 4 Yr. 3 Yr. 2 K 70% S&A 30% K 70% K 70% K 70% S&A 30% S&A 30% S&A 30% Distribution of K nowledge, S kills & A ttitude elements throughout the 4 years A B C D 82

82. Evaluation of Outcomes at Programme Level ECV3092 Civil Engineering Design (Capstone Design Course) MyOBE Software 83

83. Evaluation of Outcomes at Programme Level ECV3092 Civil Engineering Design (Capstone Design Course) MyOBE Process Module 84

84. Evaluation of Outcomes at Programme Level ECV3092 Civil Engineering Design (Capstone Design Course) MyOBE Snapshots Lecturers’ Module: Enter all course assessment marks 85

85. Evaluation of Outcomes at Programme Level ECV3092 Civil Engineering Design (Capstone Design Course) MyOBE Snapshots 86

86. Final Year Design Project Final Year Courses Third Year Courses Second Year Courses First Year Courses Final Year Project PO Attainment Knowledge Skills Affective Final Year Project Final Year Design Project Final Year Courses Third Year Courses Second Year Courses First Year Courses Knowledge Skills Affective 87

87. What constitutes strength ? Exceeds the minimum standard set by the EAC Engineering Accreditation Manual . Extensive benchmarking (not only via the external examiners path) with more established programmes/institutions. The curriculum is built on strong fundamentals (engineering sciences) and appropriate engineering knowledge according to the discipline, which transcend national boundaries. Generic attributes (professional and/or interpersonal skills) should also be evident to prepare graduates for the advanced part of their career. 88

88. What constitutes strength? Cont… A curriculum with clear (measurable) objective(s) and outcomes (that satisfies the (12) EAC stipulated outcomes) Involved stakeholders, both internal and external, extensively An appropriate working load for students determined through extensive consultation with the academics (Usually a 15 – 16 credit per semester loading) Blend of delivery methods 89

89. What constitutes strength? Cont… Programme challenges students to achieve greater heights than just satisfying the minimum standard Attain competency in the open-ended project based and problem oriented courses Majority of the staff has PhD qualification and the number available indicates a low staff-student ratio (that enables greater contact with students ) The academic staffs also conduct research that permeates /contributes to teaching and learning. 90

90. What constitutes strength? Cont… Over and above Industrial Training (extensive & distributed professional exposure) that does not compromise on the cognitive domain Ergonomics is taken seriously by the institution to reduce occupational hazard Safety culture Show that they have the plan and the completion of the quality cycles is widespread Monitoring of the QMS also indicates strength. 91

91. What constitutes strength? Cont… Students’ ability to give opinion and articulate with substance Students are clear of their goals upon graduation and highly motivated during their course of study (“constructive criticisms”) Widespread involvement of students in co-curricular activities (not forced as part of curriculum nor limited to small group of students). 92

92. What constitutes strength? Cont… Academic staff with Professional Engineer status Academic staff are actively participating in professional activities (not merely members) Design courses are taught by experienced academics (with consultancy experience or professional engineers). 93

93. What constitutes strength? Cont… Up to date facilities are made available and they exceed the recommended student-equipment ratio appropriate to the relevant discipline. Extensive electronics publications for life long learning, project based courses and the final year project 94

94. Taxonomy & Course Outcome

95. New Bloom’s Taxonomy Knowledge (list) Comprehension (explain) Application (calculate, solve, determine) Analysis (classify, predict, model,derived ) Synthesis (design, improve) Evaluation (judge, select, critique) Bloom’s Taxonomy 96

96. 97

97. Higher order lower order Intermediate 98

98. Higher order lower order Intermediate 99

99. Why are course outcomes important? D efine the type and depth of learning students are expected to achieve P rovide an objective benchmark for formative, summative, and prior learning assessment C learly communicate expectations to learners C learly communicate graduates’ skills to the stakeholders D efine coherent units of learning that can be further subdivided or modularized for classroom or for other delivery modes. G uide and organize the instructor and the learner. 100

100. Three components of a learning outcome Reaching the “Standard” ( criteria of acceptable level of performance) describe the principles used in designing X. (Verb) orally describe the principles used in designing X. (Verb & Condition) orally describe the five principles used in designing X. (Verb & Condition & Standard) design a beam. (Verb) design a beam using Microsoft Excel design template . (Verb & Condition) design a beam using Microsoft Excel design template based on BS 5950:Part 1. (Verb & Condition & Standard) 101

101. Learning outcomes by adding a condition and standard Poor Students should be able to design research. Better Students should be able to independently design and carry out experimental and correlational research. Best Students should be able to independently design and carry out experimental and correlational research that yields valid results . Source: Bergen, R. 2000. A Program Guideline for Outcomes Assessment at Geneva College 102

102. WA Knowledge Profile (WK) 103

103. WA Knowledge Profile (WK) 4 YEARS WK1 natural sciences WK3 engineering fundamentals WK2 mathematics, numerical analysis, statistics, computer and information science WK4 engineering specialist knowledge WK5 engineering design WK6 engineering practice WK8 research literature WK7 engineering in society 104

104. Theory-based natural sciences WK1 Conceptually-based mathematics, numerical analysis, statistics and formal aspects of computer and information science to support analysis and modelling WK2 Theory-based engineering fundamentals WK3 Engineering specialist knowledge that provides theoretical frameworks and bodies of knowledge for the practice areas; much is forefront WK4 WA Knowledge Profile (Curriculum) 105

105. WA Knowledge Profile Knowledge that supports Engineering design in the practice areas WK5 Knowledge of Engineering practice (technology) in the practice areas WK6 Comprehension of the role of Engineering in society and identified issues in engineering practice : ethics and professional responsibility of an engineer to public safety; the impact of engineering activity : economic, social, cultural, environmental and sustainability WK7 Engagement with selected knowledge in the Research literature WK8 106

106. WA Programme Outcome or Graduate Attributes (WA) 107

107. Washington Accord Graduate Attributes PROGRAMME OUTCOMES WA1 Engineering Knowledge Breadth & depth of knowledge WA2 Problem Analysis Complexity of analysis WA3 Design/Development of Solutions Breadth & uniqueness of engineering problems i.e. the extent to which problems are original and to which solutions have previously been identified and coded WA4 Investigation Breadth & depth of investigation and experimentation WA5 Modern Tool Usage Level of understanding of the appropriateness of the tool WA6 The Engineer and Society Level of knowledge and responsibility WA7 Environment and Sustainability Type of solutions WA8 Ethics Understanding and level of practice WA9 Individual and Team Work Role in and diversity of team WA10 Communication Level of communication according to type of activities performed WA11 Project Management and Finance Level of management required for differing types of activity WA12 Life-long Learning Preparation for and depth of continuing learning 108

108. Engineering Knowledge (WA1) Apply knowledge of mathematics, natural science, engineering fundamentals and an engineering specialisation to the solution of complex engineering problems ; (WK1 to WK4) PROGRAMME OUTCOME WK = Knowledge Profile = Curriculum WA = Programme Outcome 109

109. Problem Analysis - Complexity of analysis (WA2) Identify , formulate, research literature and analyse complex engineering problems reaching substantiated conclusions using first principles of mathematics, natural sciences and engineering sciences (WK1 – WK4) PROGRAMME OUTCOME 110

110. Design/Development of Solutions – Breadth and uniqueness of engineering problems i.e. the extent to which problems are original and to which solutions have previously been identified or codified (WA3) Design solutions for complex engineering problems and design systems, components or processes that meet specified needs with appropriate consideration for public health and safety, cultural, societal, and environmental considerations (WK5) PROGRAMME OUTCOME 111

111. Investigation - Breadth & Depth of Investigation & Experimentation (WA4) Conduct investigation of complex problems using research based knowledge (WK8) and research methods including design of experiments, analysis and interpretation of data, and synthesis of information to provide valid conclusions PROGRAMME OUTCOME 112

112. Modern Tool Usage - Level of understanding of the appropriateness of the tool (WA5) Create, select and apply appropriate techniques, resources, and modern engineering and IT tools , including prediction and modelling , to complex engineering problems, with an understanding of the limitations. (WK6) PROGRAMME OUTCOME 113

113. The Engineer and Society - Level of knowledge and responsibility (WA6) Apply reasoning informed by contextual knowledge to assess societal, health, safety, legal and cultural issues and the consequent responsibilities relevant to professional engineering practice and solutions to complex engineering problems . (WK7) PROGRAMME OUTCOME 114

114. Environment and Sustainability - Type of solutions (WA7) Understand and evaluate the sustainabilty and impact of professional engineering work in the solutions of complex engineering problems in societal and environmental contexts (demonstrate knowledge of and need for sustainable development) (WK7) PROGRAMME OUTCOME 115

115. PROGRAMME OUTCOME Ethics - Understanding and level of practice (WA8) Apply ethical principles and commit to professional ethics and responsibilities and norms of engineering practice. (WK7) 116

116. PROGRAMME OUTCOME Individual and Team Work – Role in and diversity of team (WA9) Function effectively as an individual, and as a member or leader in diverse teams and in multi-disciplinary settings 117

117. Communication – Level of communication according to type of activities performed (WA10) Communicate effectively on complex engineering activities with the engineering community and with society at large, such as being able to comprehend and write effective reports and design documentation, make effective presentations , and give and receive clear instructions PROGRAMME OUTCOME 118

118. PROGRAMME OUTCOME Project Management and Finance – Level of management required for differing types of activity (WA11) Demonstrate knowledge and understanding of engineering and management principles and economic decision-making and apply these to one’s own work, as a member and leader in a team, to manage projects and in multidisciplinary environments 119

119. PROGRAMME OUTCOME Life-long Learning – Preparation for and depth of continuing learning (WA12) Recognise the need for, and have the preparation and ability to engage in independent and life-long learning in the broadest context of technological change 120

120. WK1 natural sciences WK2 mathematics, numerical analysis, statistics, computer and information science WK3 engineering fundamentals WK4 engineering specialist knowledge WK5 engineering design WK6 engineering practice WK7 engineering in society WK8 research literature WA1 ENGINEERING KNOWLEDGE WA2 PROBLEM ANALYSIS WA3 DESIGN WA5 MODERN TOOLS WA6 ENGR & SOC WA7 ENV & SUST WA8 ETHICS WA4 INVESTIGATION WA9 IND & TEAM WA10 COMMUNICAT-ION WA11 PROJ MGMT & FINANCE WA12 LIFE LONG 121 4 YEARS

121. WA Complex Problem (WP) 122

122. Complex Problem Uncertain Change Difficult Confusing Intractable Contentious Decision Strategy Idea Product Need to think broadly and systematically and see the big picture 123

123. Difficulty & Uncertainty Complexity – the problem contains a large number of diverse, dynamic and interdependent elements Measurement – it is difficult or practically unfeasible to get good qualitative data Novelty – there is a new solution evolving or an innovative design is needed 124

124. Scientific/Technical Problems can combine to form A Complex Problem 125

125. Complex Technical 126

126. Characteristics Complex Problems No definitive problem boundary Relatively unique or unprecedented Unstable and/or unpredictable problem parameters Multiple experiments are not possible No bounded set of alternative solutions Multiple stakeholders with different views or interest No single optimal and/or objectively testable solution No clear stopping point Technical Problems Isolatable boundable problem Universally similar type Stable and/or predictable problem parameters Multiple low-risk experiments are possible Limited set of alternative solutions Involve few or homogeneous stakeholders Single optimal and testable solutions Single optimal solution can be clearly recognised 127

127. WP1 Depth of Knowledge required Resolved with forefront in-depth engineering knowledge (WK3, WK4, WK5, WK6 or WK8) which allows a fundamentals-based, first principles analytical approach WP2 Range of conflicting requirements Involve wide-ranging or conflicting technical, engineering and other issues. WP3 Depth of analysis required Have no obvious solution and require abstract thinking, originality in analysis to formulate suitable models. WP4 Familiarity of issues Involve infrequently encountered issues WP5 Extent of applicable codes Beyond codes of practice WP6 Extent of stakeholder involvement and level of conflicting requirements Involve diverse groups of stakeholders with widely varying needs . WP7 Interdependence Are high level problems including many component parts or sub-problems. EP1 Consequences Have significant consequences in a range of contexts. EP2 Judgement Require judgement in decision making Complex Engineering Problems have characteristic WP1 and some or all of WP2 to WP7, EP1 and EP2, that can be resolved with in-depth forefront knowledge Complex Problems (Need High Taxonomy Level) 128

128. Problem Oriented, Team-Based Project Work as a Learning/Teaching Device Problem-oriented project-organized education deals with the solution of theoretical problems through the use of any relevant knowledge, whatever discipline the knowledge derives from. We are dealing with KNOW WHY (Research Problems). In design-oriented project work, the students deal with KNOW HOW problems that can be solved by theories and knowledge they have acquired in their previous lectures. (Design Problems). 129

129. Preamble Complex activities means (engineering) activities or projects that have some or all of the following characteristics listed below Range of resources Diverse resources (people, money, equipment, materials, information and technologies). Level of interaction Require resolution of significant problems arising from interactions between wide ranging or conflicting technical, engineering or other issues. Innovation Involve creative use of engineering principles and research-based knowledge in novel ways Consequences to society and the environment Have significant consequences in a range of contexts , characterised by difficulty of prediction and mitigation. Familiarity Can extend beyond previous experiences by applying principles-based approaches. Complex Engineering Activities (Project based) 130

130. WA – WK – WP Relationships WA1 – Engineering Knowledge (Science, Mathematics & Engineering) (WK1, WK2, WK3, WK4) t o solve Complex Engineering Problems WK2 - mathematics , numerical analysis, statistics , computer and information science (WA1) WK1 - natural sciences (WA1) WK3 - engineering fundamentals (WA1) WK4 - engineering specialist knowledge (WA1) WP1 – Depth of Knowledge required: Resolved with forefront in-depth engineering knowledge ( WK3, WK4, WK5, WK6 or WK8) which allows a fundamentals-based, first principles analytical approach WK5 - engineering design (know how) WA3 - Design WK6 - engineering practice (know how) WA5 - Modern Tools WK8 - research literature (know why) WA4 - Investigation (know what)

131. t o solve Complex Engineering Problems WK2 - mathematics , numerical analysis, statistics , computer and information science (WA1) WK1 - natural sciences (WA1) WK3 - engineering fundamentals (WA1) WK4 - engineering specialist knowledge (WA1) WP1 – Depth of Knowledge required: Resolved with forefront in-depth engineering knowledge ( WK3, WK4, WK5, WK6 or WK8) which allows a fundamentals-based, first principles analytical approach WK5 - engineering design WA3 - Design WK6 - engineering practice WA5 - Modern Tools WK8 - research literature WA4 - Investigation WP2 Range of conflicting requirements WP3 Depth of analysis required WP4 Familiarity of issues WP5 Extent of applicable codes WP6 Extent of stakeholder involvement and level of conflicting requirements WP7 Interdependence EP1 Consequences EP2 Judgement Some or all WP2 – WP7, EP1 & EP2

132. t o solve Complex Engineering Problems WK2 - mathematics , numerical analysis, statistics , computer and information science (WA1) WK1 - natural sciences (WA1) WK3 - engineering fundamentals (WA1) WK4 - engineering specialist knowledge (WA1) WP1 – Depth of Knowledge required: Resolved with forefront in-depth engineering knowledge ( WK3, WK4, WK5, WK6 or WK8) which allows a fundamentals-based, first principles analytical approach WK5 - engineering design WA3 - Design WK6 - engineering practice WA5 - Modern Tools WK8 - research literature WA4 - Investigation WP2 Range of conflicting requirements WP3 Depth of analysis required WP4 Familiarity of issues WP5 Extent of applicable codes WP6 Extent of stakeholder involvement and level of conflicting requirements WP7 Interdependence EP1 Consequences EP2 Judgement WK7 - engineering in society WA6 - engineer & society WA7 - environment & sustainability WA8 - ethics Breadth

133. Design Course WK2 - mathematics , numerical analysis, statistics , computer and information science (WA1) WK1 - natural sciences (WA1) WK3 - engineering fundamentals (WA1) WK4 - engineering specialist knowledge (WA1) WP1 – Depth of Knowledge required: Resolved with forefront in-depth engineering knowledge ( WK3, WK4, WK5, WK6 or WK8) which allows a fundamentals-based, first principles analytical approach WK5 - engineering design WA3 - Design WK6 - engineering practice WA5 - Modern Tools WK8 - research literature WA4 - Investigation WP2 Range of conflicting requirements WP3 Depth of analysis required (WA2) WP4 Familiarity of issues WP5 Extent of applicable codes WP6 Extent of stakeholder involvement and level of conflicting requirements WK7 (WA6, WA7, WA8) WP7 Interdependence EP1 Consequences EP2 Judgement WK7 - engineering in society WA6 - engineer & society (WK7) WA7 - environment & sustainability (WK7) WA8 – ethics (WK7) WA2 - Problem Analysis (WK 1-4) WA9 - Individual and Team Work WA10 - Communication WA11 - Project Management and Finance WA12 - Life-long Learning

134. Example 1: Complex Problem Solving Two villages in Timbuktu are separated from each other by a valley, at its deepest section about 30 metres . The valley is dry all the year around, except for the four months, from October to December each year, where torrential rainfall can flood major parts of the valley to a depth of over 12 metres in some site. The soil is generally lateritic with firm bedrock underneath. A bridge connecting the two villages is in a state of disrepair and has to be replaced. Write a project brief on how would you approach to design for the replacement bridge. You are limited to the use of locally available building materials . Heavy equipment is not available for the construction. 135

135. Aspects Economics Social Environment Ethics Management Technology Analysis Evaluation 136

136. Thinking Site condition Weather Available technology Building materials Design Costing Scheduling 137

137. Solutions? Problem solving skills Formulate the problem Literature Experiment? 138

138. Assessment Report – style and content (flow) Display – attractive ? Viva / Articulation Teamwork Management – scheduling 139

139. Sandy soil Fissured Rocks Igneous rock Clayey soil Groundwater flow Spring River Example 2: Complex Problem Solving 140

140. How does complexity relates to curriculum? General Subjects Industrial Placement Core & Specialist (Engineering) Subjects – Complex Problem Solving Elective Subjects – Complex Problem Solving Design Project – Complex Engineering Activities Final Year Project – Complex Problem Solving 141

141. Closing Remarks 142

142. Thank You

143. Appendix 144

144. Complex Problem Solving (CPS) D ynamic , because early actions determine the environment in which subsequent decision must be made, and features of the task environment may change independently of the solver’s actions ; Time - dependent , because decisions must be made at the correct moment in relation to environmental demands; C omplex , in the sense that most variables are not related to each other in a one-to-one manner 145

145. Microworld CPS Model The problem requires not one decision , but a long series, in which early decisions condition later ones. For a task that is changing continuously , the same action can be definitive at moment t1 and useless at moment t2. Include novel solutions to an old dilemma in general science (external validity vs. experimental control) 146

146. Expert-novice CPS Model Expert-novice approach most of the time produces conclusions that are crystal-clear . It almost guarantees statistically significant results, because the groups compared (expert and novices) are very different and tend to perform very differently when confronted with similar experimental situations (Sternberg 1995). 147

147. Naturalistic decision making (NDM) Naturalistic decision making (NDM) (e.g., Zsambok and Klein 1997, Salas and Klein 2001) ‘real-world’ task Example interviewing firefighters after putting out a fire or a surgeon after she has decided in real time what to do with a patient. 148

148. Dynamic decision making DDM Dynamic decision making (DDM) (Brehmer 1992, Sterman 1994). Discrete dynamic decision tasks that change only when the participant introduces a new set of inputs. Variables like time pressure have been successfully integrated in models like Busemeyer and Townsend’s (1993) decision field theory 149

149. Implicit learning in system control This tradition has used tasks like the sugar factory (Berry and Broadbent 1984) or the transportation task (Broadbent et al. 1986), that are governed by comparatively simple equations. The theorization and computational modeling in this branch of CPS are extremely rich. Models are based on exemplar learning, rule learning, and both (e.g., Dienes and Fahey 1995, Gibson et al. 1997, Lebiere et al. 1998). 150

150. European complex problem solving (CPS) Initiated by Dörner ( Dörner and Scholkopf 1991, Dörner and Wearing 1995) A large number of tasks that have been considered complex problem solving are nowadays affordable for theory development and computer modeling (e.g. Putz-Osterloh 1993, Vollmeyer et al. 1996, Burns and Vollmeyer 2002, Schoppek 2002) T ransport real-life complexity to the lab in a way that can be partly controlled 151

151. Time related Time variant – time invariant (dynamic vs. static systems) Continuous time – discrete time. Degree of time pressure – decision has to be made quickly 152

152. Variable related Number and type (discrete/continuous) of variables Number and pattern of relationships between variables Non-Linear - Linear 153

153. System behaviour related Opaque - transparent. Stochastic - deterministic Delayed feedback - immediate feedback. 154

154. Delivery Knowledge-lean vs. knowledge-intensive Skill based vs planning based (reactive vs predictive Learning vs. no learning during problem solving Understanding-based vs. search-based problems Ill-defined vs. well-defined 155

155. Conclusion Problem solving has been traditionally a task-centered field. VanLehn (1989) think that ‘task’ and ‘problem’ are virtually synonymous. 156

156. 157

157. The author would like to thank the contributors of the clip arts used in this presentation 158