Organization and analysis of assessment data from alternative Methods Group 6
Introduction In order to analyze, present, and select the results of alternative forms of assessment, you need to know how to use and interpret results of descriptive and inferential statistics. You are expected to read these types of Statistics before you can report the findings from scales, checklists, and rubrics.
How do we quantify results from rubrics? In the creation of rubrics, there are scales that represent the degree of performance. This degree of performance can range from high to low degree of proficiency.
These are examples of the degree of proficiency with their corresponding points: Beginner Moving Forward proficiency Proficient Very proficient 1 Point 2 points 3 points 4 points Poor Needs improvement Good Very good 1 Point 2 points 3 points 4 points
These are examples of the degree of proficiency with their corresponding points: Minimal Partial Complete 1 Point 2 points 3 points Never Seldom Occasionally Frequently Always 1 Point 2 points 3 points 4 points 5 points
These are examples of the degree of proficiency with their corresponding points: Rare Sporadic Consitent 1 Point 2 points 3 points Novice Intermediate Advance Superior 1 Point 2 points 3 points 4 points
These are examples of the degree of proficiency with their corresponding points: Needs improvement Proficient Excellent 1 Point 2 points 3 points Inadequate Needs improvement Good Excellent 1 Point 2 points 3 points 4 points
These are examples of the degree of proficiency with their corresponding points: Limited Partial Thorough 1 Point 2 points 3 points Absent Developing Adequate Fully developed 1 Point 2 points 3 points 4 points
These are examples of the degree of proficiency with their corresponding points: Emerging Developing Achieving 1 Point 2 points 3 points Not there yet Shows growth Proficient 1 Point 2 points 3 points
These are examples of the degree of proficiency with their corresponding points: Poor Fair Good Excellent 1 Point 2 points 3 points 4 points
The points depend on the quality of the behavior shown by the learner’s performance. The reliability of the assigned points can be determined when the scoring of two or more observers to the same behavior is consistent. The consistency of the ratings can be obtained using a coefficient of concordance. The Kendall’s w coefficient of concordance is used to test the agreement among raters.
If a performance task was demonstrated by five students and there are three raters. The rubric used a scale of 1 to 4 where 4 is the highest and 1 is the lowest. Five Demonstrations Rater 1 Rater 2 Rater 3 Sum of Ratings D D² A 4 4 3 11 2.6 6.76 B 3 2 3 8 -0.4 0.16 C 3 4 4 11 2.6 6.76 D 3 3 2 8 -0.4 0.16 E 1 1 2 4 -4.4 19.36
The scores given by the three raters are first computed by summating the ratings for each demonstration. The mean is obtained from the sum of ratings ( X=8.4). The mean is subtracted to each of the Sum of Ratings (D). Each difference is squared (D²), then the sum of squares is computed (£D² = 33.2) The mean and summation of squared differences are substituted in the Kendall’s w formula. In the formula, m is the numbers of raters..
Formula: Kendall’s w coefficient of 0.38 is an of the agreement of the three raters in the five demonstrations. There is a moderate concordance among the three raters because the coefficient is far from 1.00.
How do we quantify results from scales and checklists? Scales could be a measure of noncognitive dimensions of students’ behavior. When the items in the scale are answered by students, the response format quantifies the behavior measured by the scale. The types of response format vary depending on the nature of the behavior measured.
The Likert scale is used to measure students’ favorability and unfavorablility toward a certain object. The favorability will depend on the degree of agreement or disagreement to a standpoint. Likert Scale Example: The policeman is helpful in the street. _Strongly agree _Agree _Disagree _Strongly disagree
To quantify the scales, a numerical score can be assigned to each of the responses. To get the total score for the overall scale, the points for each item can be summated. The total score is a representation of the overall trait being measured.
This is used to measure how often a habit is done. The items here are measures of a habit. Verbal Frequency Scale Example: I read a book. _Always _Often _Sometimes _Rarely _ Never
Similar to the idea of quantifying a Likert scale, a verbal frequency scale is scored by assigning numerical values for every response. The total score for the habit can also be estimated through a total score summating the scores of all items. A high score means high frequency of the habit while the low score means lower frequency for the habit.
This is used when a large array of ratings is provided among the participants within a continuum. The extreme points of the scale are provided with a descriptor. Linear Numeric Scale Example: Rate the following personnel in your school on how valuable each one is to your learning. Use the scale below and write the corresponding number for your answer.
_1. Teacher
_2. Academic Coordinator
_3. Guidance Counselor
_4. Assistant principal
_5. Principal
This scale is used to describe the object or behavior by making use of two opposite adjectives. Semantic Differential Scale Example: Rate the teaching of your teacher based on the characteristics provided.
This scale uses illustrations to represent the degree of presence or absence of the characteristics measured. This is usually for respondents, such as young children, who have limited vocabulary. Graphic Scale Example: Rate the teaching of your teacher based on the characteristics provided.
Example: Rate how you like the following food:
How do we quantify results from portfolios? Assessment data generated from portfolios can both be qualitative or quantitative . When assessing portfolios using quantitative approach, scales and rubrics can be used. Qualitative assessment requires criteria and narrative feedback provided to the learner.
The following criteria can be used when assessing portfolios: 1. Completeness of the entries – All the parts of the portfolio listed by the teachers are present. 2. Accuracy of the reflections – An authentic reflection is made for every entry in the portfolio. The reflection provides the insights on the realization of the learner about his or her weaknesses and the improvement that needs to be done.
3. Organization of content and proper sequence – The portfolio can be classified according to lesson and the entries show the draft and final work with proper label. There are markers or tabs provided so that the teacher can easily browse through the contents. There is a table of contents found at the start of the portfolio. Every entry is properly labeled.
How do we summarize results? When results of assessment are summarized, the teacher needs to think about two things:
1. The kind of scores that will be presented – The teacher may require to have the raw score, percentage, or transmuted grade. The average and summation of scores may be required depending on the grading system.
2. The tabular or graphical presentation of the scores - Scores can be presented in a tabular or graphical manner. Below is an example of a tabular presentation of learner’s scores in a formative assessment. Record of Formative Assessment in Mathematics Name of Student: Juan Dela Cruz Grade: 3 Subject: Mathematics
Learning Competency Exercise 1 (10 items) Percent Correct Exercise 2 (10 items) Percent Correct Exercise 3 (10 items) Percent Correct Multiplies three one-digit numbers using the associative property of multiplication 2 20% 5 50% 8 80%
Learning Competency Exercise 1 (10 items) Percent Correct Exercise 2 (10 items) Percent Correct Exercise 3 (10 items) Percent Correct Multiplies two- to three-digit numbers by one-digit numbers without our with regrouping. 3 30% 6 60% 9 90%
Learning Competency Exercise 1 (10 items) Percent Correct Exercise 2 (10 items) Percent Correct Exercise 3 (10 items) Percent Correct Multiplies two- to three-digit numbers by multiples of 10 and 100. 1 10% 6 60% 9 90%
Guidelines in Giving Qualitative Feedback The contents of the feedback are based and within the confines of the criteria. The feedback should be immediate to correct the error. The learner needs to be provided with an opportunity to redo and resubmit the task.
Guidelines in Giving Qualitative Feedback The feedback should inform the students on what to do to become better in their performance or behavior. The recommendation can be:
✓ a suggested procedure
✓ how to correct the errors
✓ the kind of thinking required to get the answer
✓ where to locate the answer
Guidelines in Giving Qualitative Feedback Detail the feedback if the learner needs more information. The feedback can be short if the learner knows what to do. Feedback can come in the form of verbal cues and gestures so that the learner is not disrupted while performing.