This refers to Moodle 4.3 from August 2024 onwards
The scoring algorithm for the Peer Assessment activity is based on the way in which Web-PA would grade an assessment.
Before looking at the worked example, it is important to understand how the Peer Assessment activity works for assessment. Following the Web-PA model one student submits the project files or assessment on behalf of the whole group. Once this submission has been made it cannot be changed. You then define the percentage out of 100 that will be graded based based on the submitted work (say 80%) and the remainder (20%) which will be made up of the peer assessment. So before we begin it is clear in this example that all students will get the same base grade for 80% of the assessment, based on the tutor mark for the submitted work. The 20% will be made up of a Web-PA score as detailed in the worked example below. This will be adjusted in the algorithm to take account of any students who do not submit a peer review.
Below is a worked example of the way in which the Peer Assessment activity calculates a score for students in a group. Note: This is a very simple example, which only includes one criteria. In an example with more than one criteria, the final Web-PA score for each student per criteria will be added together and divided by the number of criteria to give an accurate Web-PA score.
All calculations in this example are given to two decimal places, so there is a margin of error. The actual algorithm is able to work to many more decimal places and so will be more accurate.
First we need to evaluate the data students have provided when giving their peer assessment marks. In this example the rows across are the marks given by each student, whereas the columns down show the marks each student received from their peers. You will see that there are no marks given by student 5, as shown by the dashes in the that row.
| Student 1 | Student 2 | Student 3 | Student 4 | Student 5 | Total Given | |
|---|---|---|---|---|---|---|
| Student 1 | 4 | 4 | 3 | 2 | 1 | 14 |
| Student 2 | 3 | 5 | 3 | 2 | 0 | 13 |
| Student 3 | 4 | 4 | 4 | 4 | 4 | 20 |
| Student 4 | 3 | 5 | 4 | 3 | 1 | 16 |
| Student 5 | - | - | - | - | - | 0 |
| Total Received | 14 | 18 | 14 | 11 | 6 | 63 |
The first step is to total up the marks allocated by each student (including themselves). So total up each row. In some scenarios you might not want students to mark themselves. In this case a dash will show under the relevant column and you could use 0 in your calculation.
Student 1 awarded: 4+4+3+2+1 = 14 marks
Student 2 awarded: 3+5+3+2+0 = 13 marks
Student 3 awarded: 4+4+4+4+4 = 20 marks
Student 4 awarded: 3+5+4+3+1 = 16 marks
Student 5 awarded: 0+0+0+0+0 = 0 marks
Knowing how many marks each student allocated in this example, we can now calculate a fractional score for each. There is one cake (the group project) but students have given each other different sized slices depending on their perceived ability to meet the outline criteria.
Student 1 was allocated 14 marks, so we have to split their cake into 14 slices.
Student 1 gave themselves 4 marks = 4 / 14 = 0.29
Student 1 gave 4 marks to Student 2 = 4 / 14 = 0.29
Student 1 gave 3 marks to Student 3 = 3 / 14 = 0.21
Student 1 gave 2 marks to Student 4 = 2 / 14 = 0.14
Student 1 gave 1 mark to Student 5 = 1 / 14 = 0.07
Note: You can check your working by adding up all the fractional scores and they should equal 1. You need to repeat this process for every student.
In order to calculate a correct Web-PA score through the Peer Assessment activity, we need to know a few more pieces of information about the group and the assessment task.
How many students were in the group and how many submitted marks? In this example there were 5 students but only 4 of them submitted marks.
To take account of the one student who did not complete the task, a multiplication factor is generated to bring the total number back up to 5.
5 (students in a group) / 4 (number who submitted) = 1.25 (our multiplication factor, sometimes called the fudge factor).
Now we can calculate the Web-PA scores. Taking Student 1 as our example, we add up all the fractional scores she was awarded. We calculated fractions for Student 1 above, but we also worked out all other students in the same way.
Student 1 gave themselves 0.29 (as above)
Student 2 gave Student 1 0.23 (= 3 / 13)
Student 3 gave Student 1 0.20 (= 4 / 20)
Student 4 gave Student 1 0.19 (= 3 / 16)
Student 5 didn't submit so effectively gave a 0.00
If you add up all the individual scores above, you get a Web-PA score of 0.91 (0.29 + 0.23 + 0.20 + 0.19 + 0.00). However, student 5 did not submit, so we need to bring in our multiplication factor to generate the actual Web-PA score for student 1, which is 0.91 x 1.25 (the multiplication factor) = 1.14.
If we do the same calculation for the other students we get the following:
Student 2 Web-PA score: 1.47
Student 3 Web-PA score: 1.11
Student 4 Web-PA score: 0.85
Student 5 Web-PA score: 0.41
You will know that these Web-PA scores are correct, if you add them together. They should equal the number of students in the group. Adding up the scores gives us 4.98 (which allowing for rounding errors to two decimal places gives us 5).
We can now calculate the final grade for all students.
The group project was marked at 60 out of 100. Ordinarily, we would have given everyone in the group the same score of 60. But now we can use the Web-PA score generated from the Peer Assessment activity to give proportionally higher or lower marks according to the students' actual performance as ranked by their peers. For now we can assume we want the peer assessment part to apply to the entire group mark.
Student 1 grade = 1.14 x 80 = 91.2%
Student 2 grade = 1.47 x 80 = 117.6% (or 100% as we can't give grades above 100%)
Student 3 grade = 1.11 x 80 = 88.8%
Student 4 grade = 0.85 x 80 = 68.8%
Student 5 grade = 0.41 x 80 = 32.8%
As you can see, there is a wide variety of marks here including one student who got full marks. But according to the peer assessment, that is a fair conclusion.
In order to adjust the spread of marks, we can define how much of the mark will be fixed (just the tutor mark) and how much will be affected by the peer assessment score. In this scenario we have set 80% original group grade and 20% impacted by the peer assessment task.
So each student will get the original mark for 80% of the overall assignment (so a mark of 48, which is 80% of 60). The remaining 20% (mark of 12) will then be adjusted by the peer assessment scores.
Student 1 grade = 48 + (1.14 x 12) = 61.7%
Student 2 grade = 48 + (1.47 x 12) = 65.6%
Student 3 grade = 48 + (1.11 x 12) = 61.3%
Student 4 grade = 48 + (0.85 x 12) = 55%
Student 5 grade = 48 + (0.41 x 12) = 59.9%
As you can see now, the spread of grades has been reduced.
There is one further setting that can be applied within the Peer Assessment activity. Should you wish you can apply a non-completion penalty for any student who does not complete the peer assessment part of the assessment. You can choose to penalise them anything up to 100% of their final grade. If we use a penalty of 10% in this example, most of the students would not be affected, but Student 5 (who provided no peer assessment grade) would receive a 10% penalty.
Student 5 grade = 59.9 - 10% = 53.91%.
Please contact your local elearning support team.
|
|
|
If you have any comments on how you think this help can be improved then please email Learning-Technologies@nottingham.ac.uk
