Depending on the the way the manager has setup a flow, point-based assessment for an assessor can take two forms - Assessors are assigned to participants or Assessors are assigned to sections. This guide will try to explain an run through an example of the latter, assigned to sections. See FLOWmulti: Point-based assessment - Assigned to participants for the former.
When managers has chosen to assign assessors to sections, most of the assessment period and rules are controlled by the manager. Sections are assigned and distributed to assessors of the flow by the manager. The reasoning behind the distribution can vary. One reason could be that specific section on a specific subject might be more attuned to a specific assessor. Another reason could be that a section does no allow for automatic assessment scoring (e.g. essay questions), meaning that assessors need to manually determine the point score of the section for participants.
As an assessor, when you first enter the assessment period and open the assessment tool (Annotate) you will notice that you might not have access to all sections of the participants and that the number of sections you have access to, can change between participants. See the two pictures below for an example.
On participant one (Demo studerende 1), you only have access to one section of the test (This test example consisted of five sections totally)
On participant two (Demo studerende 2), you have access to two sections of the test.
The reason for this distribution is that sections can be distributed equally between assessors - meaning that sections can be distributed 50/50. From the flow overview (click flowname in the top of the annotate page) you can access how many sections you are assigned to for each participant.
Assessing a section
Point-based assessment, as the name suggests, is the idea that you grade student based on the participants point score in a FLOWmulti exam. When assessing a FLOWmulti, assigned to sections, the assessor has to determine the score for each section assigned. This is done via annotations. To score a section you undergo the following steps.
- Click the button to add an annotation to the section. This opens the annotation editor on the right side of your screen.
Note: to the right of the add button you see the current score of the section. This section has a automatically assessed score of 0 because it could not be assessed by the system (Essay). However there could be a score here. Have in mind, annotations only subtract or add to this score.
- (Optional) Here you can write a comment for you annotation (reasoning behind the score). This is mostly used when annotations are shared with participants or cooperating assessors (Sharing controlled by the "Private" dropdown menu in the bottom.
- Choose addition or subtraction from the current point score of the section.
Note: The amount of points you can add here is determined by the maximum (2 points) and minimum (-1 points), determined by the manager when setting up the flow.
- Close the editing view of the annotation. The annotation is saved as you edit it.
As you edit the annotation to the section, you should see the score of the page change as you go. When you have closed the annotation editor notice the following:
- There is now one annotation to the section.
- The score added or subtracted by annotation(s).
- The current score of the section.
- You can access cooperating assessors information here, if needed.
- To finish assessing a section you have to approve it (or mark it as "Done"). The button will turn blue when you have marked it "Done".
Continue this process for all section assigned to you. You can either continue to the next section of the same participant or skip to the same section for the next eligible participant by pressing the button at the bottom of the section.
Registering and ending assessment
When you have approved all sections for a participant, you can register the approved score of that participant.
Press the name of the participant in the top right corner. From here you can register the specific participant by pressing the button (toolbox) and clicking "Submit score for this participant". This will lock all the sections of the participant for the assessor. The sections can only be opened again by the manager.
You can also click "Go to submission of marks" to go to the register page. From here you can register all participant where you have approved the assessed scores. When scores have been registered, the assessors job is done. Grade of the participants is defined and registered by the manager, depending on the scores.
Cooperating assessors and score conflicts
The manager can create assessor groups where assessors have to cooperate and agree on the scores for the sections that they are assigned to together. As per usual, conflicts in the scores will unregister the section in question.