Depending on the the way the manager has set up a flow, point-based assessment for an assessor can take two forms - assessors are assigned to participants or assessors are assigned to sections. This guide will try to explain and run through an example of the latter: assigned to sections. See FLOWmulti: Point-based assessment - assigned to participants for the former.
When managers has chosen to assign assessors to sections, most of the assessment period and rules are controlled by the manager. Sections are assigned and distributed to the assessors of the flow by the manager. The reasoning behind the distribution can vary: one reason could be that specific section on a specific subject might be more attuned to a specific assessor; another reason could be that a section does not allow automatic assessment scoring (e.g. essay questions), meaning that assessors must manually determine the point score of the section for participants.
As an assessor, when you first enter the assessment period and open the assessor tool, you will notice that you might not have access to all the sections of the participants and that the number of sections you have access to can vary between participants. See the two pictures below for an example.
On participant 1 (Demo studerende 1), you only have access to one section of the test (This test example consisted of five sections totally).
On participant 2 (Demo studerende 2), you have access to two sections of the test.
The reason for this distribution is that sections can be distributed equally between assessors - meaning that sections can be distributed 50/50. From the flow overview, (click flowname in the top of the annotate page) you can see how many sections you are assigned to for each participant.
Assessing a section
With point-based assessment, as the name suggests, the idea is that you asses participants based on their point score in a FLOWmulti exam. When assigned to sections and assessing a FLOWmulti, the assessor should determine the score for each section assigned. This is done via annotations. To score a section you go through the following steps:
- Click the button to add an annotation to the section. This opens the annotation editor on the right side of your screen.
Note: to the right of the add button, you see the current score of the section. This section has an automatically assessed score of 0, because it could not be assessed by the system (due to it being an essay). However, there could be a score here. Have in mind, annotations only subtract or add to this score.
- (Optional) Here you can write a comment for you annotation (reasoning behind the score). This is mostly used when annotations are shared with participants or co-assessors (sharing controlled by the "Private" drop-down menu in the bottom).
- Choose addition or subtraction from the current point score of the section.
Note: The amount of points you can add here is limited by the maximum (2 points) and minimum (-1 points), set by the manager when setting up the flow.
- Close the editor. The annotation is saved as you edit it.
As you edit the annotation to the section, you should see the score of the section change as you go. When you have closed the annotation editor notice the following:
- There is now one annotation to the section.
- The score added or subtracted by annotation(s).
- The current score of the section.
- You can access co-assessors' information here, if needed.
- To finish assessing a section, you have to approve it (or mark it as "Done"). The button will turn blue when you have marked it "Done".
Continue this process for all section assigned to you. You can either continue to the next section of the same participant or skip to the same section for the next eligible participant by pressing the button at the bottom of the section.
Registering and ending assessment
When you have approved all sections for a participant, you can register the approved score of that participant.
Press the name of the participant in the top right corner. From here you can register the specific participant by pressing the button (toolbox) and clicking "Submit score for this participant". This will lock all the sections of the participant for the assessor, as shown below. The sections can only be unlocked by the manager.
You can also click "Go to submission of assessments" to go to the register page. From here, you can register all participant where you have approved the assessed scores. When scores have been registered, the assessors job is done. The assessments of the participants is defined and registered by the manager, depending on the scores.
Co-assessors and score conflicts
The manager can create assessor groups where assessors have to cooperate and agree on the scores for the sections that they are assigned to together. As usual, conflicts in the scores will unregister the section in question.