Depending on the the way the manager has setup a flow, point-based assessment for an assessor can take two forms - Assessors are assigned to participants or Assessors are assigned to pages. This guide will try to explain an run through an example of the latter, assigned to pages. See FLOWmulti: Point-based assessment - Assigned to participants for the former.
When managers has chosen to assign assessors to pages, most of the assessment period and rules are controlled by the manager. Pages are assigned and distributed to assessors of the flow by the manager. The reasoning behind the distribution can vary. One reason could be that specific page on a specific subject might be more attuned to a specific assessor. Another reason could be that a pages does no allow for automatic assessment scoring (e.g. essay questions), meaning that assessors need to manually determine the point score of the page for participants.
As an assessor, when you first enter the assessment period and open the assessment tool (Annotate) you will notice that you might not have access to all pages of the participants and that the number of pages you have access to, can change between participants. See the two pictures below for an example.
On participant one ("Demo studerende 1), you only have access to one page of the test (This test example consisted of five pages totally)
On participant two ("Demo studerende 2"), you have access to two pages of the test.
The reason for this distribution is that pages can be distributed equally between assessors - meaning that pages can be distributed 50/50. From the flow overview (click flowname in the top of the annotate page) you can access how many pages you are assigned to for each participant.
Furthermore, the and symbols inform you of the state of the participant for you assessment. More on this in the next segment.
Assessing a page
Point-based assessment, as the name suggests, is the idea that you grade student based on the participants point score in a FLOWmulti exam. When assessing a FLOWmulti, assigned to pages, the assessor has to determine the score for each page assigned. This is done via annotations. To score a page you undergo the following steps.
- Click the button to add an annotation to the page. This opens the annotation editor on the right side of your screen.
Note: to the right of the add button you see the current score of the page. This page has a automatically assessed score of 0 because it could not be assessed by the system (Essay). However there could be a score here. Have in mind, annotations only subtract or add to this score.
- (Optional) Here you can write a comment for you annotation (reasoning behind the score). This is mostly used when annotations are shared with participants or cooperating assessors (Sharing controlled by the "Private" dropdown menu in the bottom.
- Choose addition or subtraction from the current point score of the page.
Note: The amount of points you can add here is determined by the maximum (2 points) and minimum (-1 points), determined by the manager when setting up the flow.
- Close the editing view of the annotation. The annotation is saved as you edit it.
As you edit the annotation to the page, you should see the score of the page change as you go. When you have closed the annotation editor notice the following:
- There is now one annotation to the page.
- The score added or subtracted by annotation(s).
- The current score of the page.
- You can access cooperating assessors information here, if needed.
- To finish assessing a page you have to approve it (or mark it as "Done"). The button will turn blue when you have marked it "Done".
Continue this process for all pages assigned to you. You can either continue to the next page of the same participant or skip to the same page for the next eligible participant by pressing the button at the bottom of the page.
Registering and ending assessment
When you have approved all pages for a participant, you can register the approved score of that participant.
Press the name of the participant in the top right corner. From here you can register the specific participant by pressing the button (toolbox) and clicking "Submit score for this participant". This will lock all the pages of the participant for the assessor. The pages can only be opened again by the manager.
You can also click "Go to submission of marks" to go to the register page. From here you can register all participant where you have approved the assessed scores. When scores have been registered, the assessors job is done. Grade of the participants is defined and registered by the manager, depending on the scores.
Cooperating assessors and score conflicts
The manager can create assessor groups where assessors have to cooperate and agree on the scores for the pages that they are assigned to together. As per usual, conflicts in the scores will unregister the registered page in question.