The role of the reviewer is one of Quality Assurance. Someone who supervises and reviews the work of assessors to ensure a fair and robust assessment process has been applied. There are three options for access privileges for the reviewer: Basic Access, View Marking Information and Decide Final Grades which will be explained in more detail below.
The Reviewer Overview Screen
- Icon designates the type of flow e.g. FLOWassign
- Flow ID is a unique identifier, and is a searchable field
- Flow title and subtitle identify each flow, and is a searchable field
- Start and end dates for the marking period of each flow
- The archive holds all flows that have ended on which the reviewer has been assigned
- Toggle any of the columns shown to be on/off
- Provides access to the flow
By default, the reviewer has access to the participant's submission, the similarity report (if relevant), and the summary with comments by the assessor(s). Access to scoring information is not available.
- See the assignment and additional material for assessors
- See start and end dates of the participation and marking periods
- Click to see a list of manager(s) on the flow
- This shows which marking procedure is used on the flow
- This shows the percentage value of the similarity analysis and links directly to the detailed similarity report (only where applicable)
- Access the submission via the marking tool or go directly to the shared summary for the participant
The reviewer can view both the work submitted by the participant and the scores and feedback by the assessor.
If the manager has allowed the reviewer to see the marking information, they will have access to the same information as above, but with an added column on the flow page that shows the registered (completed) grades.
- See the assignment and additional material for assessors
- See start and end dates of the participation and marking periods
- Click to see a list of flow managers
- This shows which marking procedure is used on the flow
- Filter the list by assessors
- Download the submissions as a ZIP file (FLOWassign and FLOWlock only)
- This shows the percentage value of the similarity analysis and links directly to the detailed similarity report (FLOWassign and FLOWlock only)
- Access the submission via the marking tool or go directly to the summary for the participant
- This column shows the grade given by the assessors assigned to the participant
- If several assessors have been assigned to a flow and they have submitted the same grade, this will be shown here
It is possible to have this setting on your licence as the default for all reviewers. Contact Support for further information.
Additionally, reviewers with access to marking information can also view the statistics of a FLOWmulti exam.
For more details on flow statistics please see the statistics article.
Actions made by assessors, reviewers and managers are recorded in the Grade Audit Trail. Reviewers access the Grade Audit Trail via the flow page:
Only submitted grades are visible in the Grade Audit Trail. Entered grades will not be displayed.
If the grades from multiple assessors have been submitted but there is a conflict between the co-assessors, the conflicting grades will be displayed in the Grade Audit Trail.
Any comments submitted to accompany the submitted grades will be displayed. So too will any Alternative Grades (please see Grade Audit Trail in the article Marking Overview (Submission of Marks and Grades).
The Grade Audit Trail will also show if an assessor has withdrawn their submitted grade (please see Withdrawing a Grade in the article Marking Overview (Submission of Marks and Grades) or if a reviewer or manager has unlocked a submitted grade.
Any actions completed prior to the release of this new feature on Friday 8 October 2022 will not be recorded in the Grade Audit Trail.
When marking is done by more than one assessor, the flow can be set up that the final grade is determined by the reviewer.
When Approval by Reviewer has been chosen as the marking procedure by the manager, it is the reviewer that determines the final grade for the participant.
Further details on this process can be found in the article: Marker Agreement vs Reviewer Approval.
If you are added as a reviewer on a FLOWmulti with item-based marking and have been allowed to see or submit the grades, it will look a bit different than the other flow types:
- See the assignment and additional material for assessors
- See start and end dates of the assessment period
- Click to see a list of flow managers
- This shows the marking procedure for the flow
- Filters the list by assessors
- View Statistics of a FLOWmulti exam
- Here you can either open the paper in the marking tool or go directly to the summary for the participant
- This column shows the sum of marks given by either the automated scoring or the assessor. Those indicated by a padlock have been submitted manually by an assessor.
- By clicking on the marks in the column, you can open a small overview of the item, and here you can cancel the assessors’ submitted points by clicking the unlock icon but only if the final scores have not been submitted.
- This section will only be available if you are allowed to submit the final grades
- Opens the grade scaler, which will be used to convert marks to grades
- Submits all the available marks
- This column shows all the marks that were converted from the point in the previous column
- Here you can see all the marks that are submitted
Reviewers can be assigned to re-markings and the experience is exactly the same as detailed above. The only differences are the added labels, that clarify that it is a re-marking flow and distinguishes it from the original flow.