Managing Assessors, Reviewers and Marking Settings

Marking_and_Review.png

From the overview in the basic settings of a flow, you have the option to adjust various settings surrounding the Marking and review process of flows.

1. Manage Settings

There are various functionalities to configure the general marking process for a flow. Not all functionalities are available on all flow types.

Depending on your licence settings, not all functionalities may be available in your Marking and Review overview.

Marking Settings

Marking_and_Review_Settings2.png

Early_Release.png

1 2 3 4 5
It is possible to determine whether participants' submissions should be marked and graded by marker agreement or via reviewer approval.
Refer to this article to learn more about both marking methods.
Justification Requests

Grade_Justification.png

If enabled on your licence, you can allow participants to request a justification of their final grade from their assessor when the marking period has ended.

You can decide whether participants' justification requests should be assigned to internal assessors (if available on the flow). If a flow has more than one internal assessor you need to assign the justification request to a assessor. If not enabled, justification requests will be allocated to external assessors. 

Finally, you can allow participants to request a justification within WISEflow (as opposed to only through an external system and the API).

Similarity Control

Enable_Similarity_Control.png

Only on FLOWassign and FLOWlock: If Turnitin or Ouriginal have been integrated on your WISEflow licence, you can decide whether participants' submissions should be checked for containing similar content to other electronic material. When enabled, all papers, which have been submitted, will be sent to Turnitin/Ouriginal service automatically two hours after the (individual) participant period ends. The papers will be tested to see if they contain passages of text that also appear elsewhere (internet, library databases, papers submitted by others etc.). A similarity report will be available to assessors after a few hours.  

Default Sharing State of Marking Work

Default_Sharing_State_of_Marking_Work.png

At the licence level, the default sharing status of all annotations, comments and rubrics from assessors can be selected from Private, Participant, Co-assessors and All. This can be over-ridden at the flow level to the desired default setting. Changing this setting will only affect what the assessor sees as the default sharing status for marking to be completed. Any marked work already completed will not be affected by this change.

2. Manage Assessors
Overview Allocation

Assessor_Overview.png

  1. Decide whether assessors are allocated individually or as an assessor group to participants. If the distribution type is changed after assessors have already been allocated to participants, the assessor allocation will be reset.

    You cannot manually allocate an individual assessor to participants if assessor groups has been chosen as the distribution type. You will then have to create a assessor group for this individual assessor.

    Change_assessor_distribution_type.png

  2. Add the assessors needed for the flow by pressing the green Add assessor button, search the name of the user(s) you want to add to the flow and choose whether they should be internal or external assessor(s) then Save.

    Add_assessor.png

    There is no maximum limit of assessors that can be added to a flow. We do, however, advise you to divide large exams into smaller sub-flows to maintain a better overview. For example, if you have to manage an exam with 30 assessors, we suggest that you divide it into two flows with 15 assessors on each flow. 

    It is possible to add an assessor right up until the marking period has ended. After this, the marking period can be extended and new assessors can be added to the flow again. Assessors do not need to be added to the flow before the flow is activated or before the marking period begins. Assessors added to the flow after activation will not receive the You have been enrolled to flow notification.

  3. Internal/external assessor: This column shows whether the assessor is internal or external. Only external assessors are anonymous to the participants by default.
  4. Rubric: Shows any rubrics that have been associated with an author/author group.
  5. Assessor groups: This column is only available when the distribution has changed to assessor groups. It will contain the names of the group the assessor is part of.
  6. Edit: Change whether a assessor should be defined as internal or external.
  7. Delete: Remove the assessor from the flow.
  8. Assessor groups (only if the distribution type is assessor groups): Create and/or edit assessor groups. You can quickly allocate Allocate_to_all_Icon.png all participants to or remove Remove_Icon.png all participants from an author group.  

 

Grade Audit Trail

Actions made by assessors, reviewers and managers are recorded in the Grade Audit Trail, accessed by managers via the Work Distribution panel of the Manager Assessors page.

Manager_-_Grade_Audit_Trail.png

The Grade Audit Trail column can be added by managing the columns displayed:

Toggle_Grade_Aufit_Trail_Column.png

The Grade Audit Trail will display grades submitted by assessors along with any comments the assessors have made at the time of submission. The audit trail will also show Alternative Grades, conflicting grades from co-assessors, grades withdrawn by assessors (before being finalised) and those unlocked by managers or reviewers.

Any actions completed prior to the release of this new feature on Friday 8 October 2022 will not be recorded in the Grade Audit Trail.

3. Manage Reviewers
Overview Allocation

Add the reviewers needed for the flow by pressing the green Add reviewer button and then search for the users you want to add to the flow.

Add_Reviewer.png

When adding new reviewers, you can decide which privileges they should receive:

Decide_final_assessment_icon.png
Decide final assessment

View_assessment_icon.png
View assessment information

Allocate_to_all_Icon_Reviewer.png
Allocate to all

The reviewer will be able to decide the final grade for participants on the flow.

Only available if the procedure for submitting the final assessment has been set to Approval by reviewer

Tick this box if you want the reviewer to see the assessment information of the participants. Quickly allocate the reviewer to all the participants. You cannot manually allocate a reviewer to participants if the privilege Allocated to All has been checked.

Reviewer_overview.png

  1. You can remove or add columns of information of the reviewers by pressing this button and checking the columns you want to be displayed
  2. Shows the number and percentage of assigned participants to a reviewer.
  3. Quickly remove the reviewer from all allocated participants. Option only available if privilege Allocated to all is NOT checked.
  4. Quickly allocate the reviewer to all participants. Option only available if privilege Allocated to all is NOT checked.
  5. Edit the privileges of the reviewer, see above for more information.
  6. Remove a reviewer from the flow, alongside all of its allocations, comments and annotations.
4. Set up Rubrics

Rubrics are published by authors and can be attached to the flow by you or authors if they are associated with the flow. It is possible to attach several different rubrics to one flow. You can add and manage rubrics on a new subpage. More information can be found in the article Setting Up Rubrics and Using Them for Assessments.

5. Manage Material
There are two different ways to hand out material:
  1. Marking guidance: Files and links can be made available for assessors and/or reviewers to aid the assessment process. Authors added to the flow may also manage the marking guidance.
  2. Other material: Files can only be managed by managers on the flow and will be displayed as additional marking material to assessors and/or reviewers.

Marking_Guidance.png

Choose files and/or links to be handed out to the assessors and reviewers on the flow. When you have added the material you can also choose to which roles these files should be available. 

Add_file.png

When the material has been added, the labels indicate to whom the material is available. Clicking the small icons you have the following options:

Group_Icon.png Edit_Icon.png View_Icon.png Download_Icon.png Delete_Icon.png
Change to whom material is made available Edit filename Preview file Download file Delete file

 

How the material is displayed to assessors and reviewers

When a assessor opens the flow, the material is visible on the flow page under the marking tool and the mark overview. 

Marking_Guidance___Other_Material.png

When a reviewer opens the flow, the marking guidance is visible under Supporting material

Reviewer_view.png

The marking guidance can be added by assigned authors on the flow. If an author has been added to the flow the flow will be visible under Associated flows and the author can add the marking guidance only prior to the deadline shown. 

Associated_Author.png

Was this article helpful?
0 out of 1 found this helpful

Articles in this section