From the overview in the basic settings of a flow, you have the option to adjust various settings surrounding the Marking and review process of flows.
There are various functionalities to configure the general marking process for a flow. Not all functionalities are available on all flow types.
Depending on your licence settings, not all functionalities may be available in your Marking and Review overview.
Refer to this article to learn more about both marking methods.
To define a specific publication date, click on the pencil icon against the Show Final Grades setting. This will allow you to enter your preferred publication date when grades and feedback will be released.
If a publication date is chosen, the date and time when the grade will be released will be shown to participants on the flow page.
You can find more detailed information on this functionality in the article Item-based Marking on FLOWmulti
Please note that if rubrics-based marking is enabled, FLOWmulti flows cannot be assessed via item-based marking.
If enabled on your licence, you can allow participants to request a justification of their final grade from their assessor when the marking period has ended.
You can decide whether participants' justification requests should be assigned to internal assessors (if available on the flow). If a flow has more than one internal assessor you need to assign the justification request to a assessor. If not enabled, justification requests will be allocated to external assessors.
Finally, you can allow participants to request a justification within WISEflow (as opposed to only through an external system and the API).
Only on FLOWassign and FLOWlock: If Turnitin or Ouriginal have been integrated on your WISEflow licence, you can decide whether participants' submissions should be checked for containing similar content to other electronic material. When enabled, all papers, which have been submitted, will be sent to Turnitin/Ouriginal service automatically two hours after the (individual) participant period ends. The papers will be tested to see if they contain passages of text that also appear elsewhere (internet, library databases, papers submitted by others etc.). A similarity report will be available to assessors after a few hours.
At the licence level, the default sharing status of all annotations, comments and rubrics from assessors can be selected from Private, Participant, Co-assessors and All. This can be over-ridden at the flow level to the desired default setting. Changing this setting will only affect what the assessor sees as the default sharing status for marking to be completed. Any marked work already completed will not be affected by this change.
- Manage assessor types for the flow: find out more in our article Licenced Defined Assessor Types - Managing.
-
Decide whether assessors are allocated individually or as an assessor group to participants. If the distribution type is changed after assessors have already been allocated to participants, the assessor allocation will be reset.
You cannot manually allocate an individual assessor to participants if assessor groups has been chosen as the distribution type. You will then have to create a assessor group for this individual assessor.
-
Add the assessors needed for the flow by pressing the green Add assessor button, search the name of the user(s) you want to add to the flow and choose which assessor type they should be assigned. Find out more about assessor types in our article Licence Defined Assessor Types - Managing.
-
There is no maximum limit of assessors that can be added to a flow. We do, however, advise you to divide large exams into smaller sub-flows to maintain a better overview. For example, if you have to manage an exam with 30 assessors, we suggest that you divide it into two flows with 15 assessors on each flow.
It is possible to add an assessor right up until the marking period has ended. After this, the marking period can be extended and new assessors can be added to the flow again. Assessors do not need to be added to the flow before the flow is activated or before the marking period begins. Assessors added to the flow after activation will not receive the You have been enrolled to flow notification.
- Assessor types: This column shows the assessor type each assessor has been assigned. Find out more in our article Licenced Defined Assessor Types - Managing.
- Privileges: This column displays the privileges granted in accordance with the assessor type.
- Rubric: Shows any rubrics that have been associated with an author/author group.
- Assessor groups: This column is only available when the distribution has changed to assessor groups. It will contain the names of the group the assessor is part of.
- Edit: Change whether a assessor should be defined as internal or external.
- Delete: Remove the assessor from the flow.
-
Assessor groups (only if the distribution type is assessor groups): Create and/or edit assessor groups. You can quickly allocate
all participants to or remove
all participants from an author group.
-
Quick Allocation lets you allocate assessors/assessor groups either freely or by work distribution. Allocating by work distribution lets you split the assessors/groups evenly between the participants, keeping overlaps to a minimum. Allocating freely will plot the assessors/groups randomly where overlaps can happen meaning that multiple assessors/groups can assess single participants.
-
With the allocation matrix, you can assign each assessor/group to the participants you want to be assigned. Click in the box corresponding to allocation you wish to make. Alternatively, hold A to add assessors/groups while moving over the boxes next to participants to add multiple assessors/groups. Hold D to delete in the same manner.
- As a third option, you can assign assessors/groups to individual participants by clicking directly in the participant list, where you click on the assessors/groups you want to allocate to that specific participant.
Grade Audit Trail
Actions made by assessors, reviewers and managers are recorded in the Grade Audit Trail, accessed by managers via the Work Distribution panel of the Manager Assessors page.
The Grade Audit Trail column can be added by managing the columns displayed:
The Grade Audit Trail will display grades submitted by assessors along with any comments the assessors have made at the time of submission. The audit trail will also show Alternative Grades, conflicting grades from co-assessors, grades withdrawn by assessors (before being finalised) and those unlocked by managers or reviewers.
Any actions completed prior to the release of this new feature on Friday 8 October 2022 will not be recorded in the Grade Audit Trail.
Add the reviewers needed for the flow by pressing the green Add reviewer button and then search for the users you want to add to the flow.
When adding new reviewers, you can decide which privileges they should receive:
Decide final assessment |
|
Allocate to all |
|
The reviewer will be able to decide the final grade for participants on the flow. Only available if the procedure for submitting the final assessment has been set to Approval by reviewer |
Tick this box if you want the reviewer to see the assessment information of the participants. | Quickly allocate the reviewer to all the participants. You cannot manually allocate a reviewer to participants if the privilege Allocated to All has been checked. |
- You can remove or add columns of information of the reviewers by pressing this button and checking the columns you want to be displayed
- Shows the number and percentage of assigned participants to a reviewer.
- Quickly remove the reviewer from all allocated participants. Option only available if privilege Allocated to all is NOT checked.
- Quickly allocate the reviewer to all participants. Option only available if privilege Allocated to all is NOT checked.
- Edit the privileges of the reviewer, see above for more information.
- Remove a reviewer from the flow, alongside all of its allocations, comments and annotations.
You have 3 different options when choosing how to allocate your reviewers to the participants.
- By pressing the quick allocation button, you will gain access to a menu that can easily distribute your reviewers randomly amongst the participants based on the percentage you want each reviewer to have.
Allocating freely will plot the reviewers randomly, where overlaps can happen freely. Allocate as a work distribution lets you split the reviewers evenly between the participants, keeping overlaps to a minimum. -
With the allocation matrix, you can assign each reviewer to individual participants and get an overview of which reviewers are assigned to which participant. Hold A to add reviewers while moving over the boxes next to participants to quickly add multiple reviewers. Hold D to delete in the same manner. You can select identifiers to display reviewers (i.e. by name, email, etc.)
-
Individual participant in the participant list: last option is to click the participants directly in the participant list, where you click on the reviewers you want to allocate to that specific participant.
Rubrics are published by authors and can be attached to the flow by you or authors if they are associated with the flow. It is possible to attach several different rubrics to one flow. You can add and manage rubrics on a new subpage. More information can be found in the article Setting Up Rubrics and Using Them for Assessments.
- Marking guidance: Files and links can be made available for assessors and/or reviewers to aid the assessment process. Authors added to the flow may also manage the marking guidance.
- Other material: Files can only be managed by managers on the flow and will be displayed as additional marking material to assessors and/or reviewers.
Choose files and/or links to be handed out to the assessors and reviewers on the flow. When you have added the material you can also choose to which roles these files should be available.
When the material has been added, the labels indicate to whom the material is available. Clicking the small icons you have the following options:
| Change to whom material is made available | Edit filename | Preview file | Download file | Delete file |
How the material is displayed to assessors and reviewers
When a assessor opens the flow, the material is visible on the flow page under the marking tool and the mark overview.
When a reviewer opens the flow, the marking guidance is visible under Supporting material.
The marking guidance can be added by assigned authors on the flow. If an author has been added to the flow the flow will be visible under Associated flows and the author can add the marking guidance only prior to the deadline shown.