Annotations are an important part of the marking and feedback process in WISEFlow, allowing assessors to provide focused feedback linked to specific locations in a piece of submitted work. FLOWassign and FLOWlock support inline annotation where comments and markup can be overlaid on the submission and FLOWmulti supports the association of annotations with specific items of the test.
For FLOWhandin (supporting alternative file types) or FLOWoral / FLOWattend (where no artefact is submitted) we recommend making use of other marking and feedback tools available in WISEflow, such as rubrics.
The FLOWassign / FLOWlock marking tool offers three different inline annotation options; a text highlighting tool, a stamp tool and a freehand drawing tool.
Click the pen highlighter tool from the lower-left floating menu. Select the area in the text that you wish to highlight and click the tick (or ALT+1 keyboard shortcut) to confirm and open up the annotation editor window.
The text highlighting tool cannot be used on pictures in a PDF. Please use either the stamp or freehand drawing tool to annotate on pictures in a PDF document (see below for more information).
Click the stamp tool from the lower-left floating menu (or ALT+2 keyboard shortcut), then click anywhere in the document that you would like to place the stamp to open up the annotation editor window.
Click the freehand drawing tool from the lower-left floating menu (or ALT+3 keyboard shortcut) and use your cursor to draw within the document. Click the tick to confirm and open up the annotation editor window.
Please note that colours, highlighting formats and stamp shapes can be changed and customised from within the annotation window.
The annotation editor window
The annotation window associated with all three annotation tools is shown below.
- Jump to the area in the document with which the annotation is associated
- Close the annotation editor window
- Text formatting tools
- Commenting area
- Change the colour of the comment box and overlay on the document
- Choose whether the document overlay displays as a highlight, strikethrough or underline (where the stamp tool is used this item will allow you to select the shape of the stamp)
- Associate marks with annotation (see next section of this article)
- Sharing settings (see final part of this article)
- Delete the annotation
Please note that there is no save button for annotations. Annotations are automatically saved.
FLOWmulti supports annotations being associated with particular items of the test. To add an annotation to an item simply click the green plus symbol at the top of the item.
The Annotation Editor Window
This will open the annotation editing window which looks like the image below.
- Jump to item with which the annotation is associated
- Close the annotation editor window
- Text formatting tools
- Commenting area
- Award marks to the item (see next section of this article)
- Sharing settings (see last section of this article)
- Delete annotation
Please note that there is no save button for annotations. Annotations are automatically saved.
It is possible to assign point values to annotations in both FLOWassign / FLOWlock and in FLOWmulti, though they work in slightly different ways. Each process is detailed under the respective tabs below.
In FLOWassign and FLOWlock the use of point values associated with annotations is entirely optional and depends largely on the marking and grading method you prefer (or that which has been specified by the flow manager). For example, if rubrics are in use it may be the case that all scoring information is captured within the structured confines of the rubric and therefore any additional inline annotations are purely for feedback purposes. If, however, assessors are required to make a more holistic judgement on the submission then recording where credit is awarded (or subtracted) by associating it with individual observations can be a helpful tool to assist with reaching an eventual grading outcome.
To associate a numerical value with an annotation simply add the number in the points box in the annotation window. Note that this may also be a negative value.
WISEflow will keep a running tally of all the marks you have awarded by using comments and this is displayed at the top of the marking tool window.
Once marking has been completed it is also possible to use WISEflow's grade scaler to transform the marks awarded to fit a final grading scale. For more info on using this feature please see the article on grade scaling.
As FLOWmulti is able to support assessments that combine automatically validated MCQ-style questions with open-ended questions that require a human marking intervention, the scoring and grading tools work in a slightly different way to those in FLOWassign / FLOWlock. As discussed earlier in this article, annotations in FLOWmulti are associated with specific items of the test, and the same is also true of any point values associated with them.
To award marks on an item add the number to the points box within the annotation editor window. In the below example 12 points have been added to an open-ended essay-style question.
Note that this is recorded in the fields at the top of the item, where no points (-/15) were able to be awarded by auto-validation, but 12 points were awarded by the assessor which gives 12 marks in total for the item.
Equally, automatically validated questions can receive additional credit by using the same method. In the below example 5/5 points have been awarded by automatic validation, but the assessor has manually added 3 points which gives a total of 8 marks for the item.
The running total is also captured at the top of the marking tool window. Continuing the example, 69 of the available 100 points have been awarded by auto-validation and the assessor has added 15 points worth of additional credit making a total score of 84 marks.
Please note that using this freeform method of assessing a FLOWmulti will allow the award of additional credit to exceed the number of marks originally specified by the test author as being available at both the item level and the test level. To impose a more structured set of rules and limits on the number of available points, please see the article on item-based marking.
Sharing on Individual Annotations
There are four possible options for sharing annotations:
- Private: Comments and annotations are visible only to the assessor.
- Share with participant: Comments and annotations are visible to the assessor and the participant (upon the release of grades and feedback) only.
- Share with co-assessor: Comments and annotations are visible to the assessor and any co-assessors that are also assigned to the same participant.
- Share with all: Comments and annotations are available to the assessor, and co-assessors that are also assigned to the same participant and the participant (upon the release of grades and feedback).
These options can be configured per annotation in order to offer maximum flexibility to assessors. To change the setting on a single annotation click the dropdown menu and select the groups you would like to share the annotation with.
Please note that the default WISEflow behaviour is that comments are private unless the assessor decides to share them. However, it is possible to configure this at an individual licence level to make the default behaviour be any of the four listed options. So, for example, if the default behaviour is that annotations are shared with all, assessors would need to deselect the groups with whom they did not want to share the annotation. Please see the article on licence settings for more information.
Sharing Annotations in Bulk
As well as having control over the sharing settings at an individual annotation level, it is also possible to change the setting for all annotations in bulk. The below examples shows 3 annotations, each of which are shared by default with the participant.
To make a bulk adjustment, first, click the blue pen icon shown above. This will create a column of checkboxes, one against each annotation.
Check the box in the column header to select all rows then click the Actions dropdown. From here you have the same set of options seen at the individual comment level, only now making a change will apply it to all the selected rows.
