One of the most important elements of the WISEflow Onboarding Project is thinking about what it is you would like to achieve by the end of it. The practice of setting clear and definitive goals makes it possible to determine success criteria against which the project can later be evaluated. The obvious and immediate goal for most clients is that they want to run successful digital assessments at their institution, but it's useful to think about how this can this be broken down into more specific and measurable facets.
Success criteria can be defined in two broad areas:
- Project-based
- Deliverables-based
Project-based
Deciding on the practical project goals involves having a defined project scope and setting clear and achievable targets for its operation. WISEflow onboarding projects will sometimes start out with a small pilot involving a handful of programmes, staff and students and some initial scoping questions will need to be addressed, such as:
- Which schools, faculties and departments will be involved?
- Will there be a subset of assessment types trialled (e.g. multiple choice tests, essay-style, alternative assessments)?
- Will different assessment environments need to be included (e.g. on-site, exam hall, remote working, coursework hand-in, locked down, proctored or un-proctored)?
- Are there any time constraints affecting how the project needs to be completed?
Such objectives should be discussed and defined early in the project and you can also refer to our article on selecting assessments for your pilot project for further assistance in this area. Where the scope of the project may develop over time, the success criteria should be revisited to ensure they are still relevant and valid.
Deliverables-based
Whilst the operational aspects of the project might be executed impeccably, to time and cover everything that was originally intended, it is also important to consider whether what the project delivers is aligned with the original expectations and aims. Consider what are the longer-term drivers for exploring the use of digital assessment? Is your aim perhaps to improve the student experience of their assessments? Maybe you want to improve efficiency and reduce the administrative load, or perhaps it was prompted simply by the requirement to adapt to changing circumstances and environments.
Some areas to think about could include:
- How will you evaluate the student view of the digital assessment experience?
- Do you intend for students to be able to work on their own device?
- Will visibility of assessment criteria be increased?
- Are you looking to improve the quality of marking and feedback, and if so how?
- How will you evaluate the staff view of the digital assessment experience?
- Are you aiming for a reduced administrative burden?
- Is there a more effective workflow proposed?
- Do you perhaps intend for staff be able to work more remotely in the future?
- How will the format of your assessments translate to the digital sphere?
- Is there are target for the proportion of digitised assessments you want to achieve?
- How will you engage the different academic groups in the exercise?
- How will your internal infrastructure support the move digital assessment?
- Is the physical and technical infrastructure in place?
Wherever possible the criteria for success should be made both specific and achievable and consideration should be given to what metrics will be used to measure and evaluate the outcome.