Assessment Futures

Many thanks to everyone who attended the workshop to explore the future of digital assessment on the 1 May 2024. I wanted to build on the Future of Assessment 2035 workshop we delivered in June 2022 and use some of the learning and resources from the Imagined Future workshops we are running this year. The event provided us with an opportunity to build some future scenarios of how digital assessment is changing.

The workshop generated lots of ideas so we are collating and analysing the outputs in the Assessment 2035 Miro board. (password: jiscassessment)  This post is a summary of the workshop and I have shared some initial thoughts below.

Several people have already shared their comments on the workshop. Mentions from Dr Miriam Firth; The Uniwise team said “we hope you found it as engaging as we did” and Shane Sutherland who shared live the group’s outputs from the workshop. Rachel Forsyth’s blog captured some key points around a future focus on process rather than product, competence and the application of knowledge. There was an excessive use of post-its – I do so miss the tactile, interactive workshops we ran so often before the pandemic, so no apologies for that.

The workshop

Part one: Trends and challenges

We repeated the activity from two years ago looking at trends and challenges around assessment in response to this statement.

It is 2035 and despite the take-up of digital assessment, many universities have retained traditional exams. Course work assignments are seen as unreliable due to the dominance of contract cheating. Academics are overworked with all the marking and feedback. Employers are not satisfied with students’ employability skills.

What went wrong? What challenges and drivers got us here?

The main themes that emerged were:

• A resistance to change assessment approaches from academics, perhaps driven by a lack of time and resource, digital skills, assessment design skills and the choice of digital assessment products.
• A need to rethink how and why we assess students, and to recognise employability skills as well as subject expertise.
• The impact and opportunities of generative AI to support assessment design, provide more personalised and adaptive learning pathways and support generation of student feedback.
• Academic integrity challenges around the use of generative AI by students, use of chat tools increasing student collusion on assessments which are driving a return to in-person analogue exams and senior managers to look at e-Proctoring.
• An increase in regulation for example around grade inflation.
• The challenges of scaling digital solutions, for example, in person digital exams.

The outputs from this activity and some existing challenge cards from our imagined futures work were then used within the next activity.

Part two: What if scenarios

In the next stage teams had a choice of eight assessment What if scenarios, some of them based in the Futures Scenarios from the last workshop. Each team picked challenge cards from the first activity to help them imagine an assessment future.

What if..

• we used digital to scale an existing innovative assessment format and approaches.
• digital was used to enable more authentic/real world assessments.
• we only had portfolio and peer assessment.
• assessments are personalised – individuals choose the method/format used to demonstrate their learning.
• we reduced the number of assessments or had no assessments.
• most assessments were digital artifacts or performances.
• we only assessed skills instead of knowledge.
• we did something very different.

Part 3: Future scenarios

We had seven groups that generated ideas for assessment futures around six of the what if scenarios. Here is an initial summary.

Group 1 – What if we used digital to scale an existing innovative assessment format and approaches.

Assessment Future title: History is now. Change is convenient.

Description: The idea was that we create an assessment management process today that supports any type of assessment, makes best use of AI to improve process efficiencies in administration, marking and feedback. Then tomorrow we will be able to cope with any scale and model of assessment, as long as we agree on a few standards and approaches.

Group 2 – What if digital was used to enable more authentic/real world assessments.

Assessment Future title: APLWOS – Authentic programme level assessments with on-demand skills.

Description: Imagine every course has a programme level assessment that is authentic/work based. We used GenAI to enable the easy creation of scenarios and simulations that enable authentic assessments for more courses. Students had to demonstrate the process rather than the product.

Group 3 – What if assessments are personalised – individuals choose the method/format used to demonstrate their learning.

Assessment Future title: Personalised assessments for large cohorts.

Description: The data from each students formative and modules level assessments was used to create a personalised and adaptive end of year/course summative assessment that validated their learning and level. A sort of personalised digital viva for every student but it was auto generated and marked.

Group 4 – What if assessments are personalised – individuals choose the method/format used to demonstrate their learning.

Assessment Future title: Student choice of assessment type.

Description: Every student could select the mode, format and method of assessment whilst also requiring them to demonstrate a range of skills. Students would need access to training and support for each assessment type.

Group 5 – What if most assessments were digital artifacts or performances.

Assessment Future title: Performance defined as the production of a visible, measurable product with a process of creation.

Description: Assessment becomes more iterative with a focus on the process of creation. Knowledge and skills are surfaced and made visible to multiple stakeholders including students. Marking will include process based assessment metrics and mark schemes linked to real world doing.

Group 6 – What if we only assessed skills instead of knowledge.

Assessment Future title: Only assess skills, with a pass/fail criteria.

Description: Focus on the skills and application of knowledge through a set of assessment tasks. Assessments become digital portfolio based to demonstrate skills with pass/fail on every task. Students could be given multiple attempts to pass. All tasks would be mapped to learning outcomes. Digital credentials would be linked to the completion of each task. Challenges include cultural change for staff and students, agreement on what is a pass (having clear criteria) and would this be the end of degree classifications?

Group 7 – What if we did something very different.

Assessment Future title: All assessments are creative and immersive.

Description: Students are offered a diverse library of creative assessments including roleplay games based scenarios, VR/AR escape rooms where students solve problems to progress, and experimental artifact production based on research techniques. The assessment would be a combination of the skills (process) and/or the product.

Next steps

Finally we asked for ideas on the next steps. This generated more than I expected so I have tried to summarise below.

Support assessment transformation and continue engagement

A need to address the big challenges around changing mindsets that are making change difficult that were identified in the first activity. A suggestion to bring together other groups to further explore these challenges and look for some calls to action. A plea to keep experimenting and accept that innovation is an iterative process. Take the activities back to your own institutions and try them with your own staff.

Continue to innovate

Think radical transformation of assessment. The Higher Education sector needs to take collective action to move forward and find an unified approach. Tackle one big challenge such as inclusive assessment or the potential of using AI with adaptive learning or gamification.

Rethink the degree

There was a suggestion that to transform assessment we need to first recognise that the purpose of a degree has changed. Consider how it relates to employability skills and the relevance of degree classifications.

AI and assessment

Recognise the future role of AI and see the opportunities it can bring to transforming assessment. Whilst ensuring that AI marking and feedback do not remove the links with learning and teaching.

Assessment Tools

Institutions should work together to engage suppliers to develop better tools that meet the needs of UK assessment practices. Explore new technologies and share evaluations to allow other to make informed choices.

Finally

Some random observations from the outputs.

1. Digital has a role in all assessments, whether in the management, delivery, marking or feedback – so do we need to continue to talk about digital assessment as if it is something separate.
2. A wider team engagement benefits assessment design and also delivery including registry, IT services, estates, academics etc.
3. Employability skills and engagement with industry/employers is common practice and increasingly relevant.
4. Consider assessing the process and skills separately to the product. It is too easy now to produce a product (using AI or essay mills) without demonstrating the skills.
5. There needs to be be parity of use of AI by staff and students. If staff use it they will expect they can also.
6. Students needs support with new or unfamiliar assessment types and products (such as digital media) so we need to provide training when it is needed.

We’d welcome further thoughts and feedback, so please contact innovation@jisc.ac.uk.