Before focusing on AI related vulnerabilities and measures to secure assessment, we should first ensure assessments are fit for purpose and aligned with learning outcomes. We look at the steps to create assessments that reflect student capability.
AI is a catalyst for assessment redesign
AI represents an urgent catalyst for change. It does not just influence how student learning can be assessed, it also influences what is worth assessing and, consequently, what, and how students learn” (Lodge et al., 2023, p. 2).
Assessments play a critical role in fostering learning and the certification of students, and powerfully frames how students learn and what they achieve (Boud & Associates, 2010). In a course, there may be assessments designed to be ‘products of learning’, as well as opportunities ‘for learning’ and ‘as learning’.
Pen and paper not the solution
In recent advice from TEQSA on short term actions, it is advised for institutions to resist the urge to return entirely to conventional assessment tasks like pen-and-paper exams as these frequently fail to evaluate the entire range of knowledge, skills, and abilities that we require students to demonstrate (Lodge et al., 2024, p. 2).
Student capability creates assessment validity
This supports the focus on assessment validity, which means an assessment is only valid if it accurately represents a student’s actual capability (Dawson et al., 2024). In the context of AI, the use of AI becomes unacceptable when we cannot assure students’ capability.
Assessments vulnerable to AI
Traditional assessments like essay tasks, multiple-choice quizzes, and summarisation tasks are particularly vulnerable to the capabilities of generative AI and may not always be the best measures of student achievement (AAIN Generative AI Working Group, 2023).
This does not mean essays should not be used as assessment, but they may not be appropriate for all courses, particularly if forming arguments and writing skills are not the intended learning outcomes.
Another example is the use of remote-proctored exams, which may seem to be secure but can harm validity by the exclusionary nature of the technology and may fail to measure the actual skills and knowledge of students (Dawson, 2023).
Questions for checking assessment validity
The first step to address the challenges to assessments is to determine their validity and redesign them, if needed. Any consideration of assessment security should support, not undermine, assessment validity.
Here are some questions to help educators determine assessment validity.
- Does the assessment align with the learning outcomes?
- Are all key concepts, knowledge and skills assessed and appropriate for the course level?
- Is the assessment (e.g., group work, essay) suitable for evaluating the intended skills or knowledge?
- Does the format allow students to demonstrate their capabilities and understanding?
- Does the assessment evaluate the appropriate level of cognitive thinking?
- Have you considered how AI may be used in completing the assessment, and does the design ensure it still measures students’ knowledge and skills effectively?