If you would like to get in contact, please send us a message and we’ll be in touch soon.
Making the Most of Assessment Validation
I was reminded the other day that in just under 12 months, all ASQA registered RTOs that were already operating as at 1 January, 2015, should have validated 50% of all training products on their scope of registration.
Why? Well the ‘compliance’ answer is that Clause 1.10 (of the Standards for RTOs 2015) states that ‘each training product is validated at least once every five years, with at least 50% of products validated within the first three years of each five-year cycle’. 1st April 2018 marks three years since RTOs were required to comply fully with the Standards for RTOs 2015.
However, the ‘quality’ answer is that regular Validation of assessment is the way to a more effective assessment system with consistent valid assessment judgements! With both reasons in mind, is it not time for you to review your assessment system and validation schedule and make sure you are on track?
Unfortunately, we find that some RTOs don’t get full value from their Validation activities and opt for an approach where they conduct Validation with a compliance only focus i.e. because they have to do it when approaching audit or other compliance related deadlines.
This blog will take a look at the process of Validation, what it is, some of the excuses we hear from RTOs who fail to systematically validate their assessments and how to implement effective Validation practices in any RTO to contribute to business success.
What is Assessment Validation?
‘Validation is a review of assessment judgements made by your RTO. Validation is generally conducted after assessment is complete. The process must be undertaken in a systematic way. Validation may include engagement with industry to confirm that your RTO’s assessment system:
• produces valid assessment judgements
• ensures graduates have the skills and knowledge required by industry, as expressed in the training package or accredited course.’
(User Guide to the Standards for RTOs 2015)
Note the use of the word ‘systematic’! The quality of the final assessment judgement made in an RTO is the result of a number of factors including the quality of the assessment tools, their content, alignment to Training Package requirements and the extent to which tools and practices adhere to the Principles of Assessment and Rules of Evidence. Regular, and systematic Validation should be used to continuously improve the quality of assessments and of student outcomes. All of this should be documented in your assessment system, starting with a clear assessment policy and clearly documented procedures that make it very clear to everyone in your RTO what they need to do and why systematic review and improvement of your assessment practices is so important!
Why is Assessment Validation so poorly done?
If we assume that achieving quality student outcomes is central to the operations of all RTOs, why is it that we find so many RTOs that neglect to validate their assessments or conduct Validation without any meaningful outcomes? Why does the quality of assessment remain as one of the biggest risks managed by the regulator, across the VET sector?
This is what we find as we work with RTOs:
- Trainers and assessors are not equipped for the Validation process nor are they familiar with the Principles of Assessment or Rules of Evidence.
- Validation is conducted as a tick-a-box exercise with no desire to identify and address gaps in assessments.
- RTO Managers believe that commercial material is compliant with Training Package requirements and so Validation is not needed.
- RTOs struggle to gather their trainers and assessors together to conduct Validation.
- Small RTOs find it difficult to comply with Clause 1.10, particularly the requirement that the trainer/assessor who delivered/assessed the training product cannot conduct the Validation alone nor can they determine the Validation outcome.
Effective Assessment Validation Processes
As mentioned in the definition above ‘Validation is a review of assessment judgements made by your RTO.’ But what does this mean in a practical sense when it comes to designing the Validation process?
Because the Standards specify that there is a minimum number of training products to be validated over time, you are obliged to plan ahead and develop a Validation plan. This is best done as part of your overall quality improvement planning system, so it becomes part of your routine RTO practice and is not considered to be a burdensome add on!
1. What should you be reviewing?
You should be reviewing assessment practices and judgements and your review must be based on evidence collected, retained and recorded within your RTO.
- When reviewing assessment practices, you must look at the assessment evidence gathered from your students, and determine if it is valid, reliable, sufficient, current and authentic (i.e. satisfies the Principles of Assessment). You should determine if assessment tools have produced the intended evidence.
- When reviewing assessment judgements, you should consider whether the learners’ assessment evidence:
- complies with the assessment requirements of the relevant training product
- demonstrates that the assessment was conducted with fairness, flexibility, validity and reliability, and
- is valid, sufficient, authentic and current (i.e. satisfies the Principles of Assessment)
Note: The description of what should be reviewed does not include a breakdown of commercial versus in house developed tools. All assessments should be customised in house to suit the delivery mode and RTO context and should be subject to ongoing Validation and quality improvement.
2. How should you schedule and sample?
When making decisions about scheduling and sampling of assessments remember, you should ensure that:
- the frequency of Validation sessions is helpful for your RTO in terms of improving assessment quality and student outcomes and,
- the sampling of assessments for each session is random and large enough to ensure that the outcomes of the Validation process are representative of the whole set of judgements.
For example, if you are reviewing the assessment judgements made for an entire qualification you should choose at least two units from the qualification. You should also ensure that you review enough student evidence to be confident that the sampled assessments will accurately represent all assessments and produce an accurate Validation outcome.
Several factors should influence the regularity of Validation activities and determination of Validation sample size. These include:
- The number of qualifications (or units of competency) on your scope,
- The frequency of delivery of training for each qualification or unit of competency,
- The numbers of students completing your courses,
- The risk associated with the industry and systemic risk assigned by the regulator (e.g. are you delivering training in security or early childhood education and care?)
- The capability and experience of your trainers and assessors,
- The consistency of outcomes i.e. are the majority of students deemed competent on first attempt or is there wide variation of outcomes.
ASQA provides a Fact Sheet with practical suggestions for Conducting Validation . This Fact Sheet provides suggestions for calculating sample sizes and determining frequency of Validation activities.
3. Who should be involved?
Validation can be undertaken by one person or by a team and the excerpt below from the Fact Sheet explains the required qualifications.
Your RTO must ensure the review process is completed by people who collectively hold:
- vocational competencies and current industry skills relevant to the assessment being validated
- current knowledge and skills in vocational teaching and learning, and
- the TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor).
Validators can be employees of your RTO, or you can seek external validators. For example, you can seek participation in your Validation processes from:
- employers and employer associations
- other industry bodies
- trainers and assessors, or
You could also work with other RTOs—from the same or different industry areas—and collaboratively validate each other’s assessment practices and judgements. The trainer and assessor who delivered/assessed the training product being validated:
- can participate in the Validation process as part of a team
- cannot conduct the Validation on his/her own
- cannot determine the Validation outcome for any assessment judgements they made, and
- cannot be the lead validator in the assessment team.
(ASQA Fact Sheet, Conducting Validation, 13 May 2015).
In small RTOs where there may be only one trainer/assessor for a specific topic area it can be difficult to conduct assessment Validation that results in useful Validation outcomes. However, a panel with the collective range of capabilities/experiences can complete the Validation. So, you should form a team of people to conduct the Validation. This team can include your expert trainer/assessor and other people who collectively ensure your team has:
- appropriate vocational competencies
- current industry skills and knowledge
- an appropriate training and assessment qualification or assessor skill set
- current knowledge and skills in vocational teaching and learning.
As stated above, the trainer/assessor responsible for delivery and assessment can form part of the overall team but should not be involved in determining the final Validation outcomes. Validation of assessment is a helpful way to engage industry representatives in a meaningful way. They experience firsthand, how the students are being prepared for future roles in the industry.
4. How should you use Validation outcomes?
The development and refinement of assessment materials is an ongoing task in an RTO and should be viewed as a quality improvement task. Validation processes can identify issues with any part of the assessment system, the tools, the processes and the outcomes. Linking Validation to ongoing quality improvement enables the RTO to systematically address identified problems. Rectification of problems is subsequently reported through the RTO’s quality processes and documented for both quality and compliance purposes.
Make sure that you keep a record of each Validation session including, when and where it was conducted, who was the lead validator and who else was involved. It is also vital to record what the outcomes were and what changes were made to enable ongoing improvement of assessments.
5. How can you turn a Validation session into an opportunity for professional development?
Validation exercises are also a helpful opportunity to upskill training and assessment staff. An effective way of doing this is to start each Validation session with a professional development session. For example, you could include a session on:
- The Principles of Assessment
- The Rules of Evidence
- The art of assessment tool design and development
- Techniques to conduct assessment on online platforms
- The relevant clauses from the Standards for RTOs 2015
Inclusion of professional development in your Validation sessions helps to ensure that your trainers and assessors are equipped for the task ahead of them and provides a useful refresher for concepts that they are about to apply in the Validation session. It also contributes to building a learning community within your RTO, promotes lifelong learning and encourages the sharing of knowledge across team members.
Not sure what to do next? We can help you with:
Why not contact us on firstname.lastname@example.org and we can help you work your way through the process.
We’d also love to hear what you think of this blog, so feel free to leave your comments below.
We look forward to hearing from you!
Gillian & Karen