Edition 17, August-September 2003

Assessment validation

The Australian National Training Authority has published a kit, Learning and Assessment Strategies, containing instructions and templates to assist practitioners in working with industry when developing and validating assessment strategies.

What is assessment validation?

Assessment validation involves 'reviewing, comparing and evaluating assessment processes, tools and evidence contributing to judgments made by a range of assessors against the same standards' (Learning and Assessment Strategies Resource Guide, Australian National Training Authority, p.101).

Why is it so important?

Standard 9.2 of the Australian Quality Training Framework Standards for Registered Training Organisations states that a registered training organisation must validate its assessment strategies at least annually. To comply with this standard, registered training organisations must also 'document any action taken to improve the quality and consistency of assessment'.

What are the benefits?

Assessment validation facilitates processes leading to consistent and valid assessment. In particular, validation activities:

  • ensure that assessment strategies meet the needs of clients;
  • facilitate the professional development of assessors;
  • enable enterprises and other stakeholders to contribute to assessment processes;
  • provide a means of gathering feedback and identifying ways of improving assessment processes;
  • facilitate consistent interpretation of competency standards;
  • foster the development of informal networks and provide assessors with access to up-to-date information about what is happening in their industry;
  • help assessors working across the industry to apply consistent standards and make consistent judgements.

These processes build assessors' confidence and industry acceptance of the outcomes of the national training system.

What must be validated?

The following table shows some of the key aspects of assessment strategies that need to be validated.

 

Assessment process

Assessment tools

Evidence

� enrolments
� assessments
� recording
� reporting
� appeals

� specific assessment tasks
� instructions for candidates
� instructions for assessors
� evidence guides, checklists
� assessment criteria
� rules of judgement
� examples of acceptable responses and descriptions to typical competent performance

� third party
� simulation
� sources:
      ­- indirect
      - direct
      - supplementary

Source: On Track: Moving Towards Assessment Validation, TAFE New South Wales, 2002.

Who should be involved?

All assessors who undertake assessment within a registered training organisation should be involved in some form of an assessment validation process.

The footnote to Standard 9.2 of the Standards for Registered Training Organisations states that validation processes 'may be internal processes with stakeholder involvement or external validation with other providers and/or stakeholders'.

Therefore, validation activities may be managed and conducted by an individual registered training organisation with the involvement of key stakeholders from industry.

Alternatively, external validation involves personnel from an industry area or from a number of registered training organisations. It may involve specialist assessors, external validators and representatives of the relevant industry training advisory body, or professional or industry associations and networks.

When should validation take place?

Validation can occur before, during or after assessment takes place. Validation after assessment is the most common method used. Nevertheless, procedures for checking methods, evidence requirements and benchmarks of performance should take place before any assessment is carried out. The implementation of a new or revised training package is a good time to begin the validation process.

What approaches can be used to validate assessment strategies?

Assessment validation strategies that a registered training organisation may implement include the use of:

  • meetings, where groups of assessors have the opportunity to compare and discuss their assessment processes;
  • external assessment panels or teams of assessors;
  • external and written assessment tasks where markers' results are compared statistically;
  • benchmarking exercises with other organisations to compare assessment processes and practices;
  • common assessment tasks used by a number of assessors;
  • a 'lead assessor' to manage or oversee the assessment process;
  • an assessment panel to oversee or monitor the assessment process;
  • a mentoring system for assessors.

This list is by no means exhaustive or mandatory. The Learning and Assessment Strategies Resource Guide provides a description of the range of assessment validation strategies, as well as links to the Training Package Assessment Materials Kit, which contains more detailed information.

Selecting appropriate validation strategies

Many of the activities or practices that assessors already use may form part of an assessment validation strategy. Sometimes these activities just need to be formalised and documented.

Registered training organisations should aim, over time, to implement as many validation approaches as practicable, as part of a continuous improvement process.

Each registered training organisation will therefore need to choose the most appropriate methods of assessment validation based on the following factors:

  • business considerations, such as the organisation's quality systems, strategic planning process, market share, industry sector and issues relating to client satisfaction;
  • resource considerations, including staffing, financial and physical resources as well as time commitments;
  • organisation context, such as the type, size, location and complexity of the registered training organisation and the enterprises and industry involved;
  • regulatory requirements, such as licensing requirements, state or territory legislative requirements and/or registering body requirements (Learning and Assessment Strategies Resource Guide, Australian National Training Authority).

The registered training organisation will also need to focus the validation process on those areas considered to be of highest risk within the area being assessed.

What documentation is required?

Regardless of the assessment validation method used, the registered training organisation must maintain records of the validation processes to satisfy the requirements of Standard 9.2. The forms of evidence that show validation has taken place may include:

  • records of meetings where validation is covered (agendas, minutes, action plans);
  • feedback from clients (student and/or employer satisfaction surveys; notes recording feedback from face-to-face or telephone interviews);
  • position descriptions for specialist or lead assessors;
  • terms of reference for assessment panels;
  • records of benchmark activities;
  • records of professional development activities that focus on improving assessment strategies;
  • examples of materials used in reviewing, comparing and evaluating tools (quality criteria, review checklists).

Again these are only suggested forms of evidence; there are more examples in the Learning and Assessment Strategies Resource Guide.

More information

The Learning and Assessment Strategies kit, including customisable Word templates, can be downloaded at www.anta.gov.au/pubBundle.asp?qsID=28

The publication On track: moving towards assessment validation, produced by the Vocational Education and Assessment Centre, TAFE New South Wales, contains a practical guide and templates to assist assessors in developing an assessment validation strategy. To download a free copy of the publication visit www.veac.org.au and click on the 'research projects' link, or phone 02 8374 5404 to purchase a copy.

For more information contact Anita Roberts, Australian National Training Authority on 03 9630 9853 or email robertsa@anta.gov.au

 

This page was generated on 26 August, 2009