Posted on:

Evidence

Why should assessment of teachers on placement be based on evidence?

Graduate Teachers need to provide evidence of successful practical experience to secure employment

Assessment of learners enrolled in pre-service teacher education programs is high stakes. Low achievement or failure limits or denies access to future employment in a chosen career.

Accrediting and employing authorities want to base their decisions on evidence

Accrediting and employing authorities rely heavily on the school-based (placement) assessments when making decisions about who to employ.

Accrediting and employing authorities need to know that the grades given to all pre-service teachers during their placements are:

  • comparable
  • accurate
  • provide a basis for reasonable predictions of future independent performance and development as a professional teacher

Grades on each prac should accurately reflect the pre-service teacher’s stage of development (i.e., first prac, middle prac and final prac).

Grades on the final placement should reflect graduate teachers’ levels of achievement on the same criteria, that is all relevant knowledge and skills as specified in the Australian Professional Standards for Teachers – Graduate Teachers. The grades should also reflect the evidence-based potential for the graduate to transfer these to proficient performances in the workplace.

Grades will be comparable, accurate, and have predictive validity if the assessors have based them on the same criteria and if the same evidence has been used to decide:

  • whether each assessment criterion has been met
  • the quality of achievement on each assessment criterion.

School-based teacher educators need to ensure that their assessments are valid and reliable

Reliable and valid assessment of pre-service teachers’ level of achievement against Australian Professional Standards for Graduate Teachers will rely on assessors’ ability to:

  • ensure alignment between criteria specified in the Australian Professional Standards – Graduate Teachers, and what they have taught their pre-service teacher and enabled them to practise
  • select appropriate types of evidence to use as the basis for monitoring progress and making assessment decisions
  • provide opportunities (via explanation, demonstration, guided practice, and formative assessment followed by effective evidence-based elaboration feedback that specifies how to correct or improve) for the learner to develop a clear understanding of the types of evidence that will be used as the basis for assessment decisions including examples of what this evidence will look like in practice
  • provide opportunities for the learner to demonstrate the required evidence on multiple occasions in varied contexts and levels of complexity
  • ensure that assessment judgements are based only on the pre-specified evidence
Activity – What am I doing when I assess students on placement?

Assessments on all placements are not only taken into account but are most heavily weighted when graduating teachers are evaluated for employment. Your assessments really matter.

Your Task

Consider these questions:

Do you focus your assessment on the skills and knowledge you have taught the pre-service teacher during their school-based learning experience?

What weight do you give to creative and innovative practices when assessing their planned lessons and learning activities?

How do you assess their performance if well-planned activities fail in the classroom?

What ‘Standard’ do I use when I grade the performances of pre-service teachers – compare your expectations to the descriptors in the Australian Professional Standards – Graduate Teachers.

What will count as the ‘evidence’ and where do I find that?

The word ‘evidence’ carries connotations of objectivity, factuality, something concrete that can form the basis for correct judgement. We have to add qualifiers to the word evidence if we want it to mean anything other than that; for instance, we talk about inconclusive evidence, unreliable evidence, circumstantial evidence or false evidence if it is not something that will form the basis for accurate and informed judgements.

diagram

National Professional Standards for Teachers

Evidence must be specified in practical performance contexts such as teaching, because it comprises a collection of observable actions and/or products that taken together provide proof/verification of some more abstract state such as:

  • Knowing and understanding
  • Being skillful
  • Learning
  • Ability to think
  • Ability to transform knowledge into informed action.

Evidence consists of demonstrated knowledge, understanding, skills or processes. The evidence might be observable actions, sequence of actions, performances, products, behaviours, and/or an ability to provide accurate information about and/or participate in an informed discussion about theories that are being applied in classroom practices. For instance, evidence that might count as achievement on Graduate Standard 1.2 Understand how students learn Demonstrate knowledge and understanding of research into how students learn and the implications for teaching, could include:

diagram

Activity – Defining the evidence

Australian Professional Standards for Teachers descriptors are very broad and will need to be broken down to identify the specific, skills, knowledge, processes and products that need to be achieved.

For this activity we will explore Standard 3 Focus 3.2 Plan, structure and sequence learning programs: Plan lesson sequences using knowledge of student learning, content and effective teaching strategies. (Please view Australian Professional Standards for Teachers)

It covers the following areas of knowledge and applications:

  1. knowledge of the learning program that forms the context for the lesson
  2. knowledge of theories of student learning
  3. knowledge of teaching strategies associated with a chosen theory
  4. knowledge of and ability to plan using a 3 phase lesson sequencing format – able to plan and identify orientation – enhancing – synthesising phases
  5. knowledge of content to be taught,
  6. ability to produce an explicit written lesson plan program using a given template
  7. ability to identify and plan for aspects of the learning program that will be consolidated and which will be new to students in this lesson

Note that Standard descriptor 3.2 does not include reference to implementation of a plan.

Your Task

Select from the list provided in Possible Evidence for Judging Achievement on Graduate Standard 3 to develop a list of evidence that you would use as a basis for assessing a pre-service teacher on these criteria during a first and a final placement.

Criteria First Placement Final Placement
knowledge of the learning program that forms the context for the lesson    
knowledge of theories of student learning    
knowledge of teaching strategies associated with a chosen theory    
knowledge of and ability to plan using a 3 phase lesson sequencing format – able to plan and identify orientation – enhancing – synthesising phases    
knowledge of content to be taught    
ability to produce an explicit written lesson plan program using a given template    
ability to identify and plan for aspects of the learning program that will be consolidated and which will be new to students in this lesson    

Reflective Questions:

  1. What evidence will you use to decide whether the chosen learning theory, planned sequence and teaching strategies will be effective?
  2. What will ensure that the unquantified element “effectiveness” doesn’t invalidate the assessment?
  3. Graduate Standard 33.4 Select and Use resources: Demonstrate knowledge of a range of resources, including ICT, that engage students in their learning, includes an unquantified term “a range” when referring to the resources used by the pre-service teacher. What constitutes “a range”? Would you specify a specific number or number of types of resources? Would you be able to judge this on the evidence provided in a single lesson?
  4. Imagine you were to be assessed on Professional Standard 3.2 either as a Proficient Teacher or as a Highly Accomplished Teacher; what evidence would you hope that an assessor would take into consideration when judging your performance as a teacher? Show how performance at these Standards would be different from the evidence looked for in the performance of a pre-service teacher on their final placement.
Final Placement Proficient Teacher Highly Accomplished Teacher
     
     
     
     
     
     
     

Download the Defining the evidence activity as a pdf to use with your staff.

Special features of evidence of an Australian Professional Standard.

Evidence of a Australian Professional Standard requires consensus among all stakeholders – if there is no consensus regarding the types of evidence that can count as having achieved the Standard, assessment of performance across the nation will not deliver the Standard.

Activity – Confirming that there is a Standard

Your Task

In the activity, Defining the Evidence, in the section, “What will count as the ‘evidence’ and where do I find that?”, you developed a list of evidence you would use to assess 2 placement students, one on their first placement and another on their final placement on Standard 3 Focus 3.2 Plan, structure and sequence learning programs: Plan lesson sequences using knowledge of student learning, content and effective teaching strategies.

Compare your lists of criteria and evidence with other colleagues.

Is it possible to reach consensus regarding the types of evidence you as a group would consider essential if a pre-service teacher at different stages of their learning program is to be judged as satisfactory on Standard 3.2?

Reflective Questions:

  1. What are the barriers to consensus?
  2. What resources and dispositions facilitate the establishment of consensus?
What evidence should school-based assessors look for?

Validity and reliability of judgements about learning and achievement rely on the types of evidence used as the basis for those judgements. The evidence looked for should constitute the achievement being assessed. According to Sadler (2009, p. 4), achievement is “the attainment of an identifiable level of knowledge or skill as determined through evaluating performances on assessment tasks, or through observation of relevant behaviours in specified settings.”

School-based assessors should look for evidence that the pre-service teacher understands and can apply and specifically those listed in the handbook provided by the relevant university. This generally includes:

  • the skills, knowledge, processes and products they have taught them to use during the school-based learning experience
  • the skills, knowledge, processes and products they have provided the pre-service teacher with opportunities to practise
  • the skills, knowledge, processes and products they have formatively assessed and followed up with elaboration feedback
  • the skills, knowledge, processes and products they have agreed will be looked for during each particular assessment performance
  • the provision of evidence as described in discussions between the assessor and pre-service teacher prior to the assessment. For instance, to assess whether a student teacher can “plan lesson sequences using knowledge of student learning, content and effective teaching strategies”, assessors will usually include written and spoken as well as performance evidence. They may specify that the written lesson plan will form a major part of the evidence. Discussion of the learning theory underpinning choices of sequence and design of teaching strategies and learning activities may form part of or used to confirm evidence of understanding of learning theories

It is very important that achievement is assessed only on the specified criteria and agreed on evidence schedule. Extra ‘credits’ or ‘debits’ should not be given for elements not included in the criteria and evidence schedule. For instance, use of or failure to use an assessor’s personally preferred lesson plan format that is unfamiliar to the pre-service teacher, unrealistic expectations of on-site time commitments, marking, teaching and preparation loads, cannot attract “bestowed credits or debits” (Sadler 2009, p. 6) without reducing grade integrity. According to Sadler (2009), there are three conditions of grade integrity and all three pertain to evidence:

  1. the assessment evidence should be “of a logically legitimate type”
  2. the assessment evidence should be of sufficient scope and soundness to allow a strong inference to be drawn about the underlying achievement
  3. decisions on grades should be made by evaluating the quality of student work against fixed external anchor points, that is, against pre-specified standards and evidence (Sadler 1987)

These factors are all important for valid and reliable assessment for learning and of learning in the school-based learning contexts. If assessments are to have predictive validity, that is, they will form a basis for predicting the quality of performance in other classroom contexts, all teacher-assessors must have a shared understanding of the evidence that will count as at standard, below standard and above standard performance at particular stages of a pre-service teacher’s learning career. They must also have an ability to apply these understandings when making assessment judgements.

Download the Elaboration Feedback.pdf

Activity – Looking for the evidence

Your Task:

View either the primary video and/or written resources or secondary video and/or written resources, for this activity.

Primary Teacher Resources:

Secondary Teacher Resources:

With reference to the Possible Evidence for Judging Achievement on Graduate Standard 3, identify the evidence available in the written lesson plans and/or the video clips of the pre-service teacher in action on their final school-based learning experience.

Is there enough evidence in the two forms of performance, (i.e., the written lesson plan, and/or the observation of the lesson) for you to make a valid judgement about learning and achievement on Standard 3.2?

What other evidence source would inform your assessment?

Reflective Questions:

  1. How many types of evidence would you assess a pre-service teacher’s performance on at any one time?
  2. There is a difference between looking for and seeing. As assessors, you are required to stick to the criteria and to look only for specific types of learning even though you see other important areas of knowledge and skills during an assessed performance. How difficult is it to make valid judgements on say, the sequencing of learning activities when the learning activities themselves are evidently ineffective?
  3. How would you manage the feedback in such cases?
What evidence should pre-service teachers provide?

The pre-service teacher should provide clear evidence that s/he understands and can apply:

  • the skills, knowledge, processes and products they have learned to use during the school-based learning experience
  • the skills, knowledge, processes and products they have practised
  • all points noted in elaboration feedback
  • provided by their school-based teacher educator
  • the skills, knowledge, processes and products they have agreed will be looked for during each particular assessment performance
Activity – Making sure I demonstrate the evidence

Your Task

Your placement supervisor/mentor is coming to assess you on Standard 3.2 based on a lesson you have been asked to plan and to implement, what would you like them to see and/or take away as evidence of your skills, knowledge?

Note: Australian Professional Standards for Teachers descriptors are very broad and will need to be broken down to identify the specific, skills, knowledge, processes and products that need to be achieved.

For instance, Standard 3 Focus 3.2 Plan, structure and sequence learning programs: Plan lesson sequences using knowledge of student learning, content and effective teaching strategies, covers the following areas of knowledge and applications:

  1. knowledge of the learning program that forms the context for the lesson,
  2. knowledge of theories of student learning,
  3. knowledge of teaching strategies associated with a chosen theory,
  4. knowledge of and ability to plan using a 3 phase lesson sequencing format – able to plan and identify orientation – enhancing – synthesising phases,
  5. knowledge of content to be taught,
  6. ability to produce an explicit written lesson plan program using a given template,
  7. ability to identify and plan for aspects of the learning program that will be consolidated and which will be new to students in this lesson.

Note that Standard descriptor 3.2 does not include reference to implementation of a plan.

Key Readings

Sadler, D.R (2009). Fidelity as a precondition for integrity in grading academic achievement. Assessment & Evaluation in Higher Education, (36)6, 727-743.