Measuring impact of CPD

Why consider impact?

Since the late 1980s, educational development provision has expanded rapidly in universities across the UK and beyond, to the extent that virtually all higher education institutions offer formal and informal CPD, ranging from postgraduate courses (PgCerts, masters degrees, PhDs) to HEA accredited provision (professional conversations, reflective portfolios) to less formalised activities (networks, workshops, conferences, consultations). Alongside such growth and diversity, there is increased interest in understanding how CPD influences teaching and student learning (Brew, 2007; Devlin, 2008; Gosling 2008; Chalmers & Di, 2015).

The broad questions that might be considered are:

  • What is the effect on teachers who participate in CPD – both in the short, medium and long-term?
  • How does engagement with CPD have an impact on student learning?
  • What is the effect on (and of) the institution generally?

What do we mean by impact?

Impact can be interpreted differently and usually depends on the context in which it is being considered. The types of indicators that have appeared in the literature on impact, in the context of CPD in education include:

  • teacher satisfaction with CPD courses and events
  • teacher self-efficacy – confidence
  • teacher enhanced ‘skills’ (e.g. lecturing, giving feedback, using specific innovations or working with particular subject-based concepts or materials)
  • teacher success in receiving teaching-related grants and awards
  • teacher engagement in scholarship and research
  • teacher reflection and development over time (often after undertaking a CPD course or similar)
  • teaching attainment of departments and faculties
  • curriculum enhancements as a result of CPD
  • enhanced use of peer networks
  • institutional context for teaching enhancement:
  • policies, strategies, hiring processes in relation to teaching
  • opportunities for reward and encouragement
  • student satisfaction with teaching
  • student learning:
  • student orientation to studying
  • student ability to demonstrate critical thinking
  • student perception of their learning
  • student achievement​.

For more information about how to monitor and evaluate studen​t satisfaction and learning gain please read 'Measuring the impact that teaching-related CPD has on student outcomes​'​.

Approaches to assessing impact:

Evaluating the impact of teaching CPD is complex and there are a number of methodological issues to consider in terms of what and how it is being assessed as broader conceptual shifts and development can be difficult to measure (Stes et al., 2010).

One way of capturing this complexity is to consider using a range of quantitative and qualitative approaches including questionnaires, interviews, reflections (oral and written), focus groups, and journals. As Trigwell (2012) and others suggest, using data that is already collected by the institution can offer an additional lens through which to consider impact and may well provide a longitudinal perspective.

Approaching impact from a critical perspective:

Many researchers caution against taking too narrow an approach to assessing impact and against measuring only those things which are easily quantified. There exists a tension between being accountable (and taking measures to demonstrate this) and nurturing those aspects of development that do not deliver a quick outcome, but take rather longer to demonstrate enhancement or change.

Ways of working towards a more coherent articulation of impact might include:

  • staying alert to the complexity in relation to measuring impact
  • considering the range of perspectives from which to consider it (teacher, student, institution and programmes)
  • drawing on a range of quantitative and qualitative measures
  • exploring impact over time.​

Measuring the impact that teaching-related CPD has on student outcomes​

Making causal claims about the impact that teaching-related CPD has on student outcomes is extremely difficult to do. This relationship is complex; it is problematic attributing specific action with student learning or to establish learning gain for students. There are two main ways in which teaching-related CPD might affect student learning:

  • impact on how students acquire course content
  • impact on how students learn.
These effects can be measured in terms of student satisfaction, changes to the way they study and their academic achievements. Currently most UK institutions measure student satisfaction through Student Perception Questionnaires (SPQs), also known as module feedback forms, and the National Student Survey (NSS) (reported though Unistats) as well as monitoring student achievements through degree classifications. There is currently less emphasis on how students study although there is a growing interest in this area.

If you are interested in exploring the impact of teaching-related CPD on students there are a number of ways that this can be done.

There are instruments that explore the students' perception of teaching and capture student study habits. These provide a snapshot of student behaviours which can be harvested over time to identify impact and change. The following are suggested as examples reported to have been used across the sector:

Learn more about evaluation in our Introduction to evaluation