Evaluation

Principles of Evaluation

Evaluation may be defined as "a disciplined attempt to find out if a taught programme makes a difference". I am here (very) roughly following the Kirkpatrick model; it's not the only one, but since it is business oriented (rather than purely academic) it does emphasise the impact of taught programmes rather than simply their internal characteristics.

The model was developed as part of his Ph.D by Don Kirkpatrick in 1954—rarely can a PhD have paid off so directly and for so long! It was modified in minor respects in the "New World" version in 2010. In brief, it identifies four levels at which training programmes need to work: (Kirkpatrick's own points are italicised; my notes follow.)

Level 1: Reaction

"To what degree participants react favorably to the training."

It is of course all too easy to be seduced by this—particularly if you are a freelance trainer looking to be re-engaged, perhaps. It's not unknown for people to report that they thoroughly enjoyed a course—whether academic or technical—but did not learn anything. You could say this is the "entertainment" level.

Level 2: Learning

"To what degree participants acquire the intended knowledge, skills, attitudes, confidence and commitment based on their participation in a training event."

This is what you hope to discover from your assessment of participants' learning; and feedback to you about how successful you have been in "making learning happen" (Race, 2010). And this is as far as it goes, for educational institutions. For them, the award of their qualifications is what matters—not much more.

Level 3: Behaviour

"To what degree participants apply what they learned during training when they are back on the job."

Here we step outside the teaching/training system itself, and this is where this approach to evaluation was distinctive. Naturally, as a sponsor of or contributor to training, we want it to be effective, don't we? Actually, clearly not. It is often clear that offering training is a backside-covering tactic, particularly in health and safety areas, for example. It is sometimes merely a tick-box exercise. Indeed, it can be counter-productive; it seems to have declined somewhat now, but a popular form of trade union "industrial action" used to be the "work to rule", when workers knew that actually practising as they had been trained would bring production to a virtual halt.

But there are also other factors involved in working practice, which mean that application of what has been learned is not always straightforward. See also this piece on how behaviour might change, but not always in the intended direction.

Level 4: Results

"To what degree targeted outcomes occur as a result of the training event and subsequent reinforcement."

A big question—was a training intervention an effective way of bringing about change in the organisation? This question may go beyond the pay-grade of many education and training practitioners of course, but it is the key test.

And what about unintended and unexpected outcomes? The Kirkpatrick focus does not allow for the basic IT skills course which leads to "transformative learning" for example, and it assumes that direct intervention has intended and only intended consequences...

Up-dated 3 August 2013

Evaluation Home Principles of Evaluation Methods of Evaluation

To reference this page copy and paste the text below:

Atherton J S (2013) Learning and Teaching; [On-line: UK] retrieved from

Original material by James Atherton: last up-dated overall 10 February 2013

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License.

Search Learningandteaching.info and associated sites:

Delicious Save this on Delicious        Click here to send to a friend     Print

This site is independent and self-funded, although the contribution of the Higher Education Academy to its development via the award of a National Teaching Fellowship, in 2004 has been greatly appreciated. The site does not accept advertising or sponsorship (apart from what I am lumbered with on the reports from the site Search facility above), and invitations/proposals/demands will be ignored, as will SEO spam. I am of course not responsible for the content of any external links; any endorsement is on the basis only of my quixotic judgement. Suggestions for new pages and corrections of errors or reasonable disagreements are of course always welcome. I am not on FaceBook or LinkedIn.

Back to top