The Kirkpatrick model: what it is, its 4 levels and how to apply it

What is the Kirkpatrick model?

The Kirkpatrick model is a framework for evaluating the effectiveness of training across four successive planes: participants’ reaction, the learning they acquired, the behavior they then show on the job and the results all of that produces for the business. Donald Kirkpatrick formulated it in 1959 and, more than sixty years later, it remains the most widely used evaluation framework in corporate training.

Its real usefulness is more modest and more valuable than it’s usually made out to be: it isn’t a formula for calculating the return on training, it’s an inventory of what can be measured and of when it can be measured. And its biggest contribution is almost a rebuke: it exposes the training departments that measure only level 1 and conclude from there that the program worked.

In this guide you’ll see where the model comes from, the four levels with details on what each one costs to measure, what the research says about its weak point —it has one, and it matters— what the 2016 revision changed, and how to apply it without ending up with a report nobody uses. If what you’re after is an overview of every evaluation model available and which one to choose, you’ll find it in our guide to training evaluation.

Where the model comes from and who created it

Knowing the origin isn’t trivia: it explains why the model has the shape it has and why it carries the problem it carries.

Donald Kirkpatrick published a series of four articles in 1959 —Techniques for Evaluating Training Programs— in the Journal of the American Society of Training Directors, the magazine of the organization that is now ATD. Each article developed one of the four planes. Three and a half decades later he collected and formalized them in the book Evaluating Training Programs: The Four Levels (1994), which is the reference usually cited.

There are two details that are almost never mentioned and that are worth keeping in mind:

• Kirkpatrick didn’t call them “levels”, he called them steps. The word “level” came later and it isn’t innocent: it suggests an ascending hierarchy that the original model never asserted that firmly, and it is the source of much of the misunderstanding that followed.
• The authorship is disputed. It has been pointed out that the four questions come from the work of Raymond Katzell, whom Kirkpatrick himself cites in a 1956 article in that same magazine. It isn’t a settled matter and there are differing interpretations, but it deserves a mention: the sector’s most used framework has a less clean attribution than its name suggests.

What matters in practice is that the model was born as a descriptive taxonomy —a way of ordering what can be observed after training— and has been used for decades as though it were an explanatory model that predicts how one thing leads to another. They aren’t the same, and that’s where the criticism you’ll see below comes from.

The 4 levels of the Kirkpatrick model

Each level answers a different question and is measured at a different moment. Here’s the overview before going into detail.

Level 1: Reaction

It measures participants’ perception: whether they found the content relevant, whether the format worked for them, whether they would do it again. It is collected with a short survey at the close.

How it is measured: satisfaction survey, perceived-relevance score or a training NPS.
What it costs: almost nothing. It’s the only level that can be fully automated.
The typical mistake: treating it as proof of effectiveness. A high score indicates that the experience was pleasant and that the program has permission to keep existing. It does not indicate that anyone learned. The 2016 revision refined this level by adding two questions more useful than “did you like it?”: whether the participant perceives the content as relevant to their job and whether they were genuinely engaged during the training.

Level 2: Learning

It measures what was actually acquired: knowledge, skills and, where applicable, changes in attitude. The rigorous way to do it is to compare a prior measurement with a later one, because without the first you can’t know how much of what someone answers correctly they already knew before walking in.

How it is measured: a before-and-after test, practical exercises, case resolution or a demonstration of the skill.
What it costs: moderate. The effort isn’t in running the test, it’s in designing it well.
The typical mistake: measuring recall only. A multiple-choice question checks recognition; asking someone to reconstruct the procedure or resolve a case checks understanding. The difference shows up precisely when it has to be applied. This is where formative assessment contributes most, because it lets you correct the gaps while the program is still running instead of discovering them at the end.

Level 3: Behavior

It measures whether what was learned is being applied on the job. It’s the level that genuinely matters, because it’s where you find out whether the training was good for anything, and it’s also where most implementations of the model fail.

How it is measured: on-the-job observation, work review, the direct manager’s assessment, cross-checked self-assessment or operational process indicators.
What it costs: high, and not because of the instrument but because of the organization. It requires having defined the expected behavior in observable terms, having someone observe it, and that someone knowing what to look for.
The typical mistake: trying to measure it without first defining what you expect to see. “Applying the protocol” isn’t observable; “logging the incident with the right code within the first two minutes” is. Evaluation rubrics are the tool that makes this level reproducible, because they set explicit criteria that two different observers score the same way.

And one nuance the 2016 revision put front and center: behavior doesn’t hold up on training alone. If, back on the job, nobody reinforces it, nobody reviews it and nothing changes in the environment, the behavior reverts within weeks. That isn’t a failure of the learner or of the course: it’s a failure of the system around the course.

Level 4: Results

It measures the impact on the business indicator the program was aimed at: operational errors, process times, sales, quality, turnover, accident rates.

How it is measured: by comparing how the KPI evolves, ideally against a comparison group that didn’t go through the program.
What it costs: high, and the problem isn’t collecting the data, it’s attributing it. A lot happens at once in a company, and an indicator that improves after a training program hasn’t necessarily improved because of that program.
The typical mistake: deciding which indicator to look at once the program is already over. If the KPI wasn’t named before you started, level 4 turns into a hunt for figures that confirm what you already wanted to conclude.

The mistake of reading the levels as a causal chain

This is the part that separates those who have used the model from those who have only read about it. The usual reading is that the four levels form a chain: if people are satisfied they’ll learn, if they learn they’ll change their behavior, and if their behavior changes the business will improve. It sounds reasonable. The research doesn’t back it up.

• Alliger and Janak (1989), in Personnel Psychology, reviewed the model thirty years after it was formulated and found very limited empirical support for those causal assumptions. Specifically: reaction does not predict learning, and learning does not automatically lead to behavior change. It’s only fair to state what they did concede: the model provides a common vocabulary and a reasonable taxonomy for talking about evaluation criteria.
• Alliger, Tannenbaum, Bennett, Traver and Shotland (1997) confirmed it in a later meta-analysis: no substantial correlations between the measures at the different levels, and no evidence of the linear causality the model suggests.
• Holton (1996), in Human Resource Development Quarterly, was the most direct in his diagnosis: what Kirkpatrick proposed is a taxonomy of outcomes, not an evaluation model. An evaluation model would have to explain which variables mediate between one level and the next —the participant’s motivation, transfer conditions, the work environment, the manager’s support— and Kirkpatrick’s doesn’t account for them. He also questioned treating reaction as an outcome, given how little it correlates with learning.
• Bates (2004) and Reio, Rocco, Smith and Chang (2017) have added the practical criticism: applied literally, the model pushes you to evaluate what is easy to measure instead of what matters.

Does this mean it should be discarded? No, and nobody serious proposes that. It means using it for what it’s good for: as an ordered list of what can be observed. What you can’t do is infer upward. A level 1 of 9 out of 10 doesn’t authorize you to take anything for granted; an excellent average score at level 2 doesn’t let you claim that anyone has changed how they work. If you want to know what’s happening at level 3, you have to go and look at level 3.

Put another way: the model tells you where to look. It doesn’t spare you from having to look.

The New World Kirkpatrick Model (2016)

The answer to those criticisms came from inside the family itself. In 2016, James D. Kirkpatrick and Wendy Kayser Kirkpatrick published the revision known as the New World Kirkpatrick Model, which doesn’t change the four levels but does change two decisive things: what gets measured within each one and in what order the model is used.

The change in order is the most important and the easiest to overlook. In the classic version the model is used at the end, evaluating from the bottom up. In the revision it is used at the start, designing from the top down: first you define the business result you’re after, then which behaviors would produce it, then what people need to know to perform those behaviors and, finally, what the learning experience has to look like. It’s the difference between using the model to write a report and using it to design the program.

How to apply the Kirkpatrick model step by step

The sequence that works is the one from the 2016 version: start at the end. These are the five steps, with what has to be settled in each before moving on to the next.

Step 1. Name the business result

Before designing anything, write down the specific indicator that should move and its current value. Not “improve customer service”, but “reduce complaints escalated to second level, currently at 18%”. If you can’t name it, level 4 won’t be measurable later, however hard you try.

Step 2. Define the observable behaviors that would produce it

What people would have to do differently for that indicator to move, described so that two different people can observe the same thing and agree. This is the step that gets skipped most and the one that most determines whether level 3 will be measurable.

Step 3. Identify what people need to know and what needs to hold it up

Two questions, not one. What knowledge and skills are needed to perform those behaviors —that’s the training content— and what has to change in the environment for them to stick: who reviews, who supports, what gets recognized. Those are the required drivers, and without them the program evaporates within a month.

Step 4. Design the measurement for all four levels before launching

Which instrument, who applies it and on what date, for each level. Level 3 included — the one that never gets planned and therefore never gets measured. If at this point there is no name and no date for the on-the-job observation, that level isn’t going to exist.

Step 5. Measure, interpret each level separately and decide

Collect each level at its own moment and, above all, don’t infer from one to another. Interpret each for what it is and cross-reference the results to locate where the chain breaks. If level 2 is high and level 3 is low, the problem is in the work environment, not in the course. If level 3 is high and level 4 doesn’t move, the behavior you chose wasn’t the one that moved the indicator. That diagnostic capability is the model’s real usefulness, and it only appears if you measure more than one level.

Which level to measure for each type of program

Measuring all four levels on every program is the fastest way to measure none of them well. The practical rule: go up to the level you can support with evidence and stop at the first one that doesn’t give you a decision. This table resolves most cases.

A full worked example, from start to finish

A retail chain finds that 18% of complaints get escalated to second level when they could be resolved in store. Here’s how the full model would be applied, in the right order:

• Result (level 4), defined before starting: bring escalated complaints down from 18% to 12% within six months. Intermediate indicator so you don’t wait half a year: percentage of complaints closed on first contact, reviewed monthly.
• Behaviors (level 3), defined observably: identify the real reason within the first two minutes, check the resolution margin available and log the case with the right code. It will be observed by reviewing ten cases per store per month, using a shared rubric, starting on day 30.
• Required drivers, without which none of the above holds up: a one-page job aid with the resolution margins, five minutes of review in the weekly store meeting and monthly recognition for the team with the best first-contact resolution.
• Learning (level 2): a before-and-after test on resolution margins and three practical cases requiring a decision. Confidence is measured, not just correctness: “do you feel able to resolve it without escalating?”.
• Reaction (level 1): three questions at the close, focused on perceived relevance rather than general satisfaction.

Notice what was designed first and what was designed last. The course is the final piece, not the first.

A real case of a level 4 that did move. Clarel aligned more than 1,000 points of sale and over 3,000 employees on culture and operations almost in real time. The published figures let you see two different levels at once: 84% participation in the training —which is coverage, not impact— and a 90% increase in product accuracy rates, which is a business indicator that did move. The distinction between those two figures is exactly what this article is about. See the case study →

Common mistakes when applying the Kirkpatrick model

Almost all implementation problems repeat themselves, and almost all of them can be anticipated.

• Stopping at level 1 and drawing conclusions from there. It’s the most widespread mistake and the one the research disallows most clearly. A high satisfaction survey is evidence of nothing beyond a good experience.
• Measuring level 3 without having defined the behavior. If what you expect to see isn’t described in observable terms, observation turns into opinion and each manager scores by a different standard.
• Attributing a business result with no comparison. A KPI improving after a training program doesn’t mean it improved because of it. Without a comparison group or a historical series, it’s a correlation presented as a cause.
• Choosing the indicator once the program is over. At that point you aren’t measuring: you’re looking for figures that confirm the conclusion you wanted.
• Measuring all four levels on every program. Level 3 is expensive and so is level 4. Applying them to everything guarantees doing all of it badly.
• Forgetting the environment. This is Holton’s criticism put into practice: if nobody reinforces the behavior once people are back on the job, the program didn’t fail because of how the course was designed.

The Kirkpatrick model versus other evaluation models

It isn’t the only framework available, and knowing the others helps you place its limits. The Phillips ROI model extends it with a fifth level that converts the benefits of level 4 into monetary value and compares that with the cost of the program. Kaufman’s adds a prior level of inputs and processes and widens the results level to impact on the customer and on society. And there are several more —CIRO, Anderson, Brinkerhoff, CIPP— with different approaches depending on what you want to find out.

As a starting point, Kirkpatrick remains the sensible choice for almost any organization: it’s the best known, the cheapest to implement at its first levels, and the one that works best as a common vocabulary between L&D and management. If you want the full comparison of the seven main models, with their level of complexity and which type of organization each one suits, it’s laid out in our guide to training evaluation. And if your context is online training, here’s how it plays out in e-learning assessment.

How isEazy LMS helps you apply the model

A platform doesn’t evaluate for you, but it does determine how much manual work each level takes. With isEazy LMS you can automate the part that is automatable:

• Level 1: end-of-course surveys, with results aggregated by group and no manual exports.
• Level 2: before-and-after tests, pass thresholds, attempts and scores by module, with alerts when someone falls short of the threshold.
• Level 3: traceability of progress and of post-course activity, reports by person, department or site to use as the basis for observation, and segmentation so you know who needs observing.
• Level 4: export of training data so you can cross-reference it with business indicators in the organization’s dashboard.

And it’s worth saying plainly what no platform solves: level 3 requires observation on the job. The LMS tells you who completed what, with what score and in how long, and that’s essential; but whether someone applies what they learned with a customer in front of them is something a person has to look at. Any tool promising to solve level 3 without observation is measuring something else.

If you want to see how all of that is collected and cross-referenced in practice, you can request an isEazy LMS demo.