What if your organisation is investing heavily in change-related training—but no one can tell whether it’s actually working?
Organisational change initiatives fail at alarming rates—research consistently suggests that 70% of change programmes do not achieve their intended objectives, representing enormous waste of resources and lost opportunities. While multiple factors contribute to these failures, inadequate attention to learning and development represents one of the most significant and addressable causes.
When organisations undergo transformation—whether through digital adoption, post-merger integration, or cultural realignment—employees must acquire new skills, adopt different behaviours, and embrace unfamiliar ways of working. Without systematic assessment of learning impact, leaders are essentially flying blind, unable to determine whether their development investments are producing the capabilities required for successful change. Worse still, they may mistake activity for progress: high course completion rates or positive feedback scores can create a false sense of security while critical behavioural shifts remain unrealised.
The Kirkpatrick Model in Change Contexts
Kirkpatrick’s four-level evaluation model—reaction, learning, behaviour, and results—remains the most widely applied framework for assessing training effectiveness across organisations worldwide. During organisational change, however, each level demands thoughtful adaptation to address the specific complexities of transformation.
Reaction: Beyond Satisfaction Surveys
At the reaction level, traditional satisfaction surveys are insufficient on their own. In change environments, emotional responses such as anxiety, resistance, or hope significantly influence how participants perceive training. Therefore, assessments should also capture learners’ readiness for change, perceived relevance of content, and confidence in applying new knowledge. A high satisfaction score may mask underlying scepticism if not paired with deeper diagnostic questions.
For instance, asking “How confident are you in using this new system after today’s session?” yields more actionable insight than “How satisfied were you with the trainer?” Modern L&D teams are increasingly supplementing end-of-course surveys with pulse checks during onboarding phases to track sentiment shifts as employees move from awareness to adoption.
Learning: Applying Knowledge in Uncertainty
At the learning level, it’s not enough to test knowledge retention alone. Change-related learning must be evaluated for its applicability in fluid, uncertain conditions. Can employees use what they’ve learned when processes are still being defined or systems are in flux? Scenario-based assessments and simulations can reveal whether learners can adapt their knowledge to real-world ambiguity—a hallmark of most change initiatives.
Consider a bank rolling out a new compliance protocol amid regulatory upheaval. A quiz on policy wording proves little; instead, presenting employees with realistic client scenarios—where rules conflict or information is incomplete—tests true comprehension and judgment under pressure.
Behaviour: Measuring Real-World Application
The behaviour level becomes especially challenging during transitions. Baseline performance is often disrupted by restructuring, new reporting lines, or shifting priorities, making it difficult to isolate the effect of training from general organisational turbulence. To overcome this, organisations should establish pre-change behavioural benchmarks and use control groups where feasible.
Managers play a critical role—not just as observers, but as active coaches reinforcing desired behaviours post-training. Without managerial reinforcement, up to 87% of learning fails to transfer to the job, according to research by the Centre for Creative Leadership. Structured follow-up sessions, peer coaching circles, and “learning-in-action” checklists can bridge this gap.
It’s worth noting that the original Kirkpatrick model has been critiqued for its linear assumptions and lack of attention to contextual factors like organisational culture or external pressures [[1]]. The updated Kirkpatrick Model addresses some of these gaps by emphasising the importance of motivation and enabling work environments as prerequisites for behaviour change [[9]].
Results: Linking Learning to Strategic Outcomes
Finally, at the results level, learning must be explicitly linked to strategic change outcomes—such as adoption rates of new technology, speed of process implementation, reduction in error rates, or employee engagement scores during transition—not just generic KPIs like revenue or productivity. Only by aligning learning metrics with change milestones can organisations credibly demonstrate the contribution of L&D to transformation success.
For professionals seeking structured approaches to leading transformation while measuring learning outcomes, the Certified Professional Change Management (CPCM) Training Course at Alpha Learning Centre provides comprehensive frameworks grounded in industry best practices.
Going Beyond Kirkpatrick: The Phillips ROI Methodology
While Kirkpatrick offers a solid foundation, many organisations require a more rigorous financial lens—especially when justifying L&D budgets during cost-conscious transformations. This is where Jack Phillips’ ROI Methodology adds critical value by introducing a fifth level: Return on Investment.
The Phillips model builds directly on Kirkpatrick’s four levels but adds a quantitative financial dimension. It follows a 10-step process that begins with identifying clear, measurable business objectives and ends with calculating the monetary value of learning interventions [[12]]. The five levels are:
- Reaction and Planned Action: How satisfied were participants, and what do they plan to do differently?
- Learning: What knowledge, skills, or attitudes were acquired?
- Application and Implementation: How was the learning applied on the job?
- Business Impact: What measurable effect did the application have on key performance indicators?
- ROI: What was the financial return compared to the cost of the programme?
For example, an organisation implementing a new CRM system might calculate that improved user proficiency (Level 3) reduced sales cycle time by 15%, generating £250,000 in additional annual revenue. If the total cost of training was £50,000, the ROI would be 400%—a compelling case for future investment [[14]].
Crucially, Phillips emphasises isolating the impact of learning from other variables—a challenge in dynamic change environments. Techniques include trend-line analysis, control groups, and expert estimation, all of which are taught in depth in the Strategic Organisational Change Management Certification Training Course.
Measuring Behaviour Change Over Time
The ultimate objective of learning during change is sustained behaviour change that supports transformation objectives. Yet behaviour doesn’t shift overnight—it evolves through practice, feedback, and reinforcement. Consequently, assessing this change requires longitudinal approaches that track progress over weeks or months, not just immediately after a workshop.
Establishing Baselines and Milestones
Effective practice begins with pre-change baseline measurements. These might include performance reviews, peer feedback, or observed workplace interactions that document current behaviours. Post-training, multiple assessment points—at 30, 60, and 90 days—capture the typical learning curve: an initial dip as employees unlearn old habits, followed by gradual improvement as new practices take root.
For instance, a healthcare trust introducing a new patient handover protocol might measure communication clarity, error rates, and team confidence before launch, then re-measure at intervals to track adoption fidelity and identify pockets of resistance.
Using Mixed-Methods for Reliable Insights
A blend of qualitative and quantitative methods strengthens validity. Direct observation by trained managers or internal coaches offers rich contextual insights, though it must be standardised to reduce bias. 360-degree feedback allows colleagues, direct reports, and stakeholders to report on observable shifts in communication, decision-making, or collaboration. Digital learning platforms can also track application through completion of follow-up tasks, participation in discussion forums, or usage of job aids.
Understanding why employees resist change is key to unlocking true behavioural shift. Our resource on resistance to training programmes explores the psychological and structural barriers that often stall progress—even when people appear engaged.
Meanwhile, our practical guide on learning agility metrics and implementation offers self-assessment rubrics, manager checklists, and team reflection protocols that help organisations detect early signs of disengagement or misalignment—and intervene before momentum is lost.
Leveraging Technology for Real-Time Insights
Modern learning ecosystems offer unprecedented opportunities to assess impact in near real time. Learning Experience Platforms (LXPs), xAPI-enabled tools, and integrated HRIS systems can aggregate data on course completion, knowledge checks, social learning activity, and even on-the-job performance indicators. During change, this data becomes a vital early-warning system.
Tools like Docebo, Cornerstone, and CYPHER Learning now provide AI-powered dashboards that surface trends such as declining engagement in specific departments or repeated failure on scenario-based assessments—enabling L&D teams to pivot content or provide targeted support before issues escalate [[27]].
For example, if uptake of a new compliance module drops sharply among a regional team, it may signal deeper resistance to the broader policy shift—not just a scheduling issue. Similarly, low engagement in post-training coaching circles might reveal a lack of psychological safety to discuss change-related challenges.
Real-time analytics also enable “just-in-time” learning interventions. If data shows that customer service agents struggle with a new refund policy, microlearning nudges can be triggered automatically—reinforcing key steps at the moment of need rather than weeks after a classroom session [[22]].
Alpha Learning Centre’s courses emphasise not only what to measure but how to act on findings. Assessment isn’t an endpoint—it’s a feedback loop that informs iterative improvements to both learning design and change strategy.
Common Pitfalls and How to Avoid Them
Even well-intentioned evaluation efforts can falter without careful design. Common mistakes include:
- Measuring too late: Waiting until the end of a change programme to evaluate learning misses opportunities for mid-course correction.
- Over-relying on self-report: People often overestimate their competence or application; triangulate with observational or system data.
- Ignoring contextual noise: Market shifts, leadership changes, or budget cuts can distort results; account for these in your analysis.
- Failing to close the loop: Sharing insights back with stakeholders builds credibility and drives continuous improvement.
One powerful antidote is to embed evaluation into the change roadmap from day one. Define what “success” looks like at each phase, assign ownership for data collection, and schedule regular review sessions with sponsors and frontline managers.
Conclusion
Assessing learning impact during organisational change requires more than ticking boxes on an evaluation form. It demands adaptive frameworks like Kirkpatrick and Phillips, longitudinal measurement, and a commitment to connecting learning directly to transformation outcomes. By thoughtfully applying these models, integrating behavioural science insights, and harnessing data-driven tools, learning professionals can move from being cost centres to strategic enablers of change.
In today’s climate of constant disruption—from AI-driven transformation to post-pandemic restructures—the ability to prove that learning drives results is no longer optional. It’s essential. And with the right approach, organisations don’t just survive change—they thrive through it.
