AI Adoption has a Measurement Problem (CEO Magazine)
Published October, 2026
Dr. Carylynn Kemp Larson argues that organizations investing heavily in AI are tracking the wrong things. While most companies can report adoption metrics — how many employees have access to AI tools, how often they are used, and how much time they believe they are saving — these metrics reveal activity, not value. They show whether AI is being used, not whether it is creating meaningful organizational impact.
She draws a direct parallel to a measurement gap organizations have faced for years with leadership development, wellbeing, and engagement initiatives: participation data provided reassurance but rarely answered which investments actually improved performance. AI is now exposing the same gap between visible activity and measurable organizational value.
Dr. Larson identifies four ways AI can influence the workforce conditions that drive long-term performance — either strengthening or undermining them depending on how implementation is managed. These include managerial capacity, strategic decision-making, employee stress and stability, and capability development among newer employees still building foundational skills.
She makes the case for a broader ROI framework for AI that captures impact across engagement, retention, strategic thinking, wellbeing, and managerial effectiveness — enabling organizations to evaluate AI investments alongside other strategic investments using a shared measurement lens. In an environment where both technological change and resource scrutiny are accelerating, she argues that competitive advantage will come not just from access to AI, but from the ability to evaluate its organizational impact.
Key Takeaways
Adoption metrics reveal activity, not organizational value.
Knowing how often AI is used tells you nothing about whether it is strengthening or undermining the workforce conditions that drive long-term performance.AI’s impact extends well beyond efficiency.
Time saved and tasks automated are only part of the picture — AI also shapes engagement, managerial effectiveness, strategic thinking, and employee wellbeing.AI can strengthen or unintentionally weaken critical performance drivers.
Depending on how it is implemented, AI may increase managerial capacity and improve decision-making — or increase stress, instability, and capability gaps among newer employees.The measurement challenge AI creates is not new.
Organizations faced the same gap for years with people investments — participation data provided reassurance but rarely answered which investments actually improved business performance.Emerging ROI tools make broader measurement feasible.
New approaches make it increasingly practical to estimate the organizational impact of AI investments across multiple performance dimensions quickly, credibly, and at scale.AI investments should be evaluated alongside other strategic investments.
A shared measurement framework lets leaders compare AI against leadership development and workforce capability investments — rather than evaluating each in isolation.Decisions grounded in evidence beat decisions driven by hype or fear.
When leaders can measure how AI influences workforce conditions, investment decisions become more strategic and defensible.Competitive advantage will come from measurement, not just access.
In the years ahead, the organizations that can evaluate AI’s organizational impact — not just track its usage — will hold a meaningful edge.
Authored by Carylynn Kemp Larson, a psychologist turned speaker and leadership coach who creates space for leaders, teams, and organizations to thrive.