AI Foundations · Part 8 of 8
The Organizational Layer: Adoption, Cost, and Governance
Why 95% of AI investments show no measurable return, what workslop is, and the cost models, FinOps thinking, and guardrails that keep AI usage sane at scale.
The final part of the AI foundations series. Everything so far has been about the technology and how to build with it. This post is about the layer that determines whether any of it produces value, the organization around it.
Let me open with the statistic that reframed the whole topic for me. According to a 2025 MIT study cited in the course, 95% of organizations investing in AI adoption reported zero measurable results.
Zero. That tells you two things at once: companies are eager and investing real money, and adoption is high while transformation is low. AI is present in the organization and not making a difference.
So what separates the other 5%?
Three narratives, all partly right
Before the mechanics, it helps to recognize the stories people carry, because they directly affect adoption.
The doom scenario. AI replaces humans, all jobs get automated, people are pushed out. People who believe this fear AI.
The bubble scenario. AI is overhyped, a shiny toy, and it’s going to pop. On the Gartner hype cycle this places us at the peak of inflated expectations, about to crash into the trough. People who believe this ignore AI.
The Jevons Paradox scenario. This is the one I hadn’t considered. In 1865, economist William Stanley Jevons observed something odd about coal. The Watt steam engine was far more efficient, so everyone expected coal consumption to drop. It rose. Efficiency made coal cost-effective, so steam engines spread everywhere, factories, transport, agriculture. Less coal per job, far more jobs, more coal overall.
Apply that to AI. As the cost of using it drops, the cost of software projects drops, so companies commission more of them, and total demand for AI goes up, not down.
All three might be true in different contexts at different times. If you’re driving adoption, you need to recognize the first two mindsets in the room and work with them rather than argue.
Why adoption stalls
Rolling out a tool isn’t adoption. It’s not about access, it’s about how people use it. Four failure points:
- Clarity. People have access but don’t know why they should use it. What’s the goal, what’s the scope? Without direction, usage is random and impact is invisible.
- Learning and enablement. Like any tool, AI can be used well or badly. Left to figure it out alone, most people won’t get far. They need a structured path covering when, where, and how.
- Metrics. We say we want efficiency. How are we measuring it? Without metrics, improvement is a guess.
- Executive alignment. Leaders carry the same narratives as everyone else. Some champion it, some resist. Vision and execution both need alignment.
And measure across three dimensions: utilization (daily and weekly active users, if nobody uses it there’s nothing to measure), impact (time actually saved, quality delivered), and cost (if AI saves time but costs more than it saves, the ROI isn’t there).
Workslop, and the trap underneath it
This was my favorite concept in the whole path, partly because I’ve received this email.
Following our highly collaborative and synergistic alignment session earlier today, please find below a detailed enumeration of key action items, ownerships, and anticipated completion timelines for optimized cross-functional execution.
Versus:
Thanks for the productive discussion. Alex, finalize the API integration plan by Wednesday. Jamie, draft the product announcement by Friday.
The first is workslop: text containing some true information, written so elaborately it becomes almost useless. Buzzwords and inflated language that sound intelligent and are actually noise. Harvard Business Review flagged it as a growing drain on productivity.
The cause is cognitive offloading: outsourcing your thinking to AI rather than using it to assist your reasoning. You hand over the problem and accept whatever comes back. But the model doesn’t understand context, intent, or clarity, so it produces something polished and hollow. Then your colleagues spend energy decoding it.
The net effect is negative. Work got faster for the sender and slower for everyone downstream. That’s the real risk of careless adoption, not that AI does the work badly, but that it moves the cost onto other people where nobody measures it.
Enablement has to keep up with a field that reinvents itself
Keeping people current is genuinely hard when the ground moves this fast. The course’s comparison of AI-assisted development one year apart makes it concrete:
| 2024 | 2025 | |
|---|---|---|
| Interface | Simple IDE plugins | IDEs, CLI tools, web environments |
| Capability | Autocomplete, basic chat | Agentic coding running tasks autonomously for hours |
| Context control | Only the files you had open | Subcontexts and memory files persisting across sessions |
| Interoperability | None, every tool an island | Emerging standards like Model Context Protocol and agent-to-agent interfaces |
| Context window | ~10,000 tokens | ~1,000,000 tokens |
A hundredfold context increase and a shift from autocomplete to autonomous task execution, in twelve months.
Two kinds of knowledge need capturing:
External: learning platforms, conferences, blog posts, research papers. Keeps people connected to the wider ecosystem.
Internal: lessons from your own hackathons, proofs of concept, and A/B tests between tools and models. Every experiment generates insight; the goal is capturing it so everyone benefits.
That produces two artifacts worth maintaining: a learning path for external knowledge, and a collection of internal best practices: guidelines, case studies, patterns from your own experience.
Then keep three feedback loops running: the research loop (exploring new tools), the vendor loop (staying aligned with provider roadmaps so you can plan ahead), and the practice loop (actually trying things).
Ways to drive adoption that work
Community of practice. Instead of top-down mandate, let curious peers lead. Diversity is non-negotiable here, security people raise privacy and compliance concerns, designers focus on usability, senior engineers think about scalability and integration. Everyone sees the problem through a different lens, and the more diverse the group, the stronger the ideas and the safer the implementation.
A good starting move if the goal is productivity: map your SDLC and find the bottlenecks. Code review dragging? Documentation always behind? Testing slowing releases? Those are where AI can make a measurable difference.
Pilot programs. The bridge between experimentation and transformation. Three things make them work: clear scope (duration, boundaries, expected impact, cost comparison against doing it manually), clear metrics with a baseline established before you start, and organizational consensus at the end so the result actually gets scaled.
Set realistic expectations. It’s tempting to chase 10x, but a 10–20% efficiency gain is already a significant win if it’s measurable and repeatable.
Dedicated AI teams. Communities hit a wall, members have day jobs, and time scarcity limits how much they can support. At some point you need a dedicated team that architects scalable, observable, secure solutions; drives adoption and enablement; and provides governance through guardrails and policies.
One more practical note: not every team is ready to move fast, and that’s fine. Teams on non-critical or sandbox systems can experiment freely. Teams running critical services should adopt at a slower, controlled pace. Stability first.
Cost models and FinOps for tokens
Three common models:
| Model | How it works | Best for |
|---|---|---|
| Fixed per user | Set monthly fee per person, each gets a token allowance in a rolling window | Predictable budgeting |
| Pooled | Tokens combine into a shared pool everyone draws from | Uneven usage across teams, heavy users balance against light ones, so limits are hit less often |
| Usage-based | Pay per token, no fixed fee | Irregular or low-volume usage |
To manage them, the course borrows FinOps: the discipline for managing consumption-based spending. The core principle is that everyone takes ownership of their usage; it’s shared responsibility, not finance’s problem. And it’s not about minimizing cost, it’s about maximizing value per dollar.
The cloud-to-AI mapping is neat:
- Cloud infrastructure costs → token costs
- Resource optimization → prompt engineering (better results with fewer tokens)
- Instance types → model types (choosing the right one changes cost and performance dramatically)
Four capabilities to build in: centralized metering (usage by model, team, user), dashboards and alerts for real-time spending patterns, budgets and quotas with soft and hard limits, and access control: not every team needs the most expensive model.
The stakes are real. One interview mentioned in the course had someone running a team of agents burn nearly $1,000 of tokens in a single workday.
LLMOps: a gateway instead of scattered keys
Here’s the problem, built up step by step. One app calls one model, so it needs one API key. Fine. Add a second app, share the key and you lose cost visibility; use separate keys and it’s still manageable. Now let each app use multiple models across multiple providers. Every app-model combination needs its own key. Multiply by dozens of apps and it spirals.
And keys aren’t the only issue. You also need per-application spend tracking, budgets and limits, alerts, tracing to see which app called which model, logging of prompts and responses for debugging and audit, and guardrails against misuse.
An LLMOps platform solves this by sitting as a gateway between your applications and the models. Every AI request passes through it, so it has visibility into usage, cost, performance, and governance.
Key management gets much cleaner: vendor API keys live securely at the platform level, and each app gets a virtual key: an identifier the platform recognizes. When a request arrives, the platform identifies the app, applies the right policies, and routes to the correct model. Applications never see or manage vendor credentials.
Tools in this space include LiteLLM, Portkey, Helicone, and LLM Gateway, with new ones appearing regularly.
Guardrails
Because the gateway already sits between your apps and the models, it’s the natural control point. Four categories:
- Budget: financial limits per individual, team, or organization, triggering alerts or temporary restriction.
- Prompt: validating input before the call and scanning output after, plus logging for audit. The same checks can wrap MCP server calls.
- Rate limits: max tokens or requests per minute, so one app can’t consume all bandwidth and degrade everyone else.
- MCP server access: allow lists (explicitly permitted servers) or deny lists (block specific ones, permit the rest).
Worth emphasizing: guardrails aren’t primarily about someone maliciously trying to break the system. Unintentional misuse happens far more often: a runaway loop, a well-meaning experiment, sensitive data pasted into a prompt without thinking.
Governance is enablement, not bureaucracy
The mindset shift that stuck with me: early on, teams think move fast, fix governance later. At scale that reverses. Organizations that succeed put guardrails in place before they need them.
Because governance added after a failure feels like punishment. Governance designed upfront feels like enablement.
Governance, security, and compliance are interlocking: governance defines intent, security protects execution, compliance demonstrates accountability. Weaken one and the others suffer.
What I’m taking from the whole series
Looking back across all eight posts, the throughline is that AI systems don’t become valuable because they’re intelligent. They become valuable because they’re built and operated well: with honest expectations, representative data, deliberate architecture, real monitoring, and humans placed where they add the most value.
Or as the last course put it: the organizations that win with AI won’t be the ones that move fastest. They’ll be the ones that operate best.
That’s the end of the foundations path. Start from the beginning with What AI Actually Is if you landed here first.