Embedded Learning Systems

The transfer problem

In 2018 I tried to teach myself to code the way everyone tells you to teach yourself to code. I enrolled in online classes, watched lectures on data structures, and completed some exercises on recursion. After each course, I did well on quizzes and would feel briefly competent. Then I'd try to build something of my own, and the knowledge would evaporate. Three courses in, I had little to show for it other than some checkmarks on Coursera.

What eventually worked was not “educational.” It was moving to San Francisco and falling into a group of software engineers. Nobody sat me down and taught me anything, but suddenly I was around it. Watching them debug side projects in real time, hearing how they describe their work, picking up on what “good” looked like before I could even articulate why. After a summer in San Francisco, I felt like a “programmer.”

Learning scientists have a name for the thing that didn't happen during my online courses: the transfer problem. Knowledge acquired in one context doesn't automatically become usable in another. You learn recursion in a browser tab, and it stays in the browser tab. Recognizing an answer on a quiz is different from recognizing when an idea is useful, adapting it to a new problem, and deciding what to do next.

Context is part of this. But so are practice, feedback, and the opportunity to see how experienced people make decisions. My courses gave me concepts. The summer gave me somewhere to use them.

I bring this up because the same thing is now happening at work, at a much larger scale, and AI is part of the reason.

The exoskeleton

In a 2024 experiment, BCG gave 480 of its consultants data-science tasks they had never been trained for: cleaning and merging datasets in Python, building a predictive model. With an AI tool, their coding work reached 86% of the benchmark set by actual data scientists. Without it, 29%. But afterward, the consultants who had done the tasks knew no more about data science than the ones who hadn't. The researchers called the AI an exoskeleton. Take it off, and the capability goes with it.

It's my Coursera problem turned inside out. My courses gave me the concepts without the work. The exoskeleton gives people the work without the concepts. Either way, nothing carries over to the next problem.

And people are spending more of their day inside the exoskeleton. When Anthropic studied how its own engineers use Claude, it found them becoming more “full-stack”: a backend engineer working on front-end code they “would've been scared to touch,” researchers building their own data visualizations. About 27% of the Claude-assisted work wouldn't have been done at all otherwise. The range of what models can take on keeps growing fast. By METR's measure, the length of software task they can complete on their own has been doubling roughly every four months since 2023. Each jump pulls more people into unfamiliar work next to their own.

That raises a fair question. If a model can do the data science, why should the consultant learn it? Because someone still has to decide what to ask, notice when the answer is wrong, and know what to do next. Anthropic's engineers put it plainly: “effectively using Claude requires supervision, and supervising Claude requires the very coding skills that may atrophy from AI overuse.” The goal isn't for people to out-compete the model. It's for working with the model to build people up instead of deskilling them, so that the person and the model together keep getting more capable, and the person does too.

The usual answer to changing work is to retrain people, mostly through courses. The OECD's 2025 analysis of adult learning found that 8% of adults were in formal education and 37% in job-related training, much of it short. That has its place. But courses happen away from the work, which is exactly where my own learning failed. The unfamiliar work people are already doing every day is the better opportunity. The question is how to make it teach.

What apprenticeship gives you

Most work isn't practice. You do what you already know how to do, as well as you need to, and move on. Feedback, when it comes, comes late: a quarterly review, a bug report weeks later, a manager's edits you never see the reasoning behind.

Researchers who study expertise draw a sharp line here. Experience alone doesn't make people much better. Deliberate practice does: work aimed just past what you can currently do, with clear goals, immediate feedback, and a chance to try again. The trouble is that deliberate practice usually happens somewhere else: a piano studio, a training ground, a problem set. Away from the work.

Apprenticeship is the old exception. The apprentice does real work, and the practice is built into it. A master is close enough to see the decisions as they happen. They point out the alternative you didn't consider (“you could have cut it the other way”), review what you made, and explain why. Cognitive apprenticeship names the parts: modeling, coaching, scaffolding, asking the learner to articulate their reasoning, and fading support as the learner gets better. It's deliberate practice applied to the work you already have to do.

I often think of the time I lived in San Francisco as one of the highest growth experiences of my life. I didn't know any of this terminology at the time, but apprenticeship is what I stumbled into. My friends saw my code, told me what they would have done differently, and why. It depended entirely on the accident of who I happened to be around and the fact that they were willing to give their time.

You can't give millions of people a friend group full of senior engineers.

Or can you?

Embedded learning systems

Here's a familiar way people use AI to learn something right now. You're working on a document or a spreadsheet and you get stuck. You open a new tab, navigate to a chatbot, and try to describe what you were doing.

The tutor in a chat window can be useful. But it creates a familiar distance: you extract yourself from the context where you need the knowledge in order to go ask about it. You also have to notice the gap and know enough to describe it. And more and more often, the chatbot just does the work for you, which is the exoskeleton again.

It's also unrealistically tidy. Real work doesn't happen in one application. Even a weekly update draws on a project tracker, a Slack thread, last Tuesday's meeting, and the doc you're writing it in. The task is one continuous thread. The tools are separate worlds. An assistant confined to one of them only sees its own keyhole.

An embedded learning system would follow the task, with the learner's permission, across those boundaries. It would combine the immediate work with a persistent, inspectable model of what the learner knows, what they're trying to do, and where support might help. If it thinks I know pivot tables but not Python, I should be able to see that—and tell it when it's wrong.

Today's agents often have this context already. They can take screenshots, operate your computer, or connect to the tools you give them access to. But they mostly use that context to do the work for you, out of view. You ask, and the doc or spreadsheet comes back changed. For getting things done, that's the point. For learning, it's the exoskeleton taken one step further: you skip the practice, and you don't even see the work. The work is where you learn what good and bad look like. It's the draft you reread, the chart you compare against another, the cell you rerun to see what changed. An embedded learning system uses the same context in the opposite direction. It keeps you in the interface, attaches its feedback to your own work, and leaves the change to you.

Standards such as xAPI already make it possible to record learning experiences across platforms. But apprenticeship requires more than recording activity. The master in the workshop does more than review a transcript of the apprentice's mistakes at the end of the day. They're standing next to the apprentice while the mistakes are happening. That's the difference between a system that records your learning and a system that participates in it.

Another way to put it: pull-based learning is when you go looking for help because you know you have a gap. Push-based learning is when the system notices a possible gap and offers support. Pull requires you to know what you don't know, which is precisely the thing novices often struggle with. Push could help, provided a guess about the learner stays a guess that the learner can correct.

This idea has close precedents. Andy Matuschak's How Might We Learn? imagines AI support around authentic projects, with enough context to guide people in the middle of their work. His work with Michael Nielsen on the mnemonic medium, and the related Orbit project, puts opportunities for remembering directly into explanatory material. Judy Kay and Bob Kummerfeld's work on personal user models for lifelong, lifewide learning argues that learners should be able to scrutinize and control the models built about them.

These are important inspirations. The emphasis here is on bringing practice and feedback into the work people already do: not just helping someone finish a task, but helping them become more capable of doing the next one. Recent General User Model research demonstrates contextual modeling from computer activity, including screenshots. But interpreting activity is not the same as reliably diagnosing understanding. The engineering pieces are becoming available. The educational question still needs testing.

Renatta: learning to see the choices

Consider an illustrative case. Renatta works for a county health department in New Mexico. Her director asked her to prepare a brief for next month's board meeting comparing environmental health trends across the state's counties. A colleague pointed her to a paper with a similar dataset and the notebook used to generate its figures. “Start from their analysis,” her colleague said. “Adapt it to our question.”

Renatta knows Excel. She's built pivot tables, made charts for grant reports, filtered and sorted her way through messy spreadsheets for years. She has never used a notebook. She opens the paper's: code cells instead of a grid. She can see the outputs, but the connection between the code and what it produces is opaque.

An embedded learning system is running alongside her. With her permission, it can see the notebook, the data, and the paper. It knows, because she told it, that she's comfortable with spreadsheets and new to Python. Here's what that could look like.

The system's comments show up where the work is, next to the cell they're about. The first one connects the unfamiliar thing to something she already knows: a groupby is a pivot table. That's enough for her to start editing instead of just reading.

The second is an alternative decision. Her bar chart of raw counts says the biggest county has the biggest problem. The system doesn't fix the chart. It asks whether a count is the comparison she wants, and lets her make the change. A count asks how many cases there are; a rate asks how common they are relative to the population. Once she's made the change, she can say which question her chart answers.

The third is review. Her scatterplot doesn't look like the paper's, and the system points out two choices the authors made that aren't visible in the finished figure: a log-scaled axis, and a filter that drops counties under 10,000 people. She can preview each separately. Then it asks her to write down why she would or wouldn't make each choice. A log scale changes how differences appear. Excluding smaller counties changes which places the analysis represents. Those choices need reasons, and writing the reason is part of the practice.

What Renatta is learning is not just how to use a notebook. It's how to think about data the way an analyst does: which transformations reveal structure, which visualizations mislead, which choices are invisible in a finished chart.

The test comes later. Can she recognize the same issue in a new dataset, explain her choice, and make it without the prompt? A better chart today is useful. Better judgment tomorrow, the kind that makes her and the model sharper together, is the learning outcome. That's the difference from the exoskeleton.

What could go wrong?

First is the Clippy problem. Microsoft's Clippy tried to be contextually helpful and became reliably irritating. Systems that intervene too often will be disabled. Systems that intervene too rarely won't produce learning.

The right intervention varies across people and across moments for the same person. Deep focus makes interruption costly. Frustration makes a well-timed suggestion a gift. Playful exploration means staying quiet and letting productive mistakes happen. Pause patterns, errors, and task switching might offer clues, but they don't reveal someone's mental state with certainty.

This is a research problem, not a solved interface detail. Scrutable user models point toward one part of the answer: let learners see why help appeared, correct the system's assumptions, and control when it intervenes. They should also choose which tools it can observe and be able to delete what it remembers. Scaffolding should adapt to the person rather than the other way around.

Second is social erosion. My San Francisco story was not just about contextual learning. It was about identity formation, social accountability, and the organic unpredictability of watching real humans navigate real problems. Anthropic's engineers noticed this too: junior colleagues came to them with fewer questions. If these systems become the primary source of support, they may reduce the incentive to ask colleagues for help.

So the goal can't be a tutor that stands in for people. Andy Matuschak makes a similar argument in How Might We Learn?: support inside a project should connect learners to the communities doing that work. Jean Lave and Etienne Wenger called these communities of practice. Newcomers start at the edge with small, real contributions, and move toward the center as they become capable. An embedded system could be an on-ramp to that. It builds enough skill that you can take part, and then points you to the people: the colleague who solved this last year, the channel where the analysts argue about denominators, the moment a question is better asked of a person than a model. Handing you off to people is one more way the support fades.

Third, digital scaffolding is inherently limited. A coding environment has a property that makes it natural: the work is the screen. But a nursing student practicing intubation or a diver learning buoyancy control is developing an embodied skill. You can watch a video on holding neutral buoyancy at a safety stop, but the actual skill is in your breath. It's feeling that you're rising or sinking before your depth gauge changes. A screen doesn't capture all of that.

Who decides what to learn?

If the system decides what gap you have, when to intervene, and what to teach you, who's writing the curriculum?

In traditional education, there are at least legible structures of accountability: syllabi, accreditation boards, teachers you can argue with. A system that's watching your work and nudging your learning is making pedagogical decisions constantly. Those decisions are shaped by whoever built it, trained it, and defined what “competent” looks like.

This is why the model layer matters as much as the interface layer. If embedded learning depends exclusively on proprietary models controlled by a handful of companies, it becomes harder for educators and learners to inspect or challenge the pedagogy. Open models alone won't solve accountability, but they could make more of those choices available for scrutiny.

We need a healthy diversity here: publicly funded learning models trained on appropriately shared educational data, open-source alternatives that researchers and educators can inspect and modify, and enough competition that no single company gets to define what “learning” means inside the tools where most people work. Learners should have a say in the goals, not just permission to dismiss the prompts.

Practice where the work is

What changed for me that summer wasn't that someone made my work easier. It was that people who knew more than me could see my work while I was doing it. They showed me the decision I hadn't considered, looked at what I'd made, and told me what they'd have done differently. I did the work. They made my practice deliberate.

Most people will never get that summer. But almost everyone is about to spend their working days doing things that are new to them, like a backend engineer touching front-end code or a consultant running a regression. Right now, AI mostly helps them get that work done. The work gets finished, and the person stays where they were.

Embedded learning systems are a bet that the same moments can become deliberate practice. The feedback lives in the doc, the notebook, the spreadsheet, at the moment the decision is being made. It shows the alternative, asks for the reason, and asks less as the person starts bringing the judgment themselves. The learner can see what the system thinks they know, and can tell it when it's wrong.

The measure isn't how much work the system helps finish. It's whether the person, working with it, ends up better than the model alone, and keeps getting better.