A large ARR drop showed up in one of our dashboards. A data analyst in the FP&A team caught it on a Saturday. I’d shown him our new triage skill three days earlier.

He ran /triage on the issue and posted a full root cause analysis. Evidence, root cause, category, the most likely explanations. It showed the source data had stopped extracting: hourly records normal, then a sharp drop, then near zero.

I didn’t see any of it until Sunday morning. I was packing for the airport, a company offsite in Croatia starting that day. Nobody on my team was around. So instead of waiting, he took the first step himself.

At the airport, over lunch, fighting a dying laptop battery, I copied his root cause analysis into Claude Code. I asked it to investigate further and open a PR for the fix. I merged it that day.

I only learned afterwards that he’d had the same fix in mind. He’d asked Cursor to draft the PR. He just didn’t feel confident enough to ship it.

He had the knowledge. The skill gave him that. What he didn’t have was the authority to merge it, or the appetite to own it if it broke. The method had transferred. The mandate to act on it, and the responsibility for getting it wrong, had not. That’s the part I never thought to package, maybe because I’m not sure it packages at all.

My first reaction was: this works. My second, honestly, was disappointment that he didn’t just ship it.

What I want is to wake up to “I found the problem, here’s the fix,” review it, and have it done. I don’t need to own every fix. But handing someone the method is not the same as handing them the authority and the confidence to use it.

I could have written these skills earlier.

The reason I didn’t is uncomfortable. When a messy data issue shows up, people ask me, and I can usually figure it out.

I’ve been around long enough to know where the weirdness hides. Late sources. Models that oversimplify reality. Definitions that changed three times. Tables people trust too quickly. I move fast because I have the muscle memory: find the issue in aggregate, drill down to the lowest useful granularity, fix it there, repeat until the remaining movement is explained.

Not everybody has that, and that’s fine. Some people are stronger architecturally. Others are better at solving a problem once it’s scoped. What I’m good at is the scoping, the part where you haven’t yet figured out what kind of problem you’re looking at.

It feels useful because it is useful. The next issue comes in, it looks different enough that someone asks again, I answer again, and the work moves.

AI changed that. You can distribute a skill like this far more easily than before. The knowledge was never secret. The problem was that I was still the easiest way to get to it. I was the interface. Now there’s a realistic path to packaging what I do into something people can invoke without me in the room.

That’s a strange thing to do. You take something that lived in your head and make it usable without you. Your shortcuts, your checks, your feel for when something is probably fine versus when you need to keep digging.

Part of me wants to call that progress. Another part knows that when you’re the person everyone asks, your value is obvious. You know the weird edge cases, which source to distrust, what normal looks like. That’s expertise. It’s also status. Writing the skill means the next person gets further before you show up. In the best case, you’re not needed at all.

You built the road. People drive on it. Nobody thanks the road.

That’s the job, though. At least it should be.

But the skill didn’t remove the bottleneck. It moved it. The analyst got all the way to the fix and still didn’t ship. What stopped him wasn’t knowledge. It was that the fix wasn’t yet his to make.

The second story showed me the other half of it.

A billing issue came back, one we thought we’d fixed. It landed in the evening and I picked it up around ten the next morning. We were mid-migration between billing systems, and the complexity sat in the gap between them.

I had the triage skill, so I used it. I ran it, it didn’t find it, I told it to look harder. Go deeper. Continue. There were so many factors in play that it never figured it out.

It narrowed things down. I got a CSV of where the issue might be. I handed it to a colleague for a fresh pair of eyes and asked them to go deeper. They’re an AI skeptic, more of a data artisan. They took the manual route and found it.

The skill narrowed the search space. The last step still needed someone who distrusted the answer enough to keep looking.

Here’s the part I’m not proud of. My gut said it didn’t add up. The explanation we landed on would have implicated every monthly run, and the last few had been clean. I shipped the fix anyway because I was out of time. The skill scaled the diagnosis. It couldn’t make the judgment call, and under pressure my judgment lost to the calendar.

Two problems, two different gaps. The analyst had the diagnosis but not the authority to act on it. I had the authority and let a deadline overrule my own judgment.

Packaging what I knew didn’t make me unnecessary. It pulled apart what had always been bundled in one person: the knowledge, the judgment to know when an answer is wrong, the authority to act when it’s right, and the responsibility for what happens next. The skill could carry the knowledge. The rest defaulted to one person. To me. Again.

So the job isn’t answering more questions. It’s moving the judgment, the authority, and the responsibility toward the people who hit the problem first. That’s slower and less satisfying than writing a skill, and it’s the actual work.

You don’t stop being the bottleneck. You just move it. The work is making sure the same decisions don’t keep routing back through you behind a new interface.

There are two finance books on my desk. I bought them six weeks ago. Neither one is open.

We had to assess whether to replace one of our finance tools with an AI-native version. To judge that, I had to understand what we were replacing. That’s where I keep getting stuck.

The pressure moves faster than the thinking

A colleague told me a few weeks ago how investors had put it at another company:

Use AI. Find use cases. Personal agents in production by September. Or…

That exact phrasing is not everywhere yet, but the pressure underneath it is. Boards want movement. Leadership wants proof the company is not falling behind. The urgency is real. It is not crazy to look at what AI can already do and conclude your company should be moving faster.

But “use AI” is not a strategy. It is urgency without direction. A strategy says what should change and what should stay human. “Use AI” doesn’t answer that. It moves the question to someone else. Every team is asked to find use cases and show progress.

I knew the numbers, not the work

I’ve sat close to Finance since I joined Aiven, mostly through the data around it. I knew enough gotchas in the revenue data to feel like I understood the work. That made me overconfident.

Looking back, I think I knew it wasn’t enough. I was hoping it wouldn’t matter.

The numbers are the surface. Underneath them is the work itself: what Finance needs to produce, and where the manual work still hides. Almost none of it sits in a dashboard. Most of it lives in handoffs and tribal knowledge.

Nobody has the whole picture. Every person you ask gives you a different version depending on which part of the process they touch. You assemble it person by person, exception by exception, and the full picture only exists in the overlap between conversations.

That’s why the books are still on my desk. I bought them thinking I could catch up. Then I realized a finance textbook isn’t going to teach me what our team actually needs to produce, or which parts of the current process should disappear.

The vendor came in for a day and asked how things work on our end. They showed us what the tool does. What none of us could answer was whether it actually fits the work we run, not in a few hours.

The gap didn’t show up during the demo. It showed up when we started scoping the POC and had to think about requirements and success criteria. That’s when we realized how much of the process we couldn’t clearly describe.

The one that works

We already have an agent triaging data issues from our observability tool. I wrote the skills for it myself. Issue comes up, triage runs, it queries BigQuery, reads our dbt codebase, pulls PRs from GitHub, and figures out whether it’s a platform issue or a data quality issue. Then it routes the issue to a data engineering agent or an analytics engineering agent. Before that, every issue sat in a queue until someone on my team had time to look.

I could describe that process in thirty seconds. I knew the routing logic because I have been doing it myself for years. Putting it in a skill was straightforward. The agent works because the process was already clear enough to teach.

Same company, different team. One process I could automate because my team owns it. The other I can’t even scope properly because the knowledge isn’t mine.

The hard part comes before the tool

When you understand what the work has to produce, you can stop treating the current process as the thing to automate. You know the output and where judgment still matters. You can see where an agent might fit and where a human still needs to stay.

The danger is that you can move forward without that understanding. Buy the tool, push the agent live, and you have automated a half-understood version of the work. That is worse than the old tool, because the old tool did not pretend.

“Use AI” sounds like a tools problem. Tools problems become vendor problems. By the time the vendor is in the room, the harder question has been skipped.

The pressure to use AI isn’t going away. It shouldn’t. But I’m starting to think the real work isn’t picking the tool. It’s becoming the person who understands the work well enough to know what should change.

For the triage agent, I was already that person.

For finance, I’m not there yet. The books on my desk aren’t going to change that.

The next conversation is not with the vendor. It is with the people who know where the clean process breaks. If I cannot describe the work without the tool, I’m not ready to decide what it should replace.

The pile becomes the world

We’re in the middle of a migration. A small subset of customers is on the new system, but most are still on the old one. Nothing in the billing topic made that parallel state obvious. The old systems were in the knowledge repo, but they didn’t show up as part of the answer to how billing works today. So the model resolved the pile into one clean truth: the version with the most documentation.

A couple of weeks ago, I had a few spare hours on my hands and Karpathy had just posted his viral GitHub gist about the LLM wiki: basically, a way to build a knowledge base readable by agents. This is something I’ve been tinkering with in my role at Aiven anyway: how much of the company can we make understandable by AI? So I did the obvious thing and tried to build one for our internal context. I pointed AI at everything I could reach: Slack channels I had access to, Coda pages, dbt models, Omni metric models, system data, ownership notes, whatever I could feed into a script during one sitting. The output was a neat Obsidian repo, deployed to GitHub Pages as a Quartz site, with an MCP server on top. I could query it through Cowork and it felt like magic.

It broke on billing

The answer was confident. It explained the integrations, the ownership, the gotchas. It had a very good read on the new system. It even understood our new ARR model, which we had built ahead of go-live, even though that model wasn’t live yet. If a new hire had read that summary, they’d have come away with a clean mental model of how billing works at our company. That mental model would have been wrong about almost every customer we have today.

The model wasn’t hallucinating. It was reading the artifacts I’d given it and building the version of the company those artifacts most clearly described. And those artifacts were fresh. As fresh as they could be. Updated that day, backed by an active Slack channel, architecture docs, ticket trails, metric definitions. The problem was that most of them described the new billing system, because that’s the one people were actively building. The old one was just running. There were traces of it in the repo, but almost nothing that said: this is still the system most customers are on. That kind of information disappears in most companies. Not because anyone deletes it, but because nobody wakes up thinking: today I should document the system that has been running for years.

So it picked the version with the most signal and treated it as how things work today.

The map had no edge

This is how we write things down, anyway. Documents accumulate around the work that’s currently getting attention. Operations that just run don’t generate much of anything. And once you give that pile to a model, the pile becomes the world. It doesn’t know what is missing. It doesn’t know which parts live only in people’s heads. A human can catch one bad answer, sure. But that doesn’t fix the failure mode. The same thing happens everywhere, big and small: the model treats the documented version as complete, even when it’s only the part that happened to leave a trail.

The deeper failure isn’t that the answer was wrong. It’s that the model picked one version at all.

The honest answer to my billing question wasn’t the new system or the old one. It was that two systems are running in parallel. They don’t agree, and the right answer depends on which customer you’re asking about. The model had enough signal to hesitate. Instead it picked the cleaner story and presented it as how things work.

The complete wiki will never stay complete

A better wiki could have fixed this answer. In principle, sure. Someone could maintain a perfect billing page that captured everything: what is intended, what is running, the manual work filling the gaps, who handles the exceptions, and why they exist. The model would do better. This specific failure would probably go away.

But that is the point. The answer depends on someone maintaining the boring middle forever. The messy transition state. Which customers are where. Which model is live. Which process is designed, and which process people actually follow because the designed one doesn’t cover reality yet.

That page doesn’t exist for long unless someone keeps doing the invisible maintenance work. Not because people are careless, but because nobody wakes up thinking: today I should document the gap between the system and the business. Every company runs on that layer of human work. People know which exception still matters. They know who to ask. They know which dashboard is technically right but practically misleading. Most of that never becomes an artifact.

So the model read the pile and treated it as the complete picture. The failure is that it had no way to know where the map ended.

The next version has to assume the complete wiki will never stay complete. Whatever it is, it has to work with partial sources, conflicting sources, and suspiciously clean sources. When the company is in the middle of changing, it should not collapse that into one answer. It should say which layer it’s reading from, what seems live, what seems planned, and where it can’t see enough to be confident.

That is the next thing I want to figure out: not how to make the pile bigger, but how to show where it ends.

Greenstep HQ in Espoo, halfway through a hackathon. We had a fraud triage tool working. Four AI agents scored each customer. A human swiped through the queue with arrow keys. Our CFO walked over and asked why we were only thinking about defending revenue. Why not grow it too?

Same customer signals, different question. A usage spike is either fraud risk or expansion intent depending on the lens. We added expansion signals in the last hour and placed second out of 70 teams.

But we only applied the triage UI to the fraud side. The expansion signals sat in a dashboard.

Two weeks ago at an offsite dinner with people from our GTM team, we ended up talking about outreach. Cold calls, emails, all the usual pain. How hard it is to hit the right note. How AI-drafted messages are often close, but slightly off.

It clicked later. Same idea, but applied to leads.

It matters even more there than in fraud. Your average VP gets buried in outreach. AEs send dozens of emails a day and most of them go nowhere. If it’s not specific, it’s noise. AI can draft something decent. But a human can scan the draft, catch the wrong tone, and swap “your team’s usage is growing” for “you’re two weeks from hitting your API limit.” That’s what makes someone actually reply.

Everyone picks the wrong extreme

A company decides to “use AI” and usually picks one of two dead ends. Full automation and hope nothing breaks. Or give everyone a Claude subscription and some prompts and call it a day.

We keep asking AI to make the final call. It shouldn’t be making the call. It should be preparing the call for us.

People kept asking why we didn’t just fully automate the fraud tool. If the agents are right 85% of the time, what happens in the other 15%? In fraud, that’s suspending a legitimate account. In outreach, that’s sending a tone-deaf email to your biggest expansion opportunity.

Five seconds instead of fifteen minutes

So I built a demo. The version we should have built for the expansion opportunities at the hackathon.

Each card: company name, four signals, a draft email the AI wrote, and a send button. Press right to send, left to override, down if you want to see why the AI suggested this outreach. Most decisions take two seconds. Glance at the signals, read the subject line, press right. The ones where something feels off, you press down, read the evidence, maybe edit the draft inline.

I didn’t expect how little information you need when it’s laid out right. The fifteen minutes people spend digging through dashboards and CRM notes, that’s the AI’s job now. The human’s job is the five-second gut check.

The swipe flow: approve, override, or ask why.

Overrides are the point

This is the part most people miss. The override button matters more than the send button.

When someone presses left instead of right, that’s training data. You know which account, which signals, which human disagreed. Track enough of those and patterns show up. One agent gets overridden 40% of the time on borderline cases. Enterprise drafts get edited twice as often as mid-market ones. A specific AE overrides everything under a score of 70 because they know something the model doesn’t.

The override rate should drop over time. Not because people stop paying attention, but because the system starts to learn what they would have done anyway.

Right now it’s just a UI. I’ll update this post once we’ve put it to work and have real override data to look at.

What I’m still working out

Since the hackathon I notice the pattern everywhere. Support ticket routing, invoice approvals, screening resumes. Any queue where a human applies a gut check probably has some version of this hiding inside it.

Can overrides actually train the system over time? I think so, but I haven’t tested it yet.

But the thing I keep coming back to is that most of the time goes into the investigation, not the actual decision. Once you have a suggestion and the evidence sitting right there, the decision takes about five seconds.

I wrote up the implementation notes for the pattern here.

Crab Week, our company offsite in Šibenik, Croatia. We were standing near the hotel lobby talking about AI. I showed them Paperclip, a tool I’ve been building with. One agent delegating work to a dozen others, creating PRs while we watched. Nobody said anything. I’d been expecting questions. How does it handle errors. How do you keep the agents focused. We moved on, talked about other things. But what I kept thinking was: how do I get them to see what this could do for them, not to them.

Most people think AI threatens jobs. More often, it threatens identities.

In the days after, I kept having similar conversations. A colleague told me they were thinking about leaving for an AI-native company before they miss the boat. I thought they might be right. We’re not moving fast enough. But they weren’t really chasing speed. They knew they had to reinvent, and didn’t think they could do it here. They wanted to be somewhere figuring out what work actually becomes.

Managers know this feeling

I’ve been struggling with this for years.

It reminds me of when I became a manager. You get promoted because you’re good at the work. Then you’re supposed to stop doing it. Except you’re also the one who knows why it works in the first place, and that doesn’t transfer in a handover doc.

Our data platform team is four people. I still fix things at odd hours that I should be delegating. It’s faster, I know the code, and I tell myself I’m just unblocking the team.

That’s the tension. Let go and you lose touch. Hold on and you become the bottleneck.

AI just did this to everyone at once. And now everyone’s asking the same question managers have been asking for years: if I’m not the one doing the work, what am I?

So I made my team try it

I was doubting all week whether I should have the team try it. But after all the hype during our offsite about practical use of AI, it felt right. They needed to see Paperclip for themselves.

Four data and analytics engineers, each managing their own agents. Within an hour they’d hired a dozen agents and couldn’t get them to work together. One person wanted more governance. Another wanted to move faster and hire more. They clashed. They ended up acting as a board of directors, arguing about the goals of their company.

Nobody was writing SQL. Everyone was managing. The agents kept going, PRs kept getting created, needing more guidance than anyone expected. My team was the bottleneck for direction and quality control. The agents didn’t have enough of our tribal knowledge, the same way a new hire doesn’t.

I let it sit for a while after that. Let them think about what they’d just experienced. We’re not going all in overnight. Though we already have an agent triaging data issues from our observability tool, freeing us up to focus on what actually needs us.

The impossible balance

I’ve always identified with getting things done fast. I still do. AI makes me even faster.

But that’s also the trap.

If I only use it to move faster myself, nothing really changes. I’m still the bottleneck. Just a faster one.

The harder part is getting the team there too.

That means letting go of the parts of the job that feel safest. Writing the query. Fixing the issue. Knowing the code path. Being the person who can always make it work.

Those things still matter. But they can’t be the whole job anymore.

In the Paperclip exercise, the team didn’t need to write more SQL. They needed to decide what the agents were trying to do, what good looked like, where to slow down, and when to stop.

That’s the part I’m still figuring out.

You can find Paperclip on GitHub