In my previous post, I talked about migrating from Claude Code to Pi. Since space was limited, I only covered the basics. In this article, I will explain the specific setup in detail.

First, let’s look at the global configuration layout.

1
2
3
4
5
6
7
8
9
10

ls -al ~/.pi/agent  
    608 Jul 18 16:49 .  
     96 Jul  4 17:21 ..  
     28 Jun 23 16:25 AGENTS.md -> /Users/ctw/.claude/CLAUDE.md  
   1856 Jul  6 18:05 APPEND_SYSTEM.md  
     63 Jul  7 10:09 agents -> /Users/ctw/Workdir/ctw-skills/plugins/ct/agents  
    118 Jul 16 19:28 mcp.json  
     27 Jun 25 15:42 prompts -> /Users/ctw/.claude/commands  
    546 Jul 17 14:47 settings.json  
     63 Jul  7 10:15 skills -> /Users/ctw/Workdir/ctw-skills/plugins/ct/pi-skills

There is a core configuration file in here called settings.json.

1
2
3
4
5
6
7
8
9
10
11
12

{  
  "skills": [  
    "~/.claude/skills"  
  ],  
  "packages": [  
    "npm:pi-mcp-adapter"  
  ],  
  "extensions": [  
    "/Users/ctw/.nvm/versions/node/v22.23.0/lib/node_modules/@earendil-works/pi-coding-agent/examples/extensions/subagent",  
    "/Users/ctw/.nvm/versions/node/v22.23.0/lib/node_modules/@earendil-works/pi-coding-agent/examples/extensions/permission-gate"  
  ]  
}

This file defines where skills are inherited from and what plugins to install. Pi has very few core components. We need to customize most of it based on our needs. For example, MCP and subagents are the two most critical features, and they both run as plugins.

As we can see from the configuration, we can directly reuse most settings from Claude Code. However, a few interesting details are worth mentioning.

1. Pi does not have commands. It only has prompts. In Claude Code, commands are actually just skills with disable-model-invocable enabled. But in Pi, there are only prompts, which are just plain text. The concept is basically the same, so I linked my commands folder directly to prompts.
2. I actually use two layers of skills. The first layer is inherited from Claude Code, which is defined in settings.json. The second layer consists of skills unique to Pi, placed in my Pi skills directory. I had to do this because Claude Code and Pi handle subagent logic are entirely different. Claude Code can spawn a subagent to do work using simple prompts. This approach does not work in Pi, so I had to write them differently.
3. The MCP configuration is also inherited from Claude Code, but the approach is slightly different. Pi can import settings directly from Claude Code and then override specific variables. I chose to override idleTimeout because the Pi MCP plugin has an idle timeout setting. This setting makes the Chrome MCP disconnect if it stays idle for a while. Reconnecting Chrome MCP requires manual confirmation every time, so I decided to disable the idle timeout entirely.

1
2
3
4
5
6
7
8
9
10

{  
  "imports": [  
    "claude-code"  
  ],  
  "mcpServers": {  
    "chrome-devtools": {  
      "idleTimeout": 0  
    }  
  }  
}

Finally, we have APPEND_SYSTEM.md. As I mentioned last time, we can write system prompts in this file if we only want them applied to Pi and not Claude Code.

I defined two core behaviors in this file.

1
2
3
4
5

## Image Reading  
The default model cannot see images. When you need to read or understand an image (screenshot, error trace, UI mockup, diagram, chart, photo), delegate to the `vision` agent via `subagent`. Do NOT pass image files to `read` or refuse image tasks. Instead, delegate to `vision`.  
  
## General-Purpose Delegation  
Use the `runner` agent via `subagent` for independent verification/recalculation, heavy batch work, or any well-specified self-contained job. Same tools as the main session, no conversation history, no specialty. Do NOT use it for PR review (`pr-reviewer-*`) or images (`vision`) because those have their own agents.

The first one is vision. Since my default model does not support image recognition, I wrote a vision subagent. This allows the main session to ask for help whenever it encounters an image.

The second one is runner. Claude Code has a general-purpose subagent. Whenever the main session wants to do minor tasks, it calls the general-purpose subagent by default. I did something similar in Pi. I wrote a runner subagent so the main session has a helper ready to take over tasks when delegation is needed.

At this point, I have fully migrated Claude Code’s capabilities to Pi. As I mentioned before, I do not need over-the-top features. I only migrated what I considered essential.

Of course, there are still many details to fine-tune after the migration. A good example is the subagent-related skills I just mentioned. In Claude Code, we can create subagents through prompts, but all those skills need to be rewritten for Pi. For Pi, we must clearly define those subagents in the subagents directory.

Even though it is easy to let the agent rewrite them, there are still some minor issues to work out in practice. I kept fine-tuning my Pi-specific skills over time. Overall, the entire migration only took about half a day to a full day.

Promoting My Side Project

When using Pi or Claude Code, I ran into a few persistent pain points.

1. When I run multiple agent tasks across many terminal tabs, I need clear notifications. I need to know when a tab is done so I can take over. This is easy to do in Claude Code when using the Warp terminal because Warp has custom features for Claude Code. However, Warp is not going to build custom features for an open-source project like Pi.
2. While the subagent extension in Pi is feature-rich, it does not show the execution process. It only shows the final result. Claude Code has Agent Teams, which lets us see the details of each subagent, but Pi lacks this feature.
3. Claude Code has several remote control options. These include built-in solutions, which require a personal account and do not support API keys, and third-party options like Happy Coder. However, I have not been able to find a good remote control solution for Pi.
4. Warp supports tabs and groups, meaning we can group related sessions together. However, Warp groups do not support collapsing or expanding. When there are too many tabs, it is still hard to find what I need to focus on.
5. I need clear labels to know exactly what session is running in each tab. Warp allows us to name tabs and groups, but I still have to name them manually. Doing this for short-lived sessions feels highly inefficient. I believe session names should be generated automatically.

To address these issues, I decided to develop a plugin myself. This is how my side project was born.

It has a few key features.

1. It is a web-based service, so there is no need to install any mobile apps. Even though it is a local service, we can easily expose it and set up basic OAuth using tools like ngrok.
2. Since it is a web service, it supports parallel work across multiple devices. For example, I can manage session A on my computer browser while handling session B on my phone. We can even control the same session simultaneously from both a phone and a computer without any locking issues.
3. It has its own storage unit, so it can save session grouping details. It also supports automatic session naming, though this requires configuring an LLM API key.
4. I use this project every single day. I will keep updating and maintaining it regularly. It also has very high test coverage.

Here is a preview of the main interface. I will add new features over time as I see fit. Contributions are always welcome.

Wrap Up

I have used most GUI and CLI agent tools on the market. Right now, I am most satisfied with Pi. This is not because Pi is perfect or has everything. Quite the opposite, Pi is simple and focuses only on the core, while opening up plenty of APIs for third-party integration.

If we want a specific feature, we can almost always build an extension to solve it. For example, my side project basically gives Pi a full GUI interface, complete with remote control capabilities.

Pi’s stable yet open core is exactly what makes this so easy.

As for model capabilities, whether it is Opus 5, Fable 5, GPT-5.6, or GLM-5.2 (which I mainly use lately), they all have their pros and cons. I will not judge their performance too much here. But without a doubt, GLM-5.2 is currently the most cost-effective option for me.

Originally published on Medium

I recently migrated my AI toolchain again. The main reason was cost. My daily setup used to be Claude Code with Opus 4.7. Honestly, the cost was just too high. I did delegate specific tasks to cheaper subagents like Sonnet or Haiku. However, most of my usage still ended up on Opus.

Following a “recommendation” from our company, I was asked to try a new toolchain. This new setup combines Pi with GLM 5.2.

I have been using it for exactly a week now. I wanted to share my thoughts on the experience.

In the past, migrating toolchains lacked a solid way to measure success. We usually relied on gut feeling. If things seemed to work and conversations went smoothly, we assumed there was no rework. These are highly subjective evaluations. We always lacked quantitative metrics.

Because of this, I built a special framework for this migration. This framework analyzes all my Claude Code and Pi sessions. It also sets up a semantic evaluation standard.

I will skip the boring details. Building things in the AI era is incredibly easy. The real value lies in the concepts and the know-how.

Here is how my framework works. It parses all the session files from Claude Code and Pi. These are mostly JSONL files. Then, it extracts the dialogue from both the agent and myself.

The system pairs the agent’s response with my reply. I use this pair to perform a pure semantic analysis.

I only look at this single pair without extra context. I want the semantic analysis to be as simple as possible. This back and forth interaction is the easiest way to see what my response means. We do not need complex reasoning or extra interpretation. It is just straightforward semantic analysis.

I categorize these pairs into the following types:

• fresh_task: This represents a new task with no related preceding assistant output.
• follow_up: This is a same task extension or supplement, not a correction.
• pushback_correction: The user is unhappy with the agent’s result and wants changes.
• steer_interrupt: This is a mid stream cut in. We infer this from the phrasing of both sources using an LLM. Examples include saying “wait,” “stop,” or “no, do X instead” while the agent is in the middle of a task. No offline ground truth signal exists in either format.
• acknowledge: The user is just acknowledging the message with no task intent.

With this data, we can calculate the revision rate. This rate shows whether I was satisfied with the agent’s output. In other words, it measures first try acceptance.

Based on the first week of testing, here are the results:

• The revision rate for Claude Code was 17%.
• The revision rate for Pi was 23%.

The data shows that Pi is noticeably worse than Claude Code. This actually matches my gut feeling.

On the other side, the cost difference is massive. Pi costs only one fifth of Claude Code.

This is a huge gap. It is especially true for power users like me. Paying 20% of the cost to get 80% of the performance is an incredible return on investment.

Migration Details

Now that we covered the background, let us look at some technical details. I have migrated AI toolchains many times. Over time, I built up a lot of experience moving between these tools. This migration took me less than half a day.

You can migrate Claude Code skills without any pain. Pi has built in migration tools for this. For Claude Code commands, I used symlinks to connect them to the prompts directory in Pi. I also mapped Claude.md directly to Pi’s AGENTS.md using a symlink.

I ran into two major issues related to my daily workflow. These were browser use and subagents. Neither feature is built into Pi. Fortunately, Pi offers official extensions that you can install directly. I used the official MCP extension to run the Chrome Dev MCP. I paired this with the subagent extension to build out the required capabilities.

There are a few things to note about subagents. First, because they run as extensions, Pi does not have built in discovery. You must place any subagent you need inside the agents directory. Then, you manually call them using prompts in the main session.

Porting the agents with symlinks is easy. However, manually calling agents is incredibly painful if you are used to Claude Code. To fix this, I used a built in Pi feature called APPEND_SYSTEM.md to extend Pi’s capabilities.

I mentioned that AGENTS.md maps to Claude.md. Since this is a symlink, I did not want to pollute Claude Code with subagent information. This is where APPEND_SYSTEM.md shines. It is a mechanism to expand the system prompt. It does not use a symlink. This allows me to add system prompts meant only for Pi.

Here is a concrete example.

1
2
3
4
5
6
7
8
9
10
11

# pi only additions (not loaded by Claude Code)  
  
## Image Reading  
The default model cannot see images.   
When you need to read or understand an image (screenshot, error trace, UI mockup, diagram, chart, photo), delegate to the `vision` agent via `subagent`.   
Do NOT pass image files to `read` or refuse image tasks. Instead, delegate them to `vision`.  
  
## General Purpose Delegation  
Use the `runner` agent via `subagent` for independent verification or recalculation, heavy batch work, or any well specified self contained job.   
Same tools as the main session, no conversation history, no specialty.   
Do NOT use it for PR review (`pr-reviewer-*`) or images (`vision`), since those tasks have their own agents.

With this setup, my Pi setup has the exact same subagent delegation capabilities as Claude Code.

Also, I mentioned the vision agent earlier. This helps make up for GLM 5.2’s limitations. Since it cannot process images directly, I needed to attach a subagent for image recognition. In practice, the experience is just like Claude Code. I paste an image, and the system still processes it. It simply delegates the task to an OCR capable model rather than doing it itself.

When we integrate all of this, Pi with GLM 5.2 easily competes with Claude Code with Opus 4.7. As for the fancy features in Claude Code, I never really used them anyway. I did not lose much.

Wrap Up

Claude Code and Opus are truly expensive. In this AI era, I believe most companies will look for ways to cut tool costs. Honestly, setting a hard cap on token budgets is a bad idea. It acts as a penalty for highly efficient power users who know how to leverage AI well.

The alternative is to find a replacement tool.

After a week of usage, Pi with GLM 5.2 does feel slightly worse. However, the gap is not too wide. You can interpret the revision rate difference of 24% versus 17% in a few ways.

By the way, these numbers come from my actual daily work. This was not a head to head comparison on the exact same tasks. I simply made a clean break at a specific point in time. I switched entirely from Claude Code to Pi. The comparison is based on my Claude Code performance before the switch versus my Pi performance during the first week after.

I applied a sliding window to my Claude Code sessions. The 17% rate is highly stable. Because of this, the comparison holds up well. I must add a disclaimer that this method is not academically rigorous.

Originally published on Medium

Stop letting AI hallucinate incident reports. Master the human-in-the-loop RCA

Lately, I have been reading a lot of AI-generated RCAs and incident reports, and I noticed a glaring problem.

They scream AI.

I am not talking about the wording, tone, or structure. I do not really hide my own AI usage anyway, since almost everyone uses AI to write reports these days.

What I mean by “screaming AI” is the entire context — specifically, how the author connects the visible symptoms to the root cause. This makes it incredibly easy to spot who is actually driving the investigation. Most of the time, it is obvious that the AI is the driver.

AI outputs have a specific trait: they only scratch the surface, whether we are talking about code or reports.

Let me give you a few obvious examples.

We have all seen an AI review a module or a feature during a production incident and aggressively nitpick a bunch of issues. It then assigns severity levels to these problems like “critical” or “suggestion” with total confidence. But these issues always share a common flaw: a complete lack of domain knowledge.

The AI does not know the history or the original reasoning behind the code. It just attacks the surface-level flaws. Can it solve the problem? Sure. Are these actually the real issues? Usually, no.

It is the exact same story with incident reports. The AI sees a memory spike and immediately calls it a memory leak. It sees a CPU spike and calls it CPU saturation. Are these real problems? Yes. Did it find the root cause? Usually, no.

When we perform an RCA and see a memory spike, our very first goal must be to find the underlying trigger. What actually caused the memory to spike?

However, the AI’s first instinct is always to tell a story. It explains why the memory spiked by piecing together random details that look correct and feel like facts. The metrics and logs it pulls might indeed highlight the reality of the memory spike, and its narrative might perfectly explain the process. But is it the root cause? Usually, no.

This is exactly what I mean by that AI vibe. It writes beautifully and tells a great story, but it falls apart under scrutiny.

So, How Do I Do It?

I break it down into two parts: fundamentals and technical tactics.

The fundamentals refer to the mental model of using AI. This means using a specific methodology to control the AI and guide it toward the correct result. The technical tactics refer to how we can leverage AI tools to get a better outcome.

Honestly, neither requires you to be a domain expert. We just need to execute the right steps and provide accurate judgment.

Fundamentals

Let me start with the fundamentals. I often talk about “systems thinking.” This was the most critical skill back when software engineering was done entirely by hand, and it remains an indispensable mental framework today.

But this is not knowledge we can pick up overnight; it is an accumulation of experience.

Because of that, I only teach two tricks and one simple rule of thumb.

When we receive an AI output, no matter what it is, we must first ask two questions:

1. Is this a guess, or is it a fact?
2. Is this a cause, or is it an effect?

The first question forces the AI to track down available evidence, such as logs, metrics, design docs, or code. This stops the AI from making up stories and ensures it uses real evidence as its building blocks.

We need to pay extra attention and ask this question whenever the AI gives an answer that sounds absolutely certain. Many times, its answers are just extensions of various assumptions, yet it presents them as absolute facts.

The second question is even more interesting. When an AI sees a symptom, it immediately tries to explain it. But explaining a symptom offers limited help. Take a memory spike, for example. Once the AI starts explaining it, the conclusion is almost always to scale up or scale out. But we do not want to know if we have enough memory. We want to know the reason behind the spike. Was it a poorly handled socket? A memory leak? Inaccurate metrics? Or something else?

We do not need a story. We need an answer.

When we ask whether something is a cause or an effect, the AI realizes it has been focusing entirely on explaining the result instead of finding the cause. This gives us a chance to make it correct its course.

It is easy to see that these two questions do not require any domain knowledge. Yet, we can use them to make the AI start discovering the domain context on its own. Those are the two tricks.

Now, here is the rule of thumb.

Begin with the end in mind!

Once the AI realizes it needs to understand the cause, it might gather a ton of evidence. But then it might start overthinking or getting stuck in a dead end due to a lack of context or depth.

How do we pull it out?

Let us go back to the most fundamental method: drawing a decision tree.

What is the end? The symptoms and phenomena we see are the end. That is what the very end of the chain looks like. What is the beginning? These symptoms serve as the starting point of our decision tree.

We map out every single possibility that could lead to this result. This becomes our first layer of subtrees. Then, we look at how many options lie behind each possibility, which forms the second layer. We keep repeating this until we exhaust all possibilities. This process does not actually consume many tokens because telling stories is what AI does best.

Once we have the tree, we start pruning. We take all the evidence we gathered and eliminate the nodes on the tree that are impossible. Alternatively, we can have the AI gather evidence based on these specific nodes.

In the end, we are left with only a few possibilities. This time, the AI’s chain of thought is constrained by the tree, preventing it from wandering off while still allowing it to be creative.

This is what it means to begin with the end in mind.

Technical Tactics

Once we have the right mental model, let us look at how we can make this process more stable and reliable from a technical standpoint.

I will use Claude Code as an example, but you can achieve similar results with almost any tool.

I mentioned those two tricks earlier, and their goal is simple: to make the agent understand that appearances can be deceiving. Beyond that, we can also use fixed workflows to make everything run smoother.

For instance, my favorite approach is to use the following prompt or skill right when the agent delivers a conclusion (or something that looks like one).

Assemble an Agent Team of three agents and hold a workshop. Their context must only contain raw evidence, with absolutely no inferences. One agent will support your viewpoint but must find supporting arguments strictly from this raw evidence. Another agent will oppose your viewpoint and must also find counterevidence strictly from this raw evidence. The final agent will cross-reference the conclusions and evidence from both agents and deliver a report.

Even though this approach leverages the Agent Team feature in Claude Code, these agents do not actually need to talk to each other. So even without an Agent Team feature, we can manually summon two opposing agents to give their conclusions, and then ask a third agent to cross-reference them.

This gives us an extra layer of defense before finalizing the conclusion. It allows us to check whether we or the agent fell into a bias due to self-suggestion.

Wrap Up

We must always remember that AI is an LLM. Every single word it speaks is based on probabilities and parameters, not facts. If it happens to state a fact, it is only because we constrained all the parameters enough to bring it close to reality.

“We cannot reach human-level intelligence by scaling up Auto-Regressive Large Language Models (LLMs). It’s a dead end… They don’t understand the physical world, they don’t have persistent memory, they cannot reason, and they cannot plan.” — Yann LeCun

This Turing Award winner points out that by relying solely on LLMs, we actually drift further away from AGI.

In this AI era, we must always remember the importance of keeping a human in the loop. The human is the driver. If we do not drive the investigation ourselves, the AI is just hallucinating.

Fortunately, using the fundamental and technical prompts mentioned above can make this task a lot less daunting. In fact, it actually makes it pretty simple.

Originally published on Medium