How to Add Coding Runtimes to an AI Agent Web Service#

2026-04-19

When you start designing or building an AI agent web service, one question shows up surprisingly early:

“If these agents are supposed to do real work, where does their coding ability actually come from?”

At first, it is tempting to think the answer is straightforward. Call a strong model API, add file editing, let it run shell commands, execute tests, and package the results. But once you look more closely, the problem expands fast.

• repository checkout
• file reading and editing
• shell command execution
• test and build loops
• git diffs and pull request preparation
• secret injection
• network restrictions
• timeouts and budget limits
• retries, logging, and recovery

In other words, a “good coding agent” does not come from a smart model alone. It also needs execution environments, tool orchestration, approval flows, isolation, and observability.

That is the point where I reached a fairly simple conclusion: for a solo builder, trying to implement the entire coding capability stack from scratch is usually not a rational choice. It is often far more practical to ask how existing coding agent runtimes and frameworks can be adapted to your product context.

The terminology needs to be cleaned up first#

One reason this space feels confusing is that very different layers of the stack are all described as “agents.”

I find it helpful to separate them like this:

At first I wanted to call all of this “AI agent frameworks,” but for this discussion, coding agent runtime or developer agent execution engine feels much more precise.

The real question here is not “Which agent framework should I use?” It is closer to “Which engine should actually perform development work inside my service?”

Personal coding tools can feel awkward when moved directly into a web product#

This is where the tension starts.

Tools like Claude Code, Codex, Gemini CLI, and Cline are generally optimized around one developer working in their own environment. If you drop them into a web service without changing the framing, the result can feel off.

The reason is simple.

• Personal tools assume “my repo” and “my terminal.”
• Web services assume “many users” and “isolated execution.”
• A personal tool can fail locally with limited consequences, while a service also has to handle billing, permissions, audit logs, and tenant isolation.

Because of that, I think these tools fit better as internal execution engines than as the primary identity of the product.

Put differently:

• Awkward approach: let users operate something that is basically a hosted IDE
• Natural approach: let users manage goals and approvals while coding runtimes do the work in isolated workspaces behind the scenes

That distinction matters. The first approach makes you compete with IDEs. The second lets you build a service for operating developer agent teams.

The product should own the operating structure, not the entire coding capability#

I think it helps to step back and ask where the moat of an AI agent web service really is.

In many cases, users are not only asking for “the strongest model.” What they actually want is a combination of things like:

• who should do the work
• how roles should be divided
• which tasks should be auto-approved and which should require human approval
• how budget limits should be applied
• how work history should be preserved
• how outputs should become durable assets

That means the core value of the product is often less about coding ability itself and more about organization and operations.

So using external coding runtimes is not merely a compromise. It can be a deliberate way to focus on the layers the product truly needs to own.

Is this an SDK integration problem or a runtime integration problem?#

At first glance, that sounds like a clean either-or question. In practice, I do not think it is.

My current answer is:

The right answer is runtime-first, with SDK-driven control.

What that means is:

• the runtime performs the actual work
• the SDK is one way to control that runtime programmatically

After all, coding agents do not just return text. They operate inside a working directory. They clone repositories, inspect files, make edits, run tests, generate diffs, and stream logs. However it is packaged, that implies some kind of workspace runtime.

So from a product architecture perspective, the more accurate framing is usually:

• integration unit: not the SDK, but the workspace runtime
• control method: SDK, CLI, app-server, or headless mode depending on the product

Different products land in different places on that spectrum.

• Codex appears relatively clear in its SDK and app-server story.
• Claude Code may offer SDK and headless options, but still feels runtime-centric.
• Gemini CLI is especially interesting because it now has an official SDK.
• OpenHands and OpenCode feel closer to service-style architectures from the start.

The most natural architecture is Control Plane plus Execution Plane#

For an AI agent web service that wants to use coding runtimes, I think the cleanest mental model is a two-plane design.

1. Control Plane#

This is the heart of the product backend.

• create jobs
• approve or reject work
• manage status
• track spend
• enforce permissions
• expose logs
• persist history

In other words, it decides what should be done, by whom, and under which policies.

2. Execution Plane#

This is where the work actually happens.

• repository checkout
• coding agent execution
• test and build loops
• diff generation
• artifact upload

In other words, it is the environment that carries out the assigned work.

This separation matters because it cleanly divides the responsibility of the product backend from the responsibility of the execution runtime. That in turn makes it much easier to swap a runtime later, move from ECS to EKS, or add another vendor without destabilizing the product itself.

Job-level isolation is usually better than user-level isolation#

A natural first thought is:

“Why not give each user one long-lived runtime?”

But for coding agents, job-level isolation is often the safer default.

• one job = one runner
• reuse it briefly if the same session needs continuity
• tear it down when the work is done

This has several advantages:

• it reduces cross-user file conflicts
• it makes secrets and environment variables easier to isolate
• it improves budget and resource tracking
• one runaway task is less likely to affect unrelated work

So the realistic model is usually:

• the Execution Plane is shared infrastructure
• but it launches isolated runners per job or per session

The infrastructure does not need to be overbuilt on day one#

That brings up the next question:

“Where should these runners live? ECS? EKS? EC2?”

My current view is that ECS/Fargate is the most practical starting point.

Why?

• it is easy to launch isolated tasks per job
• isolation is reasonably clean
• operational complexity stays relatively low
• for a solo builder, Kubernetes is often premature

When does EKS become more attractive?

• when concurrent workspaces increase a lot
• when long-lived sessions become common
• when multiple workspace types appear
• when warm pools and more advanced scheduling start to matter

So the balanced path seems to be ECS/Fargate first, EKS later if scale and complexity actually demand it.

Would that migration be dangerously risky? I do not think so, as long as the boundaries are designed correctly. State, logs, and artifacts should live outside the container, and the Control Plane should know as little as possible about the details of the execution environment.

Ideally, the product only knows something like launch_workspace(), while the implementation behind it can evolve from a Fargate task to a Kubernetes pod.

Which runtimes are worth evaluating first?#

While researching this topic, these are the names that stood out to me.

The strongest initial candidates#

Useful secondary references#

Adjacent but slightly different categories#

The key question is not simply which one is most popular. It is which one can enter your product through a clean architectural boundary. On that front, Codex, Gemini CLI, OpenHands, OpenCode, and Cline seem especially worth watching.

Three takeaways that mattered most to me#

As I thought through this more carefully, three points felt especially meaningful.

1. The product can be an operating layer, not an IDE#

At first, it is easy to imagine “a web service where every user gets their own development environment.” But after thinking it through, I do not think users necessarily need to manage that environment directly.

It may be more natural for them to focus on:

• describing goals
• setting approval policies
• reviewing budgets
• changing team composition
• inspecting outputs

That means the product does not have to become a hosted IDE. It can become the layer where developer agents are organized and operated.

2. Coding ability is not the entire moat#

Good coding runtimes are likely to become more common, not less. If that happens, the product advantage shifts away from “having the one best runtime” and toward how those runtimes are composed into a governed system.

From that perspective, what the product truly owns is:

• which work can be auto-approved
• which work requires human review
• which agent plays which role
• which outputs get stored and reused

That is a much more product-shaped problem than simply picking a model.

3. The first use case should stay narrow#

Systems like this naturally invite huge ambition. But the first use case should probably stay much smaller.

Something like this is already strong enough:

1. connect a GitHub repository
2. select one issue
3. run the fix inside an isolated runner
4. show test results and a diff
5. request user approval
6. create a pull request

If that flow works well, it becomes much easier to layer in reviewer agents, QA agents, budget policies, and long-term memory later.

What the product should actually own#

I keep coming back to a fairly simple answer.

What an AI agent web service should own directly:

• user experience
• approval structures
• team coordination
• work history
• cost policies
• security and isolation

What it probably does not need to own from scratch:

• the entire coding agent loop
• the full file-editing and shell-execution harness
• every runtime control mechanism

That makes the most realistic split feel something like this:

• coding runtimes are the backend execution engines
• the product is the operating layer

Closing thought#

One of the biggest traps in this space is the feeling that “if the capability is core, I must build all of it myself.” But coding agents are evolving quickly, and new runtimes, SDKs, and managed platforms keep appearing.

In that environment, the more useful question may not be:

“Can I build the best coding agent myself?”

but rather:

“Can I organize existing coding runtimes inside my product context better than most people would?”

Right now, that second question feels much more productive to me.

And the first answer to it may be this:

The best product may not be the one that puts a coding runtime front and center. It may be the one that best weaves together people, agents, approvals, and execution behind the scenes.