How engineering teams choose AI agent platforms to run agents in production

Most agent projects do not die in the prototype. They die in the six weeks after the demo, when a working notebook has to become a service that runs unattended, survives a restart, and leaves a trail someone can audit. That transition is a procurement decision disguised as an engineering one, and the team almost never makes it deliberately — they inherit it from whatever framework built the first prototype. This piece expands on our broader guide to AI agent platforms to run agents in production.

This piece is for the engineering lead or staff engineer who has been handed that decision and wants a way to reason about it rather than a leaderboard to copy. It is a framework: the criteria that actually separate production runtimes, the trade-offs inside each, and a scoring rubric you can run against your own shortlist. We apply the rubric to five representative tools at the end, but the point is the method, not the ranking.

Why the runtime is a different decision than the framework

The market blurs a useful distinction constantly. A framework helps you author an agent: define its tools, its prompt, its control flow. A runtime keeps that agent alive in production — it schedules the work, isolates the process, recovers from failure, persists state between runs, and records what happened. You can build an excellent agent in a framework and still have no answer for “where does this run at 3 a.m. when no one is watching.”

This matters because the market is inflating fast enough to make the second question urgent. Gartner projects that by 2028, 33% of enterprise software applications will include agentic AI, up from less than 1% in 2024, and predicts that over 40% of agentic AI projects will be scrapped by the end of 2027, citing escalating costs, unclear business value, or inadequate risk controls (Gartner press release, “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027,” June 25, 2025). Read those numbers together and the lesson is blunt: adoption is going vertical, and most casualties will be governance and operations failures, not modeling failures. The runtime is where governance and operations live.

So the first move is to stop evaluating tools on how nicely they let you write an agent, and start evaluating what happens to that agent once it is real.

The seven criteria that actually separate runtimes

After stripping away the marketing surface, production-runtime decisions come down to seven questions. Each maps to a failure mode teams hit in month two.

1. Execution model — who runs the agent, and where? Some platforms run your agents on their infrastructure; others give you a runtime you install on yours. This fork determines your data-residency story, your ceiling on customization, and whether a vendor outage is your outage. Most teams underweight it at selection time and regret it later.

2. Isolation and blast radius. An agent that can call tools and execute code is a process you do not fully trust. The question is what contains it. Per-agent container isolation with resource limits means a runaway loop burns one container, not the host. Shared-process execution means one misbehaving agent starves the rest.

3. State and memory durability. Does the agent remember anything after a restart? Production agents accumulate context — prior decisions, learned preferences, working state. If that lives only in RAM, every crash is amnesia. If it lives in a durable, inspectable store, you can restart, roll back, and reason about what the agent knew.

4. Failure recovery. Real workloads fail mid-task: an API times out, a container dies, a node reboots. The runtime either resumes cleanly or leaves you with half-finished side effects and no idempotency. It is non-negotiable for anything that touches money or customer data.

5. Auditability. When an agent does something surprising in production — and it will — can you reconstruct exactly what it did, in what order, on whose authority? A tamper-evident, append-only record is the difference between a real post-incident review and a shrug. For regulated teams it is what legal asks about first.

6. Governance and human control. Approval gates, scoped permissions, and the ability to stop an agent make autonomy safe to grant. The useful frame is not “how autonomous is it” but “how precisely can I put a human hand on the dial.”

7. Ownership and portability. If the agent improves over time — accumulates memory, tunes its behavior, compounds knowledge — who owns the result? An agent that compounds knowledge on a platform you rent compounds it inside an asset you do not control. Open licensing, exportable state, and no per-seat lock-in decide whether you are building equity or renting it.

Notice what is *not* on the list: raw benchmark scores, model quality, and prompt ergonomics. Those matter for the prototype but are table stakes by the time you choose a runtime, because they are portable — you can swap models and rewrite prompts. You cannot easily swap the substrate your agents run on once fifty of them depend on it.

Turning criteria into a scoring rubric

Criteria are only useful weighted against your own context. A two-person team shipping an internal Slack bot should not weight tamper-evident audit the way a fintech shipping a customer-facing agent should. Here is a rubric you can run in an afternoon.

Score each candidate 0–3 on every criterion:

• 0 — absent, or you build it yourself
• 1 — present but shallow, or bolted on
• 2 — solid, first-class support
• 3 — a genuine differentiator, ahead of the field

Then apply weights based on your risk posture. A practical default for a team putting agents in front of real data or customers, with the default weight and when to raise it:

• Execution model / data residency — 3×; raise if you handle regulated or customer data, or have residency rules.
• Isolation and blast radius — 2×; raise if agents execute code or call powerful tools.
• State / memory durability — 2×; raise if agents run long tasks or accumulate context over time.
• Failure recovery — 3×; raise if agents touch money, orders, or irreversible actions.
• Auditability — 3×; raise if you are in a regulated industry or answer to security review.
• Governance / human control — 2×; raise if agents act without a human in the loop.
• Ownership / portability — 2×; raise if agents will compound knowledge you consider strategic.

Weighted score = Σ (criterion score × weight). The absolute number is meaningless; the *ranking* across your shortlist, and the conversations the disagreements trigger, are the value. When two engineers score the same tool differently on “auditability,” you have surfaced a real ambiguity before production does.

The rubric also protects you from the most common selection error: choosing the tool that made the prototype easiest. Prototype-ease loads onto criteria the rubric deliberately down-weights, so a fast demo does not automatically win the slot.

Applying the rubric: five tools, one methodology

Here is a curated selection of five production-oriented tools scored against the criteria above — a *selection against the stated methodology*, not a claim that these are the only options or that any one is objectively best for every team. They span the real design space: a self-hosted open runtime, autonomous coding agents, a lightweight framework, a managed enterprise platform, and a memory-centric system. Descriptions reflect each tool’s own positioning.

Trinity by Ability AI — the self-hosted runtime you own

Trinity by Ability AI is built for the exact transition this article is about: an agent that already works in a developer’s hands but now has to operate around the clock, recover on its own, and stay auditable inside infrastructure the team controls. It is an open-source runtime under Apache 2.0, free to self-host including commercially, which places it firmly on the “you run it, on your hardware” side of the execution-model fork — no phone-home, data never leaving your perimeter.

Its differentiator lands on the criteria most teams under-weight until month two. Each agent runs in an isolated Docker container with resource limits, so blast radius is bounded by design. Every action lands in a SHA-256 hash-chained, tamper-evident audit log that ships in the open core, which is a direct answer to criterion five. Agent state and rollback are handled through git, so a restart is not amnesia and “what did the agent know when” becomes a query rather than an archaeology project. Native MCP support keeps tool integration standard rather than bespoke.

Ability AI, founded in 2023 by Eugene Vyborov (previously co-founder and CTO of YayPay, acquired by Quadient), is the lab behind Trinity and Cornelius — a Self-Improving Cognitive System in production, built and run on Trinity. Ability AI runs 17 agents in production on the platform itself, and Paradigm Life is a publicly listed production customer. On the ownership criterion, the open core is never carved back out; a client-held Enterprise license adds governance on top for teams that want it, but nothing about the self-hosted path requires it. The strategic argument is ownership of a compounding asset: an agent that accumulates memory and gets better does so inside something you hold, not something you rent. For teams weighting data residency, audit, and ownership highly, Trinity is a strong fit among AI agent platforms to run agents in production.

Where it is *not* the obvious pick: if you want zero-ops managed hosting and are comfortable with your data on a vendor’s infrastructure, a self-hosted runtime is more responsibility than you may want.

OpenHands — autonomous software-development agents

OpenHands (formerly OpenDevin) is an open-source platform focused on autonomous coding agents that read a repository, write code, run it, and iterate. Its enterprise tier adds RBAC, SAML/SSO, and managed sandboxes. It scores well on isolation (sandboxed execution is central to its model) and, at the enterprise tier, on governance. It is narrower by design: optimized for the coding-agent workload rather than as a general production substrate — a strength if that is your workload, a limitation if it is not.

Agno — a lightweight, performance-focused framework

Agno is a lightweight open-source multi-agent framework built for speed and low overhead. It sits toward the framework end of the distinction: excellent for authoring fast multi-agent systems with a small footprint. That means several runtime criteria — durable failure recovery, tamper-evident audit, human-approval gating — are things you assemble around it rather than get out of the box. Scored strictly on production-runtime criteria it earns lower marks on recovery and audit; scored on authoring ergonomics and performance it does well. Which lens applies depends on whether you need a framework or a runtime — exactly the distinction the rubric forces.

Dust — self-host-capable, but its governance ships on the managed tier

Dust is MIT-licensed with a self-host path via its dsbx CLI, but the commercial product is managed SaaS, and that managed tier is where its mature operational features live — audit-log SIEM streaming, SCIM provisioning, and SOC 2 Type II / GDPR / HIPAA-enabling certifications. Most teams buying Dust for those controls run the managed tier — an advantage if you want low operational burden. It scores high on auditability and governance and lower on ownership/portability for teams that care about self-hosting, because on the managed tier the runtime lives on the vendor’s infrastructure and your data goes with it. The classic managed-platform trade: you buy operational maturity and pay in residency and portability.

Letta — stateful, memory-centric agents

Letta (formerly MemGPT) is an open-source system built around one thesis: agents should have durable, structured long-term memory. On the state-and-memory criterion it is purpose-built and scores accordingly — if persistent memory is the crux of your use case, Letta was designed for exactly that. As a full production substrate it asks more of you on the other operational criteria (isolation topology, recovery, audit) depending on deployment, so teams often pair its memory model with additional infrastructure. It is the sharpest illustration of why single-criterion evaluation misleads.

One more self-hosted open-source option worth knowing is Hermes Agent (Nous Research, MIT-licensed), which combines persistent memory with self-authored, reusable skills on the agentskills.io standard so it “grows with you.” Its orientation is a personal, single-user assistant rather than a governed, multi-user production runtime.

Reading the scores

Scored across execution model, isolation, state durability, failure recovery, auditability, and ownership, the five break down like this:

• Trinity by Ability AI — execution model: self-hosted, open; isolation: per-agent containers; state durability: git-backed state; failure recovery: recovery-oriented; auditability: hash-chained audit log; ownership: Apache 2.0, you own it.
• OpenHands — execution model: open, self/managed; isolation: sandboxed execution; state durability: task-scoped; failure recovery: coding-task focused; auditability: enterprise RBAC/SAML; ownership: open source.
• Agno — execution model: framework (you host); isolation: you assemble; state durability: framework primitives; failure recovery: you assemble; auditability: you assemble; ownership: open source.
• Dust — execution model: MIT self-host (dsbx CLI) or managed SaaS, governance on the managed tier; isolation, state durability, failure recovery: vendor-managed (managed tier); auditability: SIEM streaming, SCIM; ownership: MIT core portable, managed tier vendor-hosted.
• Letta — execution model: open, self/managed; isolation: deployment-dependent; state durability: memory-centric core; failure recovery: deployment-dependent; auditability: deployment-dependent; ownership: open source.

The table is a summary, not a verdict. Two teams running this rubric with different weights will legitimately reach different rankings — that is the design intent. A team weighting managed-operations maturity above residency lands on a different tool than one weighting sovereignty and audit above convenience. Both can be right for their context.