Hone's AI Engines Can Own Retention Only Where Buyers Keep a Holdout Group

On October 8, Hone introduced Engines along with a $60 million seed round from Benchmark and Index Ventures. The example assignment in CEO Moritz Stephan's launch post, improve customer retention, is a line a customer success leader could write on a whiteboard. The Engine onboards itself into the company's systems and works out what the goal requires. Then it keeps at it for weeks or months.

Bloomberg reported the round and a $285 million valuation for the five-month-old company, a figure that came from Hone. Cognition and Modal are named as early customers. Bloomberg reported that Hone's work at Cognition is in go-to-market operations and that the company aims to automate work such as evaluating sales leads.

Index's investment post, by partners Shardul Shah and Mark Xu, describes an Engine given a role such as procurement coordinator or outbound rep. The Engine is tested against prior decisions before it goes live, and it acts on events rather than prompts. The post ends on the bet that value will go to whoever owns the layer between the model and the enterprise. Hone's homepage lists five sample Engines, each with an outcome attached: higher inbound conversion, lower vendor spend, lower credit losses, stronger adoption and retention, and more customer commitments delivered on time.

Hone pitches owning a metric, judged by the P&L

Hone's pitch suggests buyers could contract out a metric, paying an agent to own retention the way they'd pay an agency to own a campaign. Nothing Hone has published says the contract works that way, and pricing isn't disclosed. The launch post's section on KPIs says the company's "ultimate eval is the P&L" and suggests customers judge an Engine by asking what it got done this week. That standard fits managing an employee better than billing a vendor.

The one hard number Hone has published measures something else. In its first research post, the engineering team reports on 125 internal scenarios.

Replaying a proposed change to an Engine's own harness in simulation before promoting it cut regressions from 33% to 5%. The test checks whether the agent breaks its own earlier behavior when it rewrites itself. Hone's result is a genuine engineering gain, but it says nothing about whether any customer's retention, conversion or vendor spend moved.

Hone has named the harder problem itself. Its August mission post lists the research it's working on, and one item is "grading and credit assignment across month-long trajectories." Credit assignment is attribution under a research label. It asks how much of a quarter's rise in retention the Engine actually caused.

Vendors already paid on outcomes chose events they can count

Outcome pricing for agents already exists, and the vendors running it chose countable events over rates. Intercom's Fin charges $0.99 per outcome. A resolution counts when the customer confirms the answer or leaves without asking for more help, including going quiet for 24 hours after Fin's last reply.

Zendesk counts 72 hours of silence, according to TechTarget's August look at per-resolution pricing. TechTarget also points out that the party choosing the proxy is the party billing for it. Analyst Craig Durr told TechTarget that outcome pricing is being "sold ahead of any agreed way to score it."

The closest match to Hone's shape is Sierra. On July 16, it launched Horizon, agents that pursue goals over days, weeks or months, from closing a sale to scheduling a specialist referral. Sierra told buyers they "pay for business outcomes delivered."

Even Horizon's examples are events, such as a loan originated, an appointment booked or a plan upgraded. Each one either happened or didn't, and a log shows the agent's part in it. Retention and procurement cost are rates measured against a baseline, and a rate has no log entry.

Our September read on AI pricing argued that who owns the outcome decides the pricing model, with consumption winning where the agent does the work end to end. Hone pushes that argument somewhere it hasn't been tested, to an agent that owns a rate rather than a transaction. Services buyers have watched a version of this before. Labor-based agency pay has shrunk to 19% of contracts, per the WFA data in our piece on Infosys and services repricing, and the standing objection is still that clients say they want outcomes and keep buying hours.

Attribution is where an outcome contract breaks

The best public evidence on how far a reading without a control group can drift comes from advertising, where a platform can run randomized tests at scale. Researchers from Northwestern and Facebook compared observational measurement against randomized trials across 15 Facebook ad experiments. Matching and regression methods overestimated effectiveness, and in half the studies the estimated lift was off by a factor of three.

Retention is noisier than an ad conversion. A price increase, a seasonal renewal cohort, a product release and a CSM reassignment can all move it in the same quarter. An Engine credited with the whole change would also be paid for the calendar.

Who runs the measurement matters as much as the method. When Anthropic, Gorgias and Pylon published agent scores, the numbers mostly reflected who measured them, against what baseline and under whose definition of a win. A retention contract has the same problem with more money attached.

Hone's onboarding has the Engine build its own picture of "what good looks like" from the company's history, and the homepage offers to let the Engine build evaluations from its own work. Both are useful for running the agent. Neither should set the number the vendor is judged on.

Hone's customers may never ask for a metric-based contract. If an Engine is priced like software and managed like a hire, nobody needs a counterfactual to pay the invoice. That moves the attribution problem rather than removing it. The CS leader who assigned an Engine to retention still has to defend its credit in the next budget cycle, and "the Engine says it saved these accounts" won't survive a CFO.

The agent startup enso tested its agents against a control group before it reported a lift, restructuring half of 142 pages and leaving the other half alone. A vendor that publishes its holdout is making a claim a buyer can check, which is more than a before-and-after chart offers.

Hone has published no customer result, and a holdout would produce one

The attribution problem binds every vendor that sells an agent against a rate, from Hone's retention and vendor-spend Engines to the procurement coordinator in Index's post. A buyer that keeps a random holdout of accounts the Engine never touches has a number Hone can't set for itself. So far, Hone's one published figure is an internal harness test, which cut regressions from 33% to 5% across 125 scenarios.

Retention and procurement cost are rates measured against a baseline, and a rate has no log entry.

Hone's first published result for Cognition or Modal will show whether the lift it claims was measured against a holdout group.

By Business of GTM