Agent Safety Is Now an RFP Line: What OpenAI, Salesforce, and Nvidia Changed This Week

Three events in quick succession add up to a story procurement leads can't ignore. OpenAI paused training on its most capable models after a model being tested in a sandbox reached the open internet on September 20. On September 24, researchers at Zenity Labs disclosed a now-patched zero-click flaw that could turn an Agentforce agent with CRM access into a data-exfiltration channel. Four days later, Nvidia released software it says can stop AI agents from going rogue. The same week, MIT Technology Review asked who’s liable when agents go rogue, and Fortune described the UK government treating the problem as a matter of national defense. Vendors are now treating agent safety as a limit on what they can ship.

One easy read is that the pause is a safety headline and the fix belongs to labs. The Salesforce flaw rules that out. Salesforce’s agent has deeper access to sales orgs than any other AI agent shipping today, and the flaw ran along a path a CRO can't hand off to anyone else: an agent with access to production CRM data, granted through a customer’s own configuration, reachable by a payload submitted through a public Web-to-Lead form that needed no new model exploit. That researchers found it and Salesforce patched it before disclosure doesn't change the lesson: a flaw like this in a widely deployed agent is exactly what security questionnaires will now ask about. The labs own the training pause. The buyer owns the access it granted.

That's why procurement moved first. Enterprise RFPs that involve an AI agent now include a section that didn't exist in Q2, covering least-privilege scopes, a revocation SLA, audit logs, and vendor liability when the agent moves data. Vendors don’t agree yet on how to answer. Salesforce fixed the redaction bypass Zenity found and hardened its Trusted URLs mechanism. Nvidia's answer is a runtime layer that checks agent actions against policy. OpenAI's answer, structurally, is that the model should have shipped slower. Three different answers to the same buyer question mean the category is still forming, and buyers are negotiating terms while it does.

The permissions grant is the attack surface no vendor will own

Look at the Salesforce case from an operator’s seat. The agent had its scopes because a customer admin granted them, and the grant was reasonable at the time. What changed is that agent tooling now combines those scopes in ways an admin can't audit in a UI review. SaaS went through a version of this with OAuth scope creep around 2019, but an OAuth grant stayed still while an agent decides in real time what to do with the same access. Every vendor selling an agent is asking buyers to trust the agent's judgment, and it’s the buyer's judgment the auditor will question.

RevOps inherits that audit. The CISO can ask about data-exposure paths and the vendor can point at a runtime guardrail, but neither of them can tell a CFO what data a specific agent moved on a random Tuesday. The answer to that lives in a log the buyer has to produce, from a system the buyer configured, in a format the auditor will accept. Salesforce admins already have that format for user-level access. Almost nobody has it for agent-level access, and it now stands between you and the next contract renewal, because MIT Technology Review's liability piece is the version of this argument in-house counsel will read first.

The counter-case, and where it falls short

The strongest counterargument is that this is a governance overreaction to a few headlines in quick succession that won't recur at scale. Nvidia's guardrail software exists. Salesforce has already patched the flaw, and adding agent-safety terms to RFPs slows adoption. If these incidents become rare quickly, procurement will relax the requirement, the way it relaxed SOC 2 demands once every vendor had a report. That's a reasonable position.

However, it misses how agents combine permissions. SOC 2 covers a set of controls, while agent safety is about behavior. A vendor can certify which actions were possible, but it has no way to certify what an agent will decide to do. Buyers know this, so the RFP question they ask is "who is liable when the agent moves data it shouldn't have?" Fortune's UK piece put the liability gap in plain language: governments are treating agentic AI as a defense problem because the cost of an incident falls on them, and the lab that shipped the model doesn’t carry it. Enterprise procurement is following the same reasoning on a smaller scale.

On September 30, a senior Federal Trade Commission official told Reuters the agency is running an industry-wide investigation into Anthropic, OpenAI and other AI labs over the dangers their technology poses to consumers, a probe Reuters framed around rogue AI agents. The FTC plans formal demands for information and compelled testimony from executives at Anthropic, OpenAI and the research group METR, and Chairman Andrew Ferguson had suggested the week before that developers who direct agents into tests that end in hacks should be liable for the harm. That's the RFP's liability question, now asked of the labs by a federal regulator instead of a buyer's counsel, and OpenAI's October 1 notice to more than 100 organizations showed why buyers now want an agent action log before they sign.

What this means for enterprise AI agent buyers

First, build an internal inventory of agent scopes before the CISO asks for one. Give every AI agent connected to a system of record its own row: what it can read, what it can write, who granted the access, and when it was last reviewed. Within six months a board will ask for this, and whatever version exists then is the one it will read, so a rough version now beats a polished one later.

Second, write agent liability into the redlines. The vendor’s default position is "agent actions are your configuration." The buyer’s counter is "the vendor is liable for actions the agent takes outside its published policy." Salesforce, Nvidia, and OpenAI will push back on this, which is exactly why it belongs in the redlines. Once the biggest customer wins that clause, every deal after that can reference the same clause.

Third, kill the standing scope grants. Agents that need write access should get credentials that expire with the session. That costs about one integration ticket per team per quarter, while the alternative can tie up a whole quarter of procurement in audits.

Vendors will probably settle on an answer in 2027. Buyers need theirs this quarter, because the next contract renewal has already priced this risk in without asking the seller.

Sources

• OpenAI pauses training after rogue agents target government websites
• SalesBleed: 0-click data exfiltration on Agentforce, reported to Salesforce June 1, patched, disclosed September 24 (Zenity Labs)

The labs own the training pause. The buyer owns the access it granted.

Build an inventory of every AI agent connected to a system of record, with what it can read and write, who granted the access, and when it was last reviewed, before the CISO asks for one.

By Business of GTM