85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What is a tool risk rating?

A tool risk rating is a low, medium or high classification given to each tool an AI agent can call, based on write access, reversibility, permissions and financial impact, used to decide which calls need extra checks.

Not every tool call deserves the same scrutiny. Looking up an order is harmless if it goes wrong; issuing a refund or deleting a record is not. OpenAI's A practical guide to building agents (2025) turns this into a simple rule: assess each tool, give it a rating of low, medium or high, and let the rating trigger automated actions, such as pausing for guardrail checks before a high-risk function runs or escalating to a person.

A rating moves the safety decision from the moment of the call, when a person or classifier has seconds to judge it, to design time, when the team can think through the worst case calmly.

The factors OpenAI lists

  • Read-only or write access

    A tool that only reads cannot change anything, though it can still expose data.

  • Reversibility

    A draft can be discarded and a soft delete restored; a sent email or a paid invoice cannot be taken back.

  • Required account permissions

    A tool running with an administrator's rights can do far more damage than one scoped to a single user.

  • Financial impact

    Payments, refunds and orders carry direct cost when they go wrong.

What a rating triggers

  • Low

    Runs without interruption and is logged.

  • Medium

    Runs after automated checks, such as argument validation, recipient allowlists, amount caps or rate limits.

  • High

    Pauses for guardrail checks and, where needed, a person. OpenAI's guide names cancelling orders, authorising large refunds and making payments as actions that should trigger human oversight until confidence in the agent grows.

Ratings declared by the tool itself

The Model Context Protocol lets a server attach annotations to a tool, including hints that the tool does not modify its environment, that it may make destructive updates, that repeated calls with the same arguments have no extra effect, or that it reaches an open world of external systems. The specification says clients must treat annotations as untrusted unless they come from a trusted server.

Annotations are a useful input to a rating, especially for third-party MCP servers, but the rating belongs to the team deploying the agent, because the same tool can be low risk in one workflow and high in another.

Common mistakes

  • Rating the tool, not the arguments

    A refund of 5 and a refund of 5,000 go through the same tool. Where impact varies with the arguments, the rating should too.

  • Ignoring combinations

    A tool that reads private data and a tool that fetches a URL can each look low risk, yet together form an exfiltration path. Source-sink analysis catches this.

  • Ratings that never change

    OpenAI ties human oversight to confidence in the agent's reliability, so ratings should be revisited as eval and production evidence builds up.

  • Too many highs

    If most tools are rated high, approvals become constant and stop being read.

How Motiontic's engineers apply it

Motiontic's engineers rate every tool in the scope document before it is built, with the client's owner of the underlying system, and keep the rating next to the tool's definition in code. The rating sets the control: logged, guarded by argument checks and limits, or paused for approval. Ratings are reviewed whenever a tool is added or its permissions change.

Example tool risk ratings for a support agent

Example tool risk ratings for a support agent
ToolAccessReversibleRatingControl
Look up orderReadNot applicableLowLogged
Draft replyWrite, draft onlyYesLowLogged; a person sends it
Update shipping addressWriteYes, with historyMediumCustomer verified first, argument checks
Issue refundWrite, financialPartlyHighAmount cap, approval above a threshold
Cancel orderWriteOften notHighHuman approval

Common questions

What is a tool risk rating?

A tool risk rating classes each tool an AI agent can use as low, medium or high risk, based on factors such as write access, reversibility, permissions and financial impact, so riskier calls get more checks.

Where does the idea of tool risk ratings come from?

The term is used in OpenAI's A practical guide to building agents (2025), which lists tool safeguards among layered guardrails. OWASP's LLM06 Excessive Agency risk covers the same ground from the attacker's side: too much functionality, permission or autonomy.

How do tool risk ratings relate to human-in-the-loop?

The rating decides where a person is needed. Low-risk tools run freely, while high-risk tools pause for approval, which keeps human review for the few actions where it matters.

Are MCP tool annotations a risk rating?

Not on their own. They are hints from the server about how a tool behaves, and the MCP specification says clients must treat them as untrusted unless the server is trusted.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.