I was sitting down with Claude to edit a blog post that our account executive had written. I worked through it with Claude, made some updates, and then typed the following prompt:
Summarize the changes that we made to the blog post and send them to Chris in Slack.
But before I hit Enter, I paused.
There are two Chrises in our Slack. One is our account executive. The other is a customer.
What if Claude got confused and sent the message to the wrong Chris?
If it were just edits to a blog post, that would be pretty embarrassing. But what if I had been asking Claude to send Chris information about another customer? Then it wouldn’t just be embarrassing. It could be a data breach.
The policy I put in place
I decided to do something about it.
Claude’s connectivity to our internal systems is routed through Dtwo, letting us write fine-grained policies governing what AI agents can and cannot do.
I asked Dtwo to write the following policy for me:
Don’t allow the sending of Slack messages to anyone outside the Dtwo organization.
Dtwo wrote a policy that prevents agents from sending direct messages to external users, and posting in Slack channels that include people outside our organization.
With that policy in place, I felt much more confident giving Claude an ambiguous instruction like “send this to Chris.” Even if the agent selected the wrong Chris, the policy would prevent the message from being sent outside the company.
The opposite problem: policies that are too restrictive
I’ve also encountered the opposite problem: policies that are too restrictive.
Our CEO set up a set of policies for our internal Dtwo setup. One of them prevented Claude from accessing users’ private Google Drives.
For him, that policy makes a lot of sense. His Drive contains highly confidential information, including salary information and company financials. That data should not be exposed to an AI agent.
But for someone like me, working in product management, blocking Claude from my Drive significantly reduces its usefulness without providing the same security benefit.
So we made the policy more fine-grained.
We created a group in our identity provider for people whose personal Drives contain especially sensitive information. For now, because we’re a small startup, that group contains only our CEO. As we grow, I can easily imagine members of Finance, HR, and other teams being added.
We then changed the policy so that personal Drive access is restricted only for members of that group.
The result is that our CEO’s sensitive information remains protected, while everyone else can use AI more productively.
Agents need sophisticated policies
There has always been a balance between security and usability. The power of AI agents makes that balance even harder to get right.
The answer isn’t simply more policies. It’s more sophisticated policies.
Agent security policies need to go beyond allowing or blocking individual tool calls. They need to be able to inspect inputs and outputs, understand a user’s identity, permissions, and groups in the identity provider, and apply controls across multiple systems.
They also need to understand that actions that look harmless in isolation can become dangerous when combined.
That’s what we built Dtwo to do.
The goal isn’t to lock AI down so tightly that people can’t use it. It’s to put the right guardrails in place so teams can take full advantage of AI while keeping sensitive systems and data secure.
If your team is trying to strike that balance, start a free trial.