Using Anthropic Claude in Copilot - Things to Consider

Thanks to Tony Redmond for noticing that you can switch to it, but you lose Enterprise Data Protection from Microsoft.

Photo by Markus Winkler on Unsplash

When I first read the announcement about Microsoft introducing Anthropic’s Claude LLM to Copilot, I assumed they would integrate it the same way they use OpenAI models. You would interact with Copilot and not even know which model it was using, but everything would be covered under EDP.

Tony was quick to point out last week that Microsoft has made it clear that you need to enable access to the Claude LLM, but also that the data submitted will not fall under the Enterprise Data Protection agreement that you have with Microsoft as a customer.

Use of Anthropic Models Concerns Microsoft 365 Tenants

Microsoft’s documentation covering how to connect to the Anthropic models emphasizes that:

When your organization chooses to use an Anthropic model, your organization is choosing to share your data with Anthropic to power the features. This data is processed outside all Microsoft‑managed environments and audit controls, therefore Microsoft’s customer agreements, including the Product Terms and Data Processing Addendum do not apply. In addition, Microsoft’s data‑residency commitments, audit and compliance requirements, service level agreements, and Customer Copyright Commitment do not apply to your use of Anthropic services. Instead, use of Anthropic’s services is governed by Anthropic’s Commercial Terms of Service and Anthropic’s Data Processing Addendum.

Tony’s primary concern is EU-based organizations as they pertain to data. Still, any regulated industry should take a long, hard look at what is happening with those agreements before enabling user access to Claude with Copilot Studio or the Researcher Agent, the two places it currently exists, once enabled.

Luckily, my personal tenant doesn’t have other users to worry about, so I could conduct some testing to answer different questions I have about potential differences in eDiscovery data, if any.

Tony is correct to point out that this interaction not showing up in Purview is problematic in several ways. I did not spend enough time reviewing the Audit log or DSPM for AI results because my efforts to set up the new tenant for those tools haven’t been completed yet. I did use Purview eDiscovery to collect the interactions after trying the Researcher agent once with Claude and once with the regular OpenAI model.

There isn’t much difference between the two. In fact, I was unable to determine which model was used with the data I collected in the Review Set. I also couldn’t even look at that data and tell you it came from the Researcher agent, other than assuming it came from a “deep thinking” agent due to the volume of responses.

There’s no obvious metadata that tells me it was anything other than an interaction with Copilot Chat.

That’s it.

You may find it a little frustrating if you’re trying to investigate a user’s behavior with Copilot, but this is likely due to the way Microsoft stores interactions within Exchange. There isn’t an Exchange metadata field to hold agent information. You may need to access that using other Purview tools and Copilot analytics, maybe? I’m not sure yet. I need to conduct further testing with agents in this regard while also waiting for some of the new analytics tools to roll out.

Before I delve into the details of what I collected from these Researcher agent interactions for paid subscribers, I want to note that this personal tenant, which I’m paying for, allows me to conduct this kind of testing with a fully licensed user and a licensed Copilot account, without violating my employer’s policies. Obviously, owning your own tenant, even for just a couple of users, is not inexpensive. I decided I was okay with spending the money, but it would certainly help to acquire some additional paid subscriptions to offset some of the expense.

If you’ve ever considered subscribing, now is a great time! Tomorrow is the last day to get a 20% discount on an annual subscription, as a matter of fact.