Is holding our own key worth the trouble?

It depends on whether you would ever use the lever. The arrangement buys the ability to withdraw access and creates real operational responsibility, so it earns its keep where a regulator or a customer requires it and rarely where it is adopted for reassurance alone.

What should we actually ask a vendor?

Who generates the key, where it is held, who inside the vendor can use it and under what process, how often it is replaced, and what happens if it is lost. The last one is the most revealing, because a comfortable recovery path implies a corresponding way in.

Why is this rarer for AI tools than other software?

Because the useful behaviours need readable material. Indexing, searching and generating all require the service to work with content rather than with scrambled bytes, so arrangements that keep a vendor fully out tend to remove the features that made the tool worth buying.

Glossary

Key management

Deciding who holds the keys that make encrypted data readable and who may use them, which settles how much any encryption promise is worth.

In plain terms

Everyone agrees the safe should be locked. The question that decides what the lock is worth is who has a copy of the key, where they keep it, and whether they need to tell you before opening it. That question is separate from the safe and usually harder.

Why it matters

Because encryption without this is a statement about mathematics rather than about access. Material is unreadable to anyone lacking the key and the interesting people all have one, so the arrangement around it is what turns a technical control into a commercial commitment. It is also where the practical differences between vendors sit, and where the answers are contractual rather than demonstrable.

How it works

In the default arrangement the vendor generates, holds and uses the key on your behalf. Nothing about this is dishonest and it is what almost everybody uses, because it works without any effort from the customer. It also means the vendor can read stored material whenever their systems require it, which is a description of the arrangement rather than a criticism of it.

The next step along lets a customer supply or control the key while the vendor still uses it to operate the service. What changes is that the customer can withdraw it, which turns the relationship into something with a lever in it. What does not change is that a running service reads ordinary data, because it must.

The furthest arrangements keep material unreadable to the vendor throughout, and they cost something real in what the service can do. A vendor that cannot read your material cannot index it, search it, or generate from it in the ways that make most AI tools useful, so this end of the range is rare in the AI category specifically.

Rotation is the part that gets promised and skipped. Replacing keys periodically limits how much a compromised one is worth, and it requires everything holding data encrypted with the old key to be dealt with, which is exactly the sort of work that gets deferred. A vendor's answer about how often it happens is more informative than the fact that it can.

The recovery question is the one people meet last and remember longest. Whoever can restore access after a key is lost can also grant it, so an arrangement with a comfortable recovery path has a corresponding way in. That trade cannot be avoided, only chosen deliberately.

How far the vendor is kept out

The reason to lay this out as a range rather than a choice is that the two ends are not equally available and the trade is not linear. Moving one step from the default buys a genuine commercial lever at almost no functional cost, which is why that step is where most regulated buyers land. Moving to the far end buys a property that sounds like the goal and removes most of what an AI tool does, because indexing, searching and generating all need readable material and a vendor that cannot read yours cannot do any of them. That is why the far end is common in storage products and rare in this category. The other thing the range makes visible is that every position on it still leaves material readable while it is being processed. None of these arrangements changes that, and a buyer who moves along the range expecting it to is buying operational responsibility for a property they did not acquire. What genuinely changes across the range is who can read stored material without your involvement, and how quickly you could stop them.

Seen in the wild

Asking an assistant vendor whether stored conversations use a key the customer can withdraw.
ChatGPT
Weighing a search deployment's indexing ability against how far the vendor can be kept out of the index.
Glean
Taking on key handling directly by running the model yourself, which moves the whole question in-house.
LM Studio

Common misconceptions

People assume

Holding our own key means the vendor cannot see our data.

In fact

It means they cannot read it from storage without the key being available. While the service is running and processing your material, it is readable to the systems doing the processing, because that is what processing is.

People assume

This is a technical detail for the security team.

In fact

It is mostly a commercial question. What can be arranged, what it costs, and what happens on the day you want to withdraw it are answered in the contract rather than the architecture, and the answers vary far more between vendors than the technology does.

Questions

Is holding our own key worth the trouble?: It depends on whether you would ever use the lever. The arrangement buys the ability to withdraw access and creates real operational responsibility, so it earns its keep where a regulator or a customer requires it and rarely where it is adopted for reassurance alone.
What should we actually ask a vendor?: Who generates the key, where it is held, who inside the vendor can use it and under what process, how often it is replaced, and what happens if it is lost. The last one is the most revealing, because a comfortable recovery path implies a corresponding way in.
Why is this rarer for AI tools than other software?: Because the useful behaviours need readable material. Indexing, searching and generating all require the service to work with content rather than with scrambled bytes, so arrangements that keep a vendor fully out tend to remove the features that made the tool worth buying.