What is the single most useful question to ask a vendor?

How quickly a permission change in the source system takes effect in search. Everything else about the capability is table stakes and this number is the actual exposure, because it defines the window in which somebody who has just lost access can still be shown the material through a different door.

Why is an assembled answer harder than a filtered list?

Because the restricted material can reach the reader in paraphrase without ever being displayed. Filtering links is a straightforward test applied to each result; composing a summary means every contributing source must clear the same test, and a system that gets this subtly wrong leaks content rather than links.

Can we test it ourselves before rollout?

Yes, and it is worth building the awkward cases deliberately. Take somebody who changed departments, a folder shared externally, and a document whose owner has left, then ask questions that would draw on each. Those three cover most of what a demonstration environment structurally cannot contain.

Does this slow everything down?

It shapes the architecture more than the speed. Because checking with every source system per query would be unusable, the rules are held alongside the material and applied at answer time, which keeps things fast and creates the drift question instead. The cost lands as complexity and as procurement time rather than as latency.

Glossary

Permission aware search

Permission aware search returns different results to different people according to what each may already see, which makes its failure mode a disclosure rather than a disappointing answer.

In plain terms

The same question gives different answers to different people, according to what each of them is already allowed to open. Nobody gains access they did not have. That sounds administrative and it is the reason this whole category is bought slowly, because getting it wrong does not produce a poor answer, it produces somebody reading something they should not have seen.

Why it matters

Because the consequence of an error is categorically different from everywhere else in search. A ranking mistake wastes somebody's minute and a permission mistake is a disclosure, potentially of salary data, legal advice or an unannounced restructure, and no amount of quality elsewhere compensates for it. That asymmetry explains the procurement, the price and the pace.

How it works

Each document's existing rules have to be recorded and applied per person at the moment of asking, which is a different arrangement from checking with the source system each time. Asking every system on every query would be far too slow, so the rules are copied alongside the material and must then be kept honest by a process rather than enforced by the systems that originally owned them.

Assembled answers are the genuinely hard case and the one nobody demonstrates. When a single response draws on several documents, every one of them has to clear the same test for the person asking, and a summary blending a permitted source with a restricted one leaks the restricted material in paraphrase without ever showing it. That is far harder than filtering a list of links.

Rules drift between refreshes, which turns a schedule into a correctness question. Somebody changes role, a folder's access is tightened, a contractor's rights lapse: until the copy catches up, the old rules govern, so the honest question for a vendor is not whether permissions are respected but how quickly a change in the source takes effect here.

The awkward cases are the ones absent from every demonstration environment. A person who moved between departments and kept access they should have lost, a document whose owner left, a folder shared with an external party years ago: each is a correctness problem where the failure is exposure, and none of them exists in a tenant built last week for a sales call.

Two ways to get permissions wrong

There is a feedback problem here worth naming, because it shapes how these systems age. Being too strict generates immediate, vocal, well-directed feedback: a person cannot find something they know exists, they raise it, somebody adjusts it, and the system improves. Being too loose generates nothing. The person who sees a document they should not have may not realise it was restricted, may assume it was shared deliberately, and has no incentive to report it even if they suspect otherwise. So the only signal an organisation receives points in one direction, and a team responding sensibly to the feedback they get will loosen things over time and never learn that they have. The remedy is not more caution in general, which just generates more of the complaints that are already heard. It is a deliberate test built from the cases nobody complains about: the leaver, the transfer, the external share, checked on a schedule rather than when somebody happens to notice.

Seen in the wild

Enterprise search that indexes an organisation's applications and answers with references, showing each person only what their existing permissions allow.
Glean
A curated knowledge layer whose agents give cited answers that respect what each person may see, over material maintained as its sources change.
Guru
An assistant grounded in an organisation's own files and mail under the permissions that already govern them, inside the applications people use.
Microsoft Copilot

Common misconceptions

People assume

It is a filter applied to the results.

In fact

Filtering a list is the easy version. The hard version is a written answer assembled from several documents, where each source has to clear the test independently, because a summary that blends a permitted document with a restricted one has disclosed the restricted material without ever displaying it.

People assume

If it respects permissions, it is safe.

In fact

It respects the permissions it recorded, which may be older than the ones in force. Access changes continuously and the copy catches up on a schedule, so there is a window in which the old rules govern, and the length of that window is the thing worth asking about rather than the principle.

People assume

A demonstration shows us how it behaves.

In fact

A demonstration tenant has none of the cases that matter. Nobody in it has moved departments, no folder was shared externally three years ago, no owner has left, so the environment systematically excludes exactly the situations where this goes wrong. The interesting behaviour cannot be shown in twenty minutes.

Telling them apart

Permission aware search vs Enterprise search

Permission aware search

The property that makes it deployable at all.

Enterprise search

The product category it belongs to.

Every serious product in the category has to be the first thing, which is why the capability is rarely marketed and always priced.

Questions

What is the single most useful question to ask a vendor?: How quickly a permission change in the source system takes effect in search. Everything else about the capability is table stakes and this number is the actual exposure, because it defines the window in which somebody who has just lost access can still be shown the material through a different door.
Why is an assembled answer harder than a filtered list?: Because the restricted material can reach the reader in paraphrase without ever being displayed. Filtering links is a straightforward test applied to each result; composing a summary means every contributing source must clear the same test, and a system that gets this subtly wrong leaks content rather than links.
Can we test it ourselves before rollout?: Yes, and it is worth building the awkward cases deliberately. Take somebody who changed departments, a folder shared externally, and a document whose owner has left, then ask questions that would draw on each. Those three cover most of what a demonstration environment structurally cannot contain.
Does this slow everything down?: It shapes the architecture more than the speed. Because checking with every source system per query would be unusable, the rules are held alongside the material and applied at answer time, which keeps things fast and creates the drift question instead. The cost lands as complexity and as procurement time rather than as latency.

Key takeaways

The failure mode is a disclosure, not a poor answer, and that governs everything.
Assembled answers are the hard case: every contributing source must clear the test.
Rules are copied and drift, so the refresh window is the real exposure.
Demonstration environments structurally exclude the cases that matter.

Tools that use this

Glean
Each person shown only what their existing permissions allow.
Guru
Cited, permission-aware answers over actively maintained material.
Microsoft Copilot
Grounded in tenant content under the permissions already in force.