Glossary
Search index
A search index is the prepared copy of your content that search actually reads, and almost every awkward property of an internal search tool follows from the fact that it is a copy.
In plain terms
Search does not read your documents. It reads a prepared copy of them, built in advance so that answering a question takes a moment rather than an afternoon. Nearly everything that puzzles people about internal search comes back to this: the delay before a new file appears, the answer from a document somebody deleted, the permission that has to be copied across rather than simply obeyed.
Why it matters
Because a surprising number of complaints resolve to the same sentence. A document uploaded this morning is not findable, an answer quotes a page that was removed last week, somebody sees a snippet of a file they should not have: all three are the copy being out of step with the original, and understanding that turns three mysteries into one scheduling question.
How it works
Content is processed in advance and stored in a form built for fast lookup, which is the whole reason search feels instant on a large corpus. Reaching out to every connected system per question would take far too long to be usable, so the work happens ahead of time and what you search is the result of that work rather than the systems themselves.
Freshness is therefore a schedule rather than a property, and this is the part that surprises people most. Somewhere a decision was made about how often each source is revisited, and the gap between a document changing and the copy catching up is the direct consequence, so complaints about new material not appearing are usually complaints about a refresh interval nobody chose deliberately.
Deletions are the harder half of freshness. Adding new material is a routine operation and removing material that has gone is easy to forget, so an index that is diligent about additions and lax about removals will keep answering from documents that no longer exist, with references pointing at pages that return nothing.
Permissions have to be mirrored into it, and that mirror is the thing to interrogate. Because the copy sits outside the systems that own the original access rules, those rules are recorded when material is indexed and must be updated as they change, so the useful question for any vendor is how quickly a permission change in the source takes effect here.
Why a new document is not findable yet
Seen in the wild
Enterprise search that indexes an organisation's applications and answers with references, showing each person only what their existing access allows.
GleanA curated knowledge layer that actively maintains the material behind its answers, flagging stale content as the sources it came from change.
GuruA private document tool where ingestion and local storage are handled for you, so the copy lives on hardware you control.
AnythingLLM
Common misconceptions
People assume
Search reads our documents.
In fact
It reads a copy prepared in advance, which is why it is fast and why it is sometimes wrong about what exists. Almost every confusing behaviour of an internal search tool becomes obvious once this is understood, and almost nobody is told it.
People assume
Deleting a document removes it from search.
In fact
It removes it from the source and the copy catches up on its own schedule, if removals are handled as diligently as additions. Systems that are careless here keep answering from material that no longer exists, complete with references to pages that now return nothing.
Telling them apart
Search index vs Data ingestion
Search index
The prepared copy that search reads.
The process that fills and refreshes it.
One is the artefact and the other is the pipeline that keeps it current, which is why a stale answer is nearly always an ingestion problem rather than a search one.
Questions
- Why can I not find a document I uploaded an hour ago?
- Because the copy search reads has not been rebuilt to include it yet. Every system has an interval between material changing and the copy catching up, and that interval is a configuration somebody set rather than a law. If it is causing real friction, it is usually adjustable, and asking is more productive than waiting.
- Why did it quote a page we deleted?
- Because removals lag behind additions in most arrangements. Adding new material is the routine path and taking material out is easy to under-implement, so the copy keeps a document that has gone. If this happens repeatedly it is worth raising specifically, since it is a different defect from slow indexing.
- How do permissions work if it is a copy?
- The rules are recorded alongside the material when it is indexed and applied per person when somebody asks. That means they can drift from the source between refreshes, so the question worth putting to any vendor is how quickly a permission change in the original system takes effect in the copy.
Key takeaways
- Search reads a prepared copy, not your documents; nearly everything follows.
- Freshness is a schedule somebody set, not a property of the system.
- Removals lag behind additions, which is how deleted pages keep being quoted.
- Permissions are mirrored into the copy, so ask how fast a change propagates.
Tools that use this
- Glean
Indexes the estate and answers within each person's access.
- Guru
Maintains its material actively, flagging content as sources change.
- AnythingLLM
Ingestion and local storage, so the copy stays on your hardware.
Last checked July 2026