How long should one run?

Short enough that the answer still matters when it arrives, which for most teams is weeks rather than months. The useful discipline is to set the end date when you set the question, because an experiment without one drifts into being a small system that nobody agreed to own or maintain.

Somebody close enough to the work to recognise a bad output when they see one. A technically excellent build evaluated by people who cannot judge the domain produces a confident answer to the wrong question, and that is harder to unpick later than an obviously rough prototype.

Should we tell people it is only a proof of concept?

Yes, repeatedly and in writing, because the demonstration will travel further than the caveat. Naming what was skipped at the same time as showing what worked is the only reliable way to stop a promising experiment from being read as a delivery commitment by somebody who was not there.

Glossary

Proof of concept (PoC)

A proof of concept is a deliberately small build whose only job is to answer whether an idea can work at all, using shortcuts a real system could not keep.

In plain terms

It is an experiment with a question attached. You build the smallest thing that could answer whether the idea holds up, using whatever shortcuts make it quick: a handful of documents instead of the archive, one person's login instead of proper access control, output nobody has to rely on. Then you look at the result and decide. The shortcuts are the point rather than a compromise, because they are what make the answer cheap.

Why it matters

Because a successful one looks exactly like a finished product to anybody who was not in the room. The demonstration works, the output is impressive, and the natural next sentence is that it should be rolled out. What is invisible from outside is everything the build was permitted to skip, and those omissions are usually most of the real work. Treating a proof of concept as nearly done is the commonest way an AI project acquires a schedule it cannot meet.

How it works

Write the question down before anything is built. A proof of concept exists to answer one thing, and if nobody can state that thing in a sentence the build drifts into a demonstration of the tool rather than a test of the idea. The symptom is easy to spot afterwards: everybody agrees it went well and nobody can say what it established.

Use real material, never sample material. The shortcuts belong in the plumbing and never in the inputs, because the difficulty in almost every genuine case lives in the mess: inconsistent formats, missing fields, the exceptions people handle by knowing the history. A build that works beautifully on tidy examples has answered a question nobody asked.

Give it an end date at the same time as the question. The value is in reaching a decision quickly, and an experiment with no stopping point quietly becomes a small unowned system that people start to depend on. That is the worst of both states: not reliable enough to trust and too used to switch off.

Record what was skipped while you are skipping it. Access control, error handling, the volume, the awkward cases, the second language: that list is the honest estimate of the work still to come, and it is far harder to reconstruct three weeks later when somebody asks what is left. Written at the time, it takes minutes.

Decide in the open, and keep stopping on the table. A proof of concept that answers no has done its job, and saying so plainly is what makes the next one cheap to authorise. Teams that only ever report successes gradually lose the ability to run cheap experiments at all, because every one becomes a commitment.

Where a proof of concept sits

The three stages answer three different questions and the distance between them is mostly invisible from a demonstration. A proof of concept sits at the far left: it may use twenty documents rather than twenty thousand, one person's access rather than a permissions model, and output nobody acts on. A pilot moves rightwards by removing those allowances one at a time, which is why it takes longer than the proof of concept did and surprises people who read the first result as nearly finished. Production is where somebody's job depends on the thing working on a Tuesday morning. The practical use of this picture is to locate an argument: when a team disagrees about whether something is ready, they are almost always standing at different points on this line without having said so.

Seen in the wild

Point an assistant at twenty real documents rather than the whole archive, and see whether the answers are good enough before anybody builds ingestion.
NotebookLM
Wire two steps of a process together by hand to see whether the output is usable, before deciding whether the full workflow is worth building.
Zapier
Run the same request against a few different models to find out whether the task is within reach at all, before committing to one.
OpenRouter

Common misconceptions

People assume

It worked, so we are most of the way there.

In fact

You are most of the way through the question of whether it is possible, which is usually the smaller half. What remains is access control, error handling, the awkward cases, the volume and somebody owning it, and none of that was exercised by the thing that just worked.

People assume

A proof of concept that fails was wasted effort.

In fact

It answered the question cheaply, which is precisely what it was for. The expensive version of that answer arrives nine months later with a team attached. A quick no is one of the better outcomes available and deserves to be reported as a result rather than as a disappointment.

Telling them apart

Proof of concept vs Pilot

Proof of concept

Asks whether the idea can work at all. Shortcuts allowed. Few or no real users.

Pilot

Asks whether it works in practice. Real users, real volume, real consequences for failure.

Ask who is depending on the output. Nobody yet means the first; somebody's actual work means the second.

Questions

How long should one run?: Short enough that the answer still matters when it arrives, which for most teams is weeks rather than months. The useful discipline is to set the end date when you set the question, because an experiment without one drifts into being a small system that nobody agreed to own or maintain.
Who should build it?: Somebody close enough to the work to recognise a bad output when they see one. A technically excellent build evaluated by people who cannot judge the domain produces a confident answer to the wrong question, and that is harder to unpick later than an obviously rough prototype.
What is the commonest mistake?: Using tidy sample data. It makes the build easier and the answer worthless, because the difficulty in almost every real case lives in the mess: inconsistent formats, missing fields, the exceptions people handle by knowing the history. Real material early is what makes the result mean anything.
Should we tell people it is only a proof of concept?: Yes, repeatedly and in writing, because the demonstration will travel further than the caveat. Naming what was skipped at the same time as showing what worked is the only reliable way to stop a promising experiment from being read as a delivery commitment by somebody who was not there.

Key takeaways

A proof of concept answers whether an idea is possible, nothing more.
Shortcuts belong in the plumbing and never in the inputs: use real material.
Write down what was skipped while skipping it; that list is the remaining work.
Set the end date when you set the question, or it becomes an unowned system.
An honest no is a successful outcome and should be reported as one.

Tools that use this

NotebookLM
A handful of real documents, answering the quality question before ingestion is built.
Zapier
Two steps wired by hand to test whether the output is usable at all.
OpenRouter
The same request across several models, to see whether the task is within reach.