What should be decided before it starts?

What question it answers, who takes part and what result would mean no. Without those it becomes an experience that everybody enjoyed and nobody can argue with, which is comfortable and leaves the purchase decision exactly where it was before the trial began.

Somebody sceptical and somebody with genuinely awkward work, alongside the volunteers who asked. The enthusiasts will report favourably almost whatever happens, so they confirm little, and the useful information comes from the people whose working week most resembles the average after a rollout rather than the best case.

Is it worth trialling more than one tool at once?

Comparison is more informative than a single impression, because a lone trial has nothing to be measured against and an adequate product feels good in isolation. The cost is attention split across two, which usually means giving each a narrower question.

Glossary

Trial period

A limited window to use a tool before committing, which answers questions about the product and few about your own organisation.

In plain terms

A free go before you pay. It reliably answers whether the thing works and whether people like it, and it is much weaker at the question that actually decides a purchase, which is whether anybody will still be using it in three months.

Why it matters

Because the enthusiasm a trial produces is the least reliable signal available and the most persuasive one in the room. Something new, briefly, with attention on it is a good experience almost regardless of the product, and the people who volunteer for a trial are the people most disposed to like it, so a successful trial mostly demonstrates that the volunteers were enthusiastic.

How it works

The window is set by the vendor and suits their sales cycle rather than your decision. Two weeks is long enough to form an impression and short enough that nobody reaches the point where novelty has worn off, which is exactly the point at which the useful information starts arriving.

Who takes part decides the result more than the product does. A group that asked to try it will report favourably, and a group that includes somebody sceptical and somebody with a genuinely awkward workload produces a much better reading, because those are the people whose experience predicts what happens after purchase.

The questions worth asking should exist before it starts. A trial without a written question is an experience rather than an evaluation, and it will end with a general feeling that has no way of being wrong, which is exactly the outcome a vendor is content with.

The thing it cannot show is the part that decides everything afterwards. Whether use survives the second month, whether the tool holds up on work nobody chose to showcase, and whether the people who did not volunteer ever adopt it: none of those fits inside the window.

Extending it is worth asking for and is frequently granted, because a vendor would rather have a slower decision than a fast rejection. A window of a few weeks changes what can be observed considerably more than any amount of care within a short one.

What fits inside the window

The asymmetry here is the whole reason trials mislead, and it is nobody's fault. A vendor setting a two-week window is not concealing anything; two weeks genuinely is enough to establish that the product functions, and functioning is what the vendor is confident about. The difficulty is that the left-hand column no longer discriminates. Most tools a buyer is seriously considering will pass all three lines, so a trial that answers them produces a result indistinguishable from what the competing tool would have produced, and the decision then falls back on the enthusiasm of whoever took part. The right-hand column is where products genuinely differ, and none of it can be observed inside the window. That leaves two things worth doing, neither expensive. Ask for longer, which is frequently granted because a slower decision beats a fast rejection. And write the question down first, because the difference between an evaluation and an experience is entirely whether somebody decided in advance what would count as a disappointing result.

Seen in the wild

A fortnight of assistant use by volunteers, which measured enthusiasm rather than fit.
ChatGPT
Trialling a search deployment on tidy documentation rather than the awkward material people actually ask about.
Glean
Writing down what an automation trial has to demonstrate before switching it on.
Make

Common misconceptions

People assume

A successful trial predicts a successful rollout.

In fact

It predicts that the people who chose to take part enjoyed something new while attention was on it. The behaviour that decides a rollout is what people do once it stops being new, and that lies outside the window by construction.

People assume

Longer is straightforwardly better.

In fact

Longer helps only if somebody is watching for something specific. An extended window with no written question produces a longer general impression, which is the same weak signal with more time invested in it.

Questions

What should be decided before it starts?: What question it answers, who takes part and what result would mean no. Without those it becomes an experience that everybody enjoyed and nobody can argue with, which is comfortable and leaves the purchase decision exactly where it was before the trial began.
Who should be in it?: Somebody sceptical and somebody with genuinely awkward work, alongside the volunteers who asked. The enthusiasts will report favourably almost whatever happens, so they confirm little, and the useful information comes from the people whose working week most resembles the average after a rollout rather than the best case.
Is it worth trialling more than one tool at once?: Comparison is more informative than a single impression, because a lone trial has nothing to be measured against and an adequate product feels good in isolation. The cost is attention split across two, which usually means giving each a narrower question.

Key takeaways

It measures the product; it rarely measures your organisation.
Volunteers report favourably, which is why the group composition decides the result.
A trial with no written question ends in a feeling that cannot be wrong.
What happens after novelty wears off falls outside the window.