What should we actually look at in the output?

Which records were matched, and what each one would have received. The count is the first thing worth checking, because a filter matching far more or far fewer than expected is the commonest fault and the easiest to see. After that, the individual results for anything unusual in the data.

Does every tool support this?

Not uniformly, and where it exists the boundary differs. Some withhold every outbound action, some withhold writes but still send notifications, and some offer nothing of the kind. Where it is absent the usual substitute is pointing the automation at a destination that does not matter.

Is it still worth doing for a small automation?

The value scales with how many things the automation touches rather than how complicated it is. A simple rule applied across thousands of records deserves one more than an intricate rule applied to a handful, which is the opposite of where people usually spend the effort.

Glossary

Dry run

Running a process with its final actions withheld, so what it would have done can be read before anything real is changed.

In plain terms

A rehearsal in the actual building with the audience kept outside. Everything runs, the timings are real, and nobody is affected by the mistakes. What it cannot tell you is how the room behaves once it is full, which is the part people forget when the rehearsal goes well.

Why it matters

Because the failure it prevents is the one that arrives at scale. An automation applied to a hundred records does the wrong thing a hundred times before anybody reads the first result, and the mistakes that produce that are rarely subtle: a filter that matches more than intended, a field mapped to the wrong destination, an assumption about the data that holds for the examples somebody checked and not for the rest.

How it works

The final actions are withheld and everything before them runs for real. Triggers fire, records are read, conditions are evaluated, and the step that would send, create or update reports what it would have done instead of doing it. That boundary is the whole design, and where a tool draws it decides how much the exercise is worth.

Reading the output is where the value is, and it is the part most often skipped. A run that completes without errors proves the automation can execute, not that it should. What it did to each record is the thing worth looking at, especially the records nobody had in mind when the rules were written.

The most useful input is the awkward data rather than the clean data. Rules are built against examples somebody chose, and those examples are usually tidy. The record with an empty field, the name with an unusual character, the entry created by a different process years ago are where the rules turn out to have been assumptions.

What it cannot show is anything that depends on the actions happening. A workflow whose later steps read what the earlier ones wrote will behave differently once writing is real, and anything involving timing, contention or limits imposed by the other side is not exercised at all. Those are exactly the ones that surface at volume.

It is a check on intent rather than a guarantee of safety, and the two get conflated. Passing means the rules do what the person who wrote them believed. It says nothing about whether that belief was correct, which is why a small real run against a handful of records usually follows.

What the rehearsal covers

Keeping these apart matters because a clean result is read as a general reassurance, and it is a specific one. The left-hand column covers the faults that come from somebody misunderstanding their own rules, and those are both common and cheap to fix once seen: a filter matching four hundred records instead of forty is obvious the moment the number is in front of you. The right-hand column covers faults that only exist once the actions are real, and no amount of care in the rehearsal reaches them, because the thing being withheld is exactly the thing they depend on. That is not an argument against the exercise. It is an argument for what usually follows it, which is a small real run against a handful of records where somebody reads every result, because that is the cheapest way to reach the right-hand column without reaching it across the whole dataset at once.

Seen in the wild

Checking which records a new filter actually matches before letting the workflow write to any of them.
Make
Reading what an automation would have sent to customers, before it is allowed to send anything.
Zapier
Running a workflow against awkward historical records rather than the tidy recent ones it was built against.
n8n

Common misconceptions

People assume

A clean dry run means the automation is safe to switch on.

In fact

It means the automation runs and does what its author intended. Whether that intention was right, and whether anything depending on the actions actually happening behaves the same way, are both outside what the exercise can show.

People assume

It is the same as testing on a copy of the data.

In fact

A copy is stale and a dry run uses what is there now, which is its advantage. It is also weaker in one respect, because a copy can be written to and read back while a dry run withholds exactly the step that would make later steps meaningful.

Questions

What should we actually look at in the output?: Which records were matched, and what each one would have received. The count is the first thing worth checking, because a filter matching far more or far fewer than expected is the commonest fault and the easiest to see. After that, the individual results for anything unusual in the data.
Does every tool support this?: Not uniformly, and where it exists the boundary differs. Some withhold every outbound action, some withhold writes but still send notifications, and some offer nothing of the kind. Where it is absent the usual substitute is pointing the automation at a destination that does not matter.
Is it still worth doing for a small automation?: The value scales with how many things the automation touches rather than how complicated it is. A simple rule applied across thousands of records deserves one more than an intricate rule applied to a handful, which is the opposite of where people usually spend the effort.

Key takeaways

It proves the rules do what their author intended, not that the intention was right.
The record count is the first thing worth reading in the output.
Awkward historical data finds faults that tidy recent data will not.
Anything depending on the actions actually happening is not exercised.