When is it the right choice?

When there is no other way in and the volume is modest. A supplier portal with no documented interface, an internal application nobody maintains, a process across three systems that will never be connected: in each case the comparison is against doing it by hand, and that comparison it wins comfortably.

How reliable is it in practice?

Reliable enough for supervised work and not for unattended work at volume. Expect it to stop when something on screen changes, which happens more often than people anticipate, and plan for a person noticing rather than for a system recovering on its own.

What about the account it uses?

Give it its own, with the narrowest access that lets the job run, and make the login traceable to it rather than to a person. It cannot be constrained the way a proper connection can, so the account is the only real boundary available and deserves the attention that implies.

Should we wait for a proper connection instead?

If one is genuinely coming, almost always yes. If it is not, build this as the permanent answer rather than as a stopgap, because the difference shows in whether anybody bothers to add checks, monitoring and a named owner, and the ones treated as temporary rarely get those.

Glossary

Computer use

Computer use means driving a screen the way a person would, clicking and typing through ordinary software, and its whole value is reaching systems that offer no other way in.

In plain terms

Driving the screen the way a person does: looking at what is displayed, clicking, typing, waiting. It is slower and less reliable than talking to software properly. It is also the only option when the software in question has no proper way in, which describes a great deal of what businesses actually run on.

Why it matters

Because the systems that most need help are frequently the ones that offer no way to help them. A supplier portal, an old internal application or a government website may have no documented interface at all, and this is the difference between the work being possible and not, which is why the fragility is worth accepting where it applies.

How it works

It reads the screen and acts on what it sees, which is exactly what makes it general and exactly what makes it fragile. Nothing needs to be built in advance and no cooperation from the software is required, and equally nothing guarantees that the button will be where it was yesterday.

A visual change breaks it in ways a proper connection would not notice. A redesign, a moved control, a new consent banner or an unexpected dialogue can all stop it, and the failure looks like confusion rather than an error, which makes it harder to detect automatically than a connection returning a clear rejection.

It is slow because it does what a person does, at roughly the pace a person does it. Waiting for pages, moving through screens and typing into fields all take real time, so this suits work measured in tens of items rather than thousands and stops being sensible well before volume gets interesting.

Credentials and blast radius are the serious questions rather than the technical ones. Something driving a screen is logged in as somebody, can do whatever that somebody can do, and cannot be constrained by the finer permissions a proper connection would offer, so the account it uses deserves more thought than the tool does.

It is a bridge rather than a destination in most sensible deployments. Where a proper connection exists or is coming, waiting for it is nearly always right; where none will ever exist, this is the permanent answer and should be built with that in mind rather than treated as temporary for years.

Two ways into the same system

The instinct to be embarrassed by this approach is understandable and mostly wrong. It is unmistakably a workaround, it is slower than every alternative, and it fails in ways that would be unacceptable anywhere a real interface existed. Then you look at what a typical organisation actually runs on: a portal a supplier insists on, an internal system whose author left in 2018, a regulator's website with a form and no other route, a licensing tool that has never been connected to anything. None of these will grow a proper interface, and the work still has to happen. Judged against the alternative that exists, which is a person doing it by hand every Tuesday, a fragile automated version that stops when something changes is a substantial improvement. The mistake is not using it in those places. It is using it where a real interface was sitting there unused.

Seen in the wild

An autonomous agent that plans and executes in a cloud sandbox, operating a browser to gather and compile from the live web.
Manus
An open-source agent running on your own machine that takes actions rather than only producing text, mostly through interfaces rather than by driving the screen.
OpenClaw
A tool that watches a process being performed on screen and turns it into an annotated written guide, which is reading a screen rather than driving one.
Scribe

Common misconceptions

People assume

It is the general solution now that it works.

In fact

It is the method of last resort and improves on the margins rather than in kind. Where a proper connection exists it remains faster, cheaper and vastly more reliable, and choosing to drive a screen instead is choosing fragility for no return.

People assume

If it completes the task, it worked.

In fact

Completion and correctness come apart here more than elsewhere. Something can click through a whole sequence and act on a misread field, and because the failure is visual rather than logical it produces no error, so anything consequential needs its result checked rather than its run monitored.

People assume

The main risk is that it breaks.

In fact

Breaking is the benign failure, because it is visible. The serious risk is what it can reach: it operates as a logged-in user with that user's full reach, and the finer permissions available to a proper connection are simply not there to constrain it.

Telling them apart

Computer use vs Browser agent

Computer use

Anything on the screen, including desktop software.

Browser agent

The web, where pages are the whole world.

The wider one is broader and slower; the narrower one can use the structure of a page rather than only its appearance.

Questions

When is it the right choice?: When there is no other way in and the volume is modest. A supplier portal with no documented interface, an internal application nobody maintains, a process across three systems that will never be connected: in each case the comparison is against doing it by hand, and that comparison it wins comfortably.
How reliable is it in practice?: Reliable enough for supervised work and not for unattended work at volume. Expect it to stop when something on screen changes, which happens more often than people anticipate, and plan for a person noticing rather than for a system recovering on its own.
What about the account it uses?: Give it its own, with the narrowest access that lets the job run, and make the login traceable to it rather than to a person. It cannot be constrained the way a proper connection can, so the account is the only real boundary available and deserves the attention that implies.
Should we wait for a proper connection instead?: If one is genuinely coming, almost always yes. If it is not, build this as the permanent answer rather than as a stopgap, because the difference shows in whether anybody bothers to add checks, monitoring and a named owner, and the ones treated as temporary rarely get those.