Modalities and media

Producing music, sound effects or other audio from a written description, distinct from reading text aloud in a chosen voice.

A presenter-style video built from a synthetic person reading a script, used for training and internal communications without filming anyone.

The field concerned with getting software to interpret what is in an image or video: objects, faces, defects, movement.

The kind of model behind most image generators, which starts from visual noise and refines it step by step towards the picture your description asks for.

Pulling structure and meaning out of documents such as invoices or contracts, rather than only converting them to plain text.

Changing an existing image by instruction, such as removing a background or replacing an object, rather than generating one from nothing.

Producing a new picture from a written description rather than retrieving or editing an existing one.

Converting text from one language to another automatically, now good enough for most working use and still worth reviewing where wording carries obligation.

Able to work with more than one kind of input or output: text, images, audio, sometimes video.

Reading the text inside a picture of a page, so a scan or a photograph becomes words a computer can search and copy.

Turning recorded or live speech into written words, which is what produces a meeting transcript or live captions.

Turning a written description straight into a video clip, the video counterpart of asking for an image in words.

Turning written words into spoken audio, used for narration, accessibility and assistants that answer out loud.

Producing moving footage from a written description or a still image, typically in short clips that are then edited together.

A model that reads images and text together, so it can answer questions about a screenshot, chart, form or photograph.

AI you speak to and that speaks back, joining speech recognition, a language model and synthetic speech into one conversation.

Building a synthetic copy of a particular person's voice from samples, which raises a consent question before any practical one.