Pegasus 1.5 Turns Video Review Into Timestamped Segments. Human Review Still Matters

A two-hour recording is not hard to store; it is hard to search when nobody remembers where the relevant moment occurs. Pegasus 1.5 changes the question from “summarize this video” to “find every segment matching these definitions and return its time range and fields.” Amazon’s October 19 Bedrock article describes structured time-based metadata, video segmentation, reference-image prompts and processing for long clips. That can turn a passive archive into an index, but the index still needs a human quality gate.

From general answers to defined segments

Pegasus 1.5 supports a general analysis mode and a time_based_metadata mode. In the latter, the caller supplies segment definitions with descriptions and fields; the response contains segments with start and end times plus the requested values. The model can also return definition errors, and responses include a stop reason that indicates whether generation completed or reached its output limit. Those fields matter to a production pipeline: malformed or partial output should not silently become searchable truth. ^[2]

Possible workflows include finding speaker changes in an interview, identifying a play type in sports footage, or locating editorial topics in a broadcast archive. The model analyzes what it sees, hears and reads on screen, according to AWS. These are examples of intended use, not independent evidence of accuracy for any particular archive. ^{[1] [4]}

Choose the input and region deliberately

The Bedrock model documentation lists input limits of up to two hours and less than 2 GB for S3-hosted video, or up to 25 MB for base64 input. The API accepts InvokeModel and streaming calls. Current documentation says the model is available through cross-Region inference only, using the geo or global inference IDs rather than the base ID. Availability, inference profile and data-handling requirements therefore belong in the design review, not as a late deployment surprise. ^{[2] [3]}

The AWS announcement says the model was initially available through the US and Global cross-Region profiles, and the newer technical post identifies the US West (Oregon) launch region. Because model availability can change, check the live model card and supported-region list when planning a deployment. Charges are billed through AWS Marketplace under TwelveLabs, not grouped as ordinary Bedrock model charges, according to the AWS post. ^{[1] [3]}

Build an index with a review boundary

Start with a small, representative archive. Define a controlled vocabulary for the moments that matter, such as “speaker change,” “goal attempt” or “product failure,” and specify the fields the downstream system actually needs. Keep definitions precise enough that different reviewers would recognize the same segment. Run the model on clips whose ground truth your team can review, then measure missed events, false detections, timestamp error and field completeness.

Treat each segment as a candidate record. Store the source video identifier, model and prompt version, processing time, returned timestamps and review state. A person should be able to open the exact interval and confirm the label, especially for safety investigations, legal review, journalism or decisions affecting a person. A model summary is a navigation aid; the original footage remains the evidence.

For large libraries, include cost and latency monitoring. Pegasus publishes invocation, output-token, latency and client-error metrics in CloudWatch, and AWS describes application inference profiles for attribution. Use those signals to estimate cost per reviewed minute, not just model calls. A cheap automated pass can still create expensive editorial work if its segments are poor. ^[1]

Searchability is a workflow, not a model response

Timestamped output creates a useful bridge between generative video understanding and ordinary databases: define a schema, extract candidate segments, validate them, then index approved records for search. The model does not decide which categories your organization needs, whether a segment is important, or whether it can be published. Those remain product and editorial decisions.

The strongest early use is likely a queue that helps a human find moments faster, rather than automatic publication or evidence extraction. Measure whether reviewers locate relevant footage with less effort while maintaining the same accuracy bar. If the system cannot link every label to a reviewable interval in the original, it is not yet a dependable archive index.

Related reading: how multimodal models tokenize images, audio and video and multimodal local search.