OpenBrowse

Structured output

How outputSchema becomes a live, validated answer store with per-field coverage tracking, a completeness gate on done, and refusal of values without evidence.

Pass an outputSchema and OpenBrowse does more than validate the final result. The schema is compiled into a validation model before the run, an empty answer store in that shape is created, and the agent fills it in place with writes that are each checked live. A value that does not fit is rejected at the moment of writing, with a message the agent can act on, rather than surfacing as a malformed blob at the end.

This page covers what the schema may contain, how the store behaves, and why the result you get back does not contain invented data.

What the schema may contain

outputSchema is a JSON Schema (2020-12) object. The supported subset is what Zod and Pydantic emit in practice:

  • object, array and the primitives string, integer, number, boolean
  • Optional fields as anyOf/oneOf with a {"type": "null"} branch, or as a type array ["string", "null"]
  • String enums, enforced as literal types
  • Nested objects and arrays, $ref and $defs, additionalProperties (false forbids unknown keys, anything else allows them), and description on any field

A genuine multi-branch union (more than one non-null type for a field) cannot be represented. In that case the run falls back to prose mode: the schema is appended to the task text, the agent works without the store, and the final answer is validated once at the end, with a single reformat pass if it does not parse.

Field names carry constraints

String fields gain automatic shape guards from their format, or failing that from their name:

FieldGuard
format: "uri", "url", or a name ending in Url, Uri, Href, LinkMust be an absolute http(s) URL. Relative links are rejected with an instruction to resolve them against the page URL
format: "email" or a name ending in EmailMust be an email address
format: "uuid" or a name ending in UuidMust parse as a UUID
A name ending in IdMust be a single token, no whitespace, at most 128 characters

Values are kept byte-identical rather than normalised, because scraped data should round-trip exactly. Declaring any other explicit format opts a field out of the name guards.

Schema design tips

  • Write description fields; they are instructions attached to exactly the field they govern, such as "seniority as stated on the page, null if not stated".
  • Prefer enum for categorical fields. Enum writes are evidence-checked (below), which is a stronger guarantee than free text.
  • Make fields the site might not publish nullable. A required non-nullable field the site never shows can only end in a failed run.
  • If you want unmapped observables kept rather than dropped, give items a field shaped as a list of {key, value} objects (or a plain object field); the draft mapper routes leftovers there.

How the store behaves during a run

The agent writes through a fixed set of actions, and every one validates before it lands:

ActionWhat it does
add_itemAppends one item to the answer array
update_item / update_itemsMerges fields into one item, or many in one call, reporting per-entry failures without aborting the rest
set_fieldSets a top-level, non-list field
remove_itemsDeletes items by index, for duplicates or rows that should never have been records
mark_absentSettles a field the source genuinely does not publish, with a reason
read_output / search_outputReads back or searches what has been built, windowed and elided so a large store never floods the context
add_items_from_file / update_items_from_fileBulk-loads items or merges from a saved JSON file in one step

The same operations are exposed inside the code sandbox, with identical validation, so a script cannot bypass the store's rules. After every successful write the store is mirrored to output.json, and the write's response includes a one-line coverage summary: item count, then item fields grouped as filled on all, partial, empty on all, and marked absent. That summary is the agent's verification; it never needs to dump the whole store to know what is missing.

The guards that stop plausible nonsense

  • Evidence-checked enums. Text from every page read this session accumulates into an evidence corpus. Writing an enum value that no read page states is rejected: rejected: no read page states it. Enum values must be observed on a page, never inferred or defaulted. This is aimed directly at the classic failure of filling seniority: "Senior" because it is a plausible default.
  • The stub throttle. An item whose detail URL has not been visited and which carries no substantial content is a bare list-row stub. The store holds at most two of those; a third add_item is refused with an instruction to read the items' own pages first. This stops a listing page being batch-loaded as the finished answer.
  • Earned absence. mark_absent is refused while item pages remain unread, because absence must be observed, not assumed. It is also refused for a field that has a value on some items once every page has been read: a partial field is already complete, and the honest state of the remaining rows is null.
  • URL, email, UUID and id guards apply at the store boundary too, so a malformed link never becomes part of the answer.

The bulk-read path

For list-shaped extractions, the intended golden path takes four steps regardless of how many records there are:

  1. find_links(...) collects the listing's links with a selector, scrolling the page (and any embedded panel) until the count is stable. It is the only tool that reads links inside a cross-origin embed.
  2. read_pages() opens every found link in parallel tab waves (up to 48 URLs per call, visible in the live view), waits for each page's real content to render, and saves {url, title, text, jsonld, links} per page to pages.json. It also prefills rows_draft.json: one schema row per page, deterministically mapped, the page URL into the item's own-URL field, JSON-LD scalars token-matched onto schema fields (datePublished onto publishedAt), labelled specs harvested from the visible text, the description HTML-stripped, and every value validated against its field before inclusion. What the page does not state stays empty.
  3. add_items_from_file('rows_draft.json') loads all rows in one validated step.
  4. One update_items call fixes the judgement fields, mark_absent settles what no page publishes, and done finishes.

read_pages defends the integrity of step 2 aggressively. If several pages come back with near-identical text while cross-origin embeds exist, they were shell reads (the embedding page instead of each page's real content); they are flagged as failures and automatically retried inside the panel. If sibling pages rendered their embedded panel and one did not, that lone fallback is flagged too. A page that fails because the embed never attached is recorded separately from a genuinely dead URL, and never unlocks mark_absent.

The completeness gate

The first time the agent calls done while schema fields are still empty, the call is bounced with the exact list of deficiencies: unset top-level fields, item fields empty on N of M items, an empty answer array, or a link deficit (find_links captured more usable links than the store holds items, with the unmatched URLs listed). The bounce includes shortcuts when they exist, such as a captured raw key that appears to fill an empty schema field, or a reminder that published dates usually live in each page's JSON-LD rather than its visible text.

The gate is one-shot: the second done is always accepted, so it can never loop. Fields settled with mark_absent do not count as deficiencies, and a field that is partial once every item page has been read is treated as finished. When the gate passes, the feed shows a schema event whose action is completeness with the final coverage summary.

Two consequences worth knowing:

  • The judge reviews the store's actual content (with long values elided for display, never truncated mid-record), so a complete store cannot be failed for looking abbreviated.
  • A run that dies before done but leaves a complete, schema-valid store is still recorded as a success, with a feed entry saying so.

Getting the result

The final output on the session is the store's content, validated once more against the schema; isTaskSuccessful reflects both the agent's completion and that validation.

curl http://<your-host>:8420/v3/sessions/<session-id> \
  -H "Authorization: Bearer $OPENBROWSE_API_KEY" | jq .output

The dashboard's session page can export the same thing (output scope), or the full step log with it; see the live view.

Fields the site never published come back as null, with the absence recorded deliberately during the run. If you expected values there, check the feed for the mark_absent reasons before assuming the run failed; the reason names where the agent looked.

On this page