AIDocument v2.0

{
  "schema": {
    "name":    "AIDocument",
    "version": "2.0",
    "ref":     "aidoc:sha256:7a14e9b2d8f3c901b42e5a77c0f19a34"
  },
  "source": {
    "url":              "https://example.com/page",
    "canonical_url":    "https://example.com/page",
    "fetched_at":       "2026-05-13T...",
    "render_mode":      "static" | "rendered" | "static_after_render_failure",
    "status_code":      200,
    "freshness_policy": "cache_first" | "force_refresh"
  },
  "cache": {
    "status":           "hit" | "miss" | "refreshed" | "stale_revalidated",
    "origin_contacted": false,
    "body_fetched":     false
  },
  "identity": {
    "title":        "...",
    "description":  "...",
    "language":     "en",
    "content_type": "article"
  },
  "content": {
    "markdown": "# ..."
  },
  "structure": {
    "headings":        [{ "level": 1, "text": "...", "id": "..." }],
    "links":           [{ "url": "...", "text": "...", "internal": true, "rel": "..." }],
    "images":          [{ "url": "...", "alt": "..." }],
    "structured_data": { /* parsed JSON-LD blocks, merged into one object */ }
  },
  "signals": {
    "word_count":           1552,
    "reading_time":         7,
    "has_json_ld":          true,
    "heading_hierarchy_ok": true
  },
  "economics": {
    "raw_html_tokens_approx": 121803,
    "output_tokens_approx":   2460,
    "token_savings":          119343,
    "token_savings_percent":  0.978,
    "estimated_cost_usd": {
      "raw_html":   0.366,
      "our_output": 0.0074,
      "savings":    0.3586
    },
    "pricing_basis": {
      "input_price_per_1k_usd": 0.003,
      "model_class":            "frontier-1m-context"
    }
  }
}

Contract guarantees

• Backward-compatible additions only. New top-level groups and new fields within existing groups may appear over time; existing fields never change shape within a major version.
• Breaking changes earn a new major version. v3 would be a new endpoint or accept-header negotiation; existing v2 callers continue working unchanged.
• Cache state is observable. cache.status + the two booleans tell you exactly what happened on our side for this call. 304 Not Modified is unambiguous; force_refresh is distinguishable from a regular cache miss.
• Render mode is observable. source.render_mode tells you whether the bytes came from static fetch or headless Chromium. You can filter or score by this if your downstream cares.
• Status codes are real. source.status_codereflects the network event, not the rendered DOM. A 404 stays a 404 regardless of whether the page's SPA shell renders something.
• Economics numbers are computed, not estimated. The token counts are approximate (tokenizer approximation), but the dollar math derives from real token rates configured server-side and surfaced in economics.pricing_basis.
• Document refs are stable. schema.ref hashes the non-volatile document fingerprint (URL, title, markdown, structure). A re-crawl of an unchanged page produces the same ref; a re-crawl with changed content produces a new one. Use it for citation.

Index lookup: GET /v1/document + fields=

GET /v1/document?url=... is the index-only lookup: it returns a cached AIDocument if we already have one for that URL and 404s otherwise (never crawls). It supports a fields= projection that trims the response to only the listed top-level keys for callers who want a smaller payload. The resolving endpoint POST /v1/aidocument currently returns the full grouped envelope; per-group projection support on /v1/aidocument is on the roadmap.

Examples

# Full cached AIDocument (default)
GET /v1/document?url=https://example.com/post

# Title + markdown only: what most reading agents need
GET /v1/document?url=https://example.com/post&fields=title,markdown

# Title, description, headings: for routing / classification
GET /v1/document?url=https://example.com/post&fields=title,description,headings

# Lightest answer: title + canonical URL
GET /v1/document?url=https://example.com/post&fields=title,canonical_url

# With economics: numbers reflect the projected payload, not the full doc
GET /v1/document?url=https://example.com/post&fields=title,markdown,economics

Allowed fields

Any of the top-level keys on the cached document:

canonical_url   crawl           description     economics
headings        images          links           markdown
meta            structured_data title           url

Unknown field names return HTTP 400 with aninvalid_fields error and the full allowed-set echoed back, so typos surface immediately. The cached shape is flat; the v2 grouped envelope above is the wire shape that ships through POST /v1/aidocument.