Blog.
Engineering notes on building the AI-readable web index: AIDocuments, agents that read the web, and the infrastructure behind Lyrenth.
Add web reading to Claude Desktop, Claude Code, or Cursor with one MCP block
A five-minute quickstart: drop one MCP block into your assistant and it reads any URL as a clean AIDocument. Covers read_url, read_urls, and check_usage.
Agents don't browse, they read: what an index for machine readers looks like
Humans browse the web through layout and pixels; AI agents read it as text, structure, and provenance. That difference inverts how an index should be built.
How to feed web pages to an LLM without blowing the context window
Raw HTML burns your context window on nav, scripts, and boilerplate. Here is how to feed web pages to an LLM as clean AIDocuments, with real token numbers.
The publisher's guide to AI bots, robots.txt, and getting represented correctly
Who reads your site now, what a well-behaved AI-index bot looks like, how robots.txt and llms.txt apply, and how to verify your domain with Lyrenth for free.
What is an AIDocument? One clean shape for agents that read the web
An AIDocument is one stable JSON shape for any web page: Markdown body, title, description, structure, and measured economics. Here is how it works.
Why your scraper gets empty HTML from JavaScript sites (and what rendering actually takes)
Fetch a React or SPA site and you get a hollow div, not content. Here is why JavaScript sites return empty HTML, and what real rendering takes at index scale.