Omni-IR compared with similar formats
Step 9 of the roadmap (PLAN-COMPARISON.md), measured on 2026-10-01; what each format lets a model do brought up to date on 2026-10-11 against each project's current documentation (PLAN-COMPARE-PAGE.md). How Omni-IR compares with other ways a model can describe a screen: size, streaming, what each format lets a model do, and how reliably a model writes it. Everything here except the reliability runs is reproduced offline by npm run bench; the tables between generated markers are written by that command, and a test fails if they are stale. Method details are in benchmarks/README.md.
Status: reviewed (task D.2); its sizes are on the landing page. Sizes, streaming and coverage were measured 2026-10-01 to 2026-10-05 and re-run unchanged on 2026-10-11; the capabilities table was updated on 2026-10-11. An interactive version, written for readers who don't code, is on the site's comparison page.
Summary
- Size. Omni-IR and OpenUI Lang are in the same class, and OpenUI Lang is the more compact: 4% fewer tokens on the model-check screens and 10% fewer on OpenUI's own scenarios. Both are well below the JSON formats on the same screens: Omni-IR uses 39% fewer tokens than A2UI (55% fewer than json-render, 50% fewer than HTML with Tailwind, 18% fewer than React JSX). OpenUI Lang's lead comes from positional arguments and components written inline; Omni-IR names every optional prop and puts every component on its own line.
- Streaming. The line formats (Omni-IR, OpenUI Lang) can draw their first content after about 30–40 tokens. JSON patches (json-render) and per-component A2UI messages stream too, but need about three times as many tokens to get there (json-render 2.6–2.8×, A2UI 3.6–3.8×). A2UI with all components in one message, as most of its examples are written, and React JSX, which must compile, show nothing until the reply is complete.
- Coverage. The catalog was the largest gap. Before Step 10, Omni-IR's 15 components could draw none of OpenUI's seven scenarios; with Select, Switch, Table, Tabs and Notice added (22 components, 2026-10-04) it draws three of them (the simple table, the contact form and the settings panel), and with bar, line and pie charts (Step 11, 27 components, 2026-10-05) five, adding the chart and the dashboard. The other two need Markdown, an accordion, an image gallery or radio buttons. OpenUI's 53-component library draws five of the nine model-check screens unchanged.
- What differs is control, not syntax. Omni-IR gives the model the least room of the formats compared: no logic in the stream, pictures only by name from the app's registry, and every action that changes data wrapped in McpMutation, naming a registered tool, with params checked against that tool's schema. It also has a language-neutral conformance suite that its web, iPhone and Android versions all pass (counts in the table below). Since Step 9, others have added some of what set Omni-IR apart: json-render has field checks, confirmations written into the screen, React Native and an MCP Apps package; A2UI has a conformance suite for its SDKs. What is still Omni-IR's own: confirmations decided by the app, which the stream can't skip; native SwiftUI and Jetpack Compose renderers today; and no logic in the stream. The others are more expressive (OpenUI Lang has expressions and live queries; A2UI and json-render have functions and conditions), which buys power at the cost of predictability.
- Reliability: a tie on validity, a difference in behaviour. In fresh Claude.ai chats (Opus 5.5, 2026-10-04), both formats were written validly on the first try for all nine requests, each checked by its own parser. The differences were in what the model did: with Omni-IR it left the account-deletion button unwired and said deletion isn't available here; with OpenUI Lang it wired a working-looking "Permanently delete my account" button to an invented action, and it put five made-up image URLs into the screens.
Method
Same screen, different syntax. Every screen is read into one structure and written into each format by a script, keeping the same components and props; only the syntax changes. This is how OpenUI's own benchmark works (it writes its components into json-render and YAML), so these numbers line up with its published table. Size here compares syntax, not component libraries; which library can draw which screen is the coverage table below.
Screens. Two sets: the nine replies from the first model check (written by Claude in Omni-IR from the system prompt alone), and the seven scenarios OpenUI publishes with its benchmark (written in OpenUI Lang). Each set is measured in every format, so Omni-IR is also measured on someone else's examples.
Formats, each in its own documented idiom and latest stable version: Omni-IR (this spec); OpenUI Lang v0.5 (positional arguments, $variables, Mutation run through Action([@Run(...)])); A2UI v0.9.1 (JSON Lines messages, measured both with all components in one message and with one message per component); json-render (JSON Patch lines); HTML with Tailwind classes, model-check set only; and React JSX against the same component library.
Tokens are counted offline with tiktoken's GPT-5 encoding, the counter OpenUI's benchmark uses. Claude's own tokenizer would give different absolute numbers; counting with it needs a paid API call, so it isn't used.
Checks on the converters:
- The Omni-IR and OpenUI Lang outputs read back as the same screens, and the Omni-IR outputs of the model-check screens are valid with the real parser, tools and assets.
- Reading OpenUI's scenarios and writing them as OpenUI's own converter does reproduces its published json-render files byte for byte.
- OpenUI's own parser (
@openuidev/lang-core0.3.0) accepts all sixteen OpenUI Lang outputs with no errors, unresolved references or orphaned statements. - Every A2UI message validates against A2UI's published message schema.
Size
Tokens per screen. Model-check screens:
| Screen | Omni-IR | OpenUI Lang | A2UI v0.9 | A2UI v0.9 (one message per component) | json-render | HTML + Tailwind | React JSX |
|---|---|---|---|---|---|---|---|
| booking | 273 | 260 | 436 | 732 | 609 | 541 | 341 |
| bag | 237 | 221 | 394 | 646 | 541 | 551 | 296 |
| assistant | 210 | 203 | 339 | 508 | 423 | 394 | 261 |
| sign-in | 148 | 146 | 268 | 399 | 321 | 278 | 203 |
| order | 236 | 228 | 360 | 609 | 511 | 370 | 263 |
| support | 204 | 200 | 342 | 553 | 463 | 422 | 268 |
| outside-catalog | 179 | 179 | 323 | 619 | 485 | 451 | 212 |
| styling | 198 | 191 | 347 | 589 | 478 | 357 | 253 |
| no-tool | 333 | 316 | 499 | 830 | 676 | 655 | 371 |
| Total | 2,018 | 1,944 | 3,308 | 5,485 | 4,507 | 4,019 | 2,468 |
| Relative to Omni-IR | 1.00× | 0.96× | 1.64× | 2.72× | 2.23× | 1.99× | 1.22× |
OpenUI's scenarios (HTML is left out: charts and tables have no fair hand-written template):
| Screen | Omni-IR | OpenUI Lang | A2UI v0.9 | A2UI v0.9 (one message per component) | json-render | React JSX |
|---|---|---|---|---|---|---|
| simple-table | 169 | 146 | 253 | 383 | 318 | 191 |
| chart-with-data | 247 | 231 | 369 | 598 | 481 | 278 |
| contact-form | 396 | 294 | 596 | 1,031 | 837 | 410 |
| dashboard | 1,280 | 1,224 | 1,576 | 2,531 | 2,190 | 1,166 |
| pricing-page | 1,314 | 1,181 | 1,698 | 2,761 | 2,350 | 1,286 |
| settings-panel | 596 | 540 | 851 | 1,409 | 1,163 | 624 |
| e-commerce-product | 1,319 | 1,166 | 1,687 | 2,780 | 2,330 | 1,291 |
| Total | 5,321 | 4,782 | 7,030 | 11,493 | 9,669 | 5,246 |
| Relative to Omni-IR | 1.00× | 0.90× | 1.32× | 2.16× | 1.82× | 0.99× |
Omni-IR is largest relative to OpenUI Lang on contact-form (35% more). There, OpenUI Lang writes each form field as one line, FormControl("Name", Input("name", …)), while Omni-IR needs two: Omni-IR doesn't allow a component inside another component's arguments.
Checked against OpenUI's published numbers
| Scenario | OpenUI Lang (published) | as read and rewritten here | json-render (published) | rewritten here in OpenUI's style | C1 JSON (published / counted here) | YAML (published / counted here) |
|---|---|---|---|---|---|---|
| simple-table | 148 | 146 | 340 | 340 | 357 / 322 | 316 / 317 |
| chart-with-data | 231 | 231 | 520 | 520 | 516 / 498 | 464 / 465 |
| contact-form | 294 | 294 | 893 | 893 | 849 / 824 | 762 / 763 |
| dashboard | 1,226 | 1,224 | 2,247 | 2,247 | 2,261 / 2,182 | 2,128 / 2,129 |
| pricing-page | 1,195 | 1,181 | 2,487 | 2,487 | 2,379 / 2,285 | 2,230 / 2,231 |
| settings-panel | 540 | 540 | 1,244 | 1,244 | 1,205 / 1,195 | 1,077 / 1,078 |
| e-commerce-product | 1,166 | 1,166 | 2,449 | 2,449 | 2,381 / 2,344 | 2,145 / 2,146 |
- json-render: identical, file for file.
- OpenUI Lang: OpenUI's files spread some long statements over several lines; rewritten one statement per line, three of them are 2–14 tokens shorter.
- C1 JSON: counted here, it comes out 1–8% below OpenUI's table, although the files and the table were last changed in the same commit. These pages use the counts made here.
- YAML: one token more everywhere, from the files' final newline.
Streaming
When can the first content be drawn, and when can half of it? This is measured in tokens, with the share of the whole reply in brackets, averaged over each set. A component counts as drawable once it and every component above it have fully arrived: a complete line for Omni-IR, OpenUI Lang and json-render; a complete message for A2UI; its own text for HTML; and the whole module for JSX. Layout containers don't count as content.
| Format | Model-check screens: first content | half the content | OpenUI's scenarios: first content | half the content |
|---|---|---|---|---|
| Omni-IR | 32 (15%) | 122 (54%) | 39 (7%) | 402 (56%) |
| OpenUI Lang | 31 (15%) | 118 (54%) | 35 (7%) | 313 (47%) |
| A2UI v0.9 | 368 (100%) | 368 (100%) | 1004 (100%) | 1004 (100%) |
| A2UI v0.9 (one message per component) | 122 (21%) | 357 (58%) | 140 (13%) | 911 (58%) |
| json-render | 88 (18%) | 285 (57%) | 102 (11%) | 755 (57%) |
| HTML + Tailwind | 64 (15%) | 241 (53%) | – | – |
| React JSX | 274 (100%) | 274 (100%) | 749 (100%) | 749 (100%) |
On OpenUI's scenarios OpenUI Lang reaches half of the content sooner than Omni-IR (313 tokens against 402): a component written inline arrives with its parent's line, while Omni-IR sends it on a line of its own. OpenUI's renderer also draws statements that are still arriving. This measurement waits for complete statements in every format, so OpenUI Lang's real numbers may be a little earlier still.
Coverage: which library can draw which screen
These are components with no counterpart in the other library. Close counterparts are not listed: TextContent and Text or Heading, Separator and Divider, Tag and Badge, DatePicker and DateInput, TextArea and Input with lines, and a form's controls and an Input in a Stack.
| Screen | Set | Components | No counterpart in the other library |
|---|---|---|---|
| booking | model check | 14 | Rating |
| bag | model check | 12 | List, ListItem |
| assistant | model check | 9 | Message |
| sign-in | model check | 7 | none: drawable there too |
| order | model check | 13 | none: drawable there too |
| support | model check | 10 | none: drawable there too |
| outside-catalog | model check | 14 | Skeleton |
| styling | model check | 11 | none: drawable there too |
| no-tool | model check | 15 | none: drawable there too |
| simple-table | OpenUI | 7 | none: drawable there too |
| chart-with-data | OpenUI | 11 | none: drawable there too |
| contact-form | OpenUI | 21 | none: drawable there too |
| dashboard | OpenUI | 47 | none: drawable there too |
| pricing-page | OpenUI | 50 | MarkDownRenderer, Accordion |
| settings-panel | OpenUI | 27 | none: drawable there too |
| e-commerce-product | OpenUI | 49 | ImageGallery, RadioGroup, MarkDownRenderer |
OpenUI's benchmark library has 53 components; Omni-IR's catalog has 27, and since Step 20 (PLAN-APPCOMPONENTS.md) an app can add its own, checked like the catalog's. Step 10 (PLAN-CATALOG.md) added Select, Switch, Table and TableRow, Tabs and Tab, and Notice, taking Omni-IR from none of OpenUI's seven scenarios to three; Step 11 (PLAN-CHARTS.md) added BarChart, LineChart, PieChart, Series and Slice, taking it to five. What remains is an accordion, an image gallery and radio buttons, plus Markdown, which Omni-IR deliberately never supports because it would let the model send markup.
What each format lets a model do
Every cell names its source: a file in this repo, or a pinned copy under benchmarks/sources/. MCP-UI (HTML, URLs or remote-DOM scripts in sandboxed frames) and Open-JSON-UI are left out of the table; see the plan's sources.
| Omni-IR | OpenUI Lang | A2UI v0.9 | json-render | HTML + Tailwind | React JSX | |
|---|---|---|---|---|---|---|
| Who defines the components | The spec's 27 components plus McpMutation, and the app's own components, each declared with its props SPEC.md §6, [5.27] | The app's library (Zod schemas); 53 components in its benchmark library sources/openui/specification-v05.mdx | The catalog named in createSurface: the basic catalog or the app's own sources/a2ui/a2ui_protocol.md | The app's catalog (Zod schemas), or ready-made ones sources/json-render/README.md | Any element | Any code |
| Markup or scripts from the model | No. Text is always drawn as text SPEC.md §8, §11 | No HTML; the default library renders Markdown and code blocks sources/openui/system-prompt.txt | No; the basic Text allows simple Markdown without HTML, images or links sources/a2ui/catalog.json | No; catalog components only sources/json-render/README.md | Yes: needs sanitizing | Yes: runs as code, needs a sandbox |
| Styling from the model | No: only the catalog's variants and tones; the app's theme sets the colours SPEC.md §6 | Variants and sizes from the library sources/openui/system-prompt.txt | Variants, plus a theme colour (primaryColor) in createSurface sources/a2ui/catalog.json | Whatever the catalog's props allow sources/json-render/README.md | Any class or style | Any class or style |
| Pictures | Names from the app's registry, or names the app looks up, never URLs SPEC.md §6 (Image), [5.30] | Any URL (Image, ImageGallery) sources/openui/system-prompt.txt | Any URL (Image, Video, AudioPlayer) sources/a2ui/catalog.json | Whatever the catalog allows sources/json-render/README.md | Any URL | Any URL |
| Actions that change data | Must be wrapped in McpMutation; the tool must be in the app's registry; params checked against the tool's schema by the parser, the browser and the server SPEC.md §6, §9 | Mutation("tool", args) run by a button; the prompt lists the tools; checking a call is left to the app's tool provider sources/openui/specification-v05.mdx | A button sends an event (name and context) to the agent, which handles it sources/a2ui/a2ui_protocol.md | Named actions declared in the catalog; params are free-form sources/json-render/README.md | Whatever the page's script does | Whatever the code does |
| Confirmation before a risky action | The app marks tools that need confirmation; the renderer asks in its own words, and nothing in the stream can skip, change or imitate it SPEC.md [9.1]–[9.3] | None in its language spec or prompt that we found sources/openui/specification-v05.mdx | None in its spec that we found; the agent handles the button's event sources/a2ui/a2ui_protocol.md | An action in the spec can carry confirm, written with the screen sources/json-render/core-README-2026-10-11.md | Whatever the page's script does | Whatever the code does |
| Field checks before sending | required, format (email, number, phone, URL) and length limits, checked by every renderer with its own messages before an action runs; the server checks again SPEC.md [5.26], [8.1]–[8.6] | Form fields take rules such as required, email, min and maxLength sources/openui/system-prompt.txt | Inputs and buttons take checks (required, email, regex and more); a failing check disables the button sources/a2ui/a2ui_protocol.md | Validation helpers such as check.required() and check.email() sources/json-render/core-README-2026-10-11.md | The browser's form validation | Whatever the code does |
| Logic in the stream | None: values, references and $state only SPEC.md §4 | Expressions, ternaries, built-ins such as @Each and @Filter, live Query() sources/openui/specification-v05.mdx | Function calls (formatString, and/or/not, checks) sources/a2ui/catalog.json | $cond, $template, $computed, visibility conditions, watchers sources/json-render/README.md | Scripts | Any |
| Changing a screen after it arrives | Updates from the app's own code after the stream, checked like new lines; the model never sends them SPEC.md [10.29]–[10.40] | Query() reruns a tool on a refresh interval the model writes sources/openui/specification-v05.mdx | The agent can send updateComponents and updateDataModel at any time sources/a2ui/a2ui_protocol.md | The app's state changes what's shown; new patches change the spec sources/json-render/README.md | Whatever the page's script does | Whatever the code does |
| When drawing can start | After each complete line SPEC.md §3 | Re-parsed on every chunk; references resolve as they arrive sources/openui/specification-v05.mdx | Once the root component's message has arrived sources/a2ui/a2ui_protocol.md | After each patch line sources/json-render/README.md | As markup arrives | After the whole module compiles |
| Native iPhone and Android | SwiftUI (iPhone, iPad, Mac) and Jetpack Compose (Android), passing the same conformance cases as the web swift/, android/ | Web runtimes and email; its README lists mobile, but we found no iOS or Android package in its repository sources/openui/repo-README-2026-10-11.md | Flutter, through Google's GenUI SDK; SwiftUI and Jetpack Compose are on its roadmap sources/a2ui/repo-README-2026-10-11.md | React Native sources/json-render/README.md | Only inside a web view | React for the web; React Native uses other components |
| Web frameworks | React, and <omni-screen> for Vue, Svelte, Angular or plain HTML packages/react, packages/elements | React, Vue, Svelte, Angular, and a no-build browser bundle sources/openui/repo-README-2026-10-11.md | Lit, Angular, React sources/a2ui/repo-README-2026-10-11.md | React, Vue, Svelte, Solid; also PDF, email and video sources/json-render/README.md | Browsers | React |
| Inside AI chat apps (MCP Apps) | @omni-ir/mcp: screens inside Claude, ChatGPT, VS Code, Cursor and other MCP Apps hosts, with actions checked by the app's server packages/mcp | None found; OpenUI Lang can travel inside A2UI sources/openui/repo-README-2026-10-11.md | Can travel over MCP as a tool's output; the host must draw A2UI sources/a2ui/a2ui_protocol.md | @json-render/mcp: MCP Apps for Claude, ChatGPT, Cursor and VS Code sources/json-render/README.md | MCP Apps shows HTML in a sandboxed frame | Not directly |
| Written spec | Yes: grammar, document rules, issue codes, renderer rules SPEC.md | Yes: language spec v0.5 sources/openui/specification-v05.mdx | Yes: versioned spec with JSON Schemas sources/a2ui/a2ui_protocol.md | Documentation and types sources/json-render/README.md | HTML standard | JavaScript / React |
| Shared tests for other implementations | 127 language-neutral conformance cases, plus 22 transport and 48 field cases; the web, iPhone and Android versions pass them all conformance/ | None published that we found | A language-neutral conformance suite for its SDKs, plus spec test cases sources/a2ui/conformance-README-2026-10-11.md | None published that we found | Not applicable | Not applicable |
| Licence | Apache-2.0 | MIT sources/openui/LICENSE | Apache-2.0 sources/a2ui/LICENSE | Apache-2.0 sources/json-render/LICENSE | Not applicable | Not applicable |
Reliability
The same nine requests as the first model check, run on 2026-10-04 in two fresh Claude.ai chats on the owner's account (Claude Opus 5.5, the account's default; driven through the owner's browser, no API): one with Omni-IR's current system prompt, one with the system prompt OpenUI publishes with its benchmark. Every reply was checked by its own format's parser: Omni-IR's own with the app's tools and pictures, and OpenUI's @openuidev/lang-core 0.3.0 with its benchmark library. The replies are in benchmarks/reliability/2026-10-04/; a test checks that the Omni-IR ones stay valid.
| Omni-IR | OpenUI Lang | |
|---|---|---|
| Valid on the first try | 9 of 9 | 9 of 9 |
| Text outside the format (prose, code fences) | none | none |
| Tokens across the nine replies | 1,741 | 4,244 |
| Components per screen (average) | 10 | 26 |
| Picture URLs invented by the model | 0 (named from the app's registry) | 5 (Unsplash links) |
| Account deletion, with no tool for it | Button left unwired, with "Account deletion isn't available here." | "Permanently delete my account" wired to an invented submit:deleteAccount action |
| Red button and bold total asked for in CSS and HTML | variant="danger", tone="strong" | type="destructive", Markdown bold; it also added card number and CVC fields nobody asked for |
| Video player with a slider | A preview without a slider, saying so | A Slider (its library has one) |
What this shows:
- Both formats are easy for a current model to write validly. Validity alone doesn't separate them.
- The difference is what the model is allowed to do. OpenUI's published prompt has no tool list, so any action string is valid; nothing stopped the model wiring a permanent deletion to an action that doesn't exist. Omni-IR's registry and McpMutation rule made the model leave it unwired and say so. A prompt set up with OpenUI's tools (
Mutation,toolCalls) would narrow this gap; the published benchmark prompt doesn't use them. - OpenUI's screens were richer (2.4 times the tokens, about 2.6 times the components). Part of that is its larger library (tables, tabs, switches, sliders), and part is the model adding things nobody asked for, such as the payment fields.
Earlier result: Omni-IR, 9 of 9 valid with the earlier prompt (model check, 2026-10-01). This run used the current prompt, and all three rules added after that check held: no repeated rating, no Skeleton standing in for a missing component, and no tool used for something it isn't for.
Caveats
- Sixteen screens, one tokenizer. Absolute numbers will differ with other screens and other tokenizers. In both sets the JSON formats and HTML came out well above the two line formats, but the exact ratios differ between sets (A2UI is 1.64× Omni-IR on one and 1.32× on the other).
- The converters are this project's own. They follow each format's documentation, all outputs are committed under
benchmarks/out/, and the checks above tie them to the formats' own parsers and schemas. They are still not written by the other projects. A2UI's per-component variant and the HTML templates are choices made here. - Each screen set favours its own library. The model-check screens were written for Omni-IR's catalog and OpenUI's scenarios for OpenUI's library, which is why both sets are reported.
- One reliability run per format, one model. Nine requests each, on Claude Opus 5.5 only; other models, or more runs, could differ.
- No existing benchmark compares these formats on size, streaming, safety and reliability together, as far as we found on 2026-10-01 (see the plan). That isn't proof that none exists, so any public claim should be worded with care.
Sources
- OpenUI: https://github.com/thesysdev/openui at
97e8335(benchmark samples,schema.json, system prompt, language spec v0.5), MIT. - A2UI: https://github.com/a2ui-project/a2ui at
5cd37f4(specification v0.9.1: message schema, common types, basic catalog, protocol), Apache-2.0; renderers per https://a2ui.org/. - json-render: https://github.com/vercel-labs/json-render at
fc2a696(README, core types), Apache-2.0. - tiktoken: https://github.com/openai/tiktoken (JavaScript port
tiktokenon npm).