From aaa857bbc83e427617387a245c22202e479db80b Mon Sep 17 00:00:00 2001 From: Tanner Linsley Date: Thu, 1 Oct 2026 08:33:47 -0600 Subject: [PATCH] docs: clarify Markdown contracts and measurements --- README.md | 9 +++------ docs/comparison.md | 20 +++---------------- docs/core-concepts/document-model.md | 4 ++-- docs/core-concepts/syntax-profile.md | 2 +- docs/guides/ai-streaming.md | 4 ++-- docs/guides/docs-preset.md | 2 +- docs/guides/octane.md | 2 +- docs/guides/performance.md | 17 +++------------- docs/installation.md | 4 ++-- docs/overview.md | 8 ++++---- docs/project/architecture.md | 2 +- docs/project/faq.md | 2 +- docs/reference/octane.md | 2 +- skills/_artifacts/domain_map.yaml | 6 +++--- skills/_artifacts/skill_spec.md | 2 +- skills/render-markdown/SKILL.md | 5 +++-- .../references/ast-and-options.md | 6 ++++-- 17 files changed, 36 insertions(+), 61 deletions(-) diff --git a/README.md b/README.md index 7a6e5b1..2fc157a 100644 --- a/README.md +++ b/README.md @@ -20,17 +20,14 @@ A tiny, fast, deterministic Markdown parser and renderer for blogs and documentation. -- 5.2 KB gzip parser -- 7.0 KB gzip HTML renderer -- 7.0 KB gzip React adapter -- 7.0 KB gzip Octane adapter +- separately importable parser, HTML renderer, React adapter, and Octane adapter - zero runtime dependencies - serializable AST - safe defaults for raw HTML and executable URLs - optional docs extensions and external syntax highlighting - optional AI streaming profile -Bundle sizes include the parser and exclude framework runtimes and syntax highlighters. +The [generated bundle report](./reports/sizes.md) records minified, gzip, and Brotli sizes for reproducible browser import profiles. Renderer measurements include the parser and exclude framework runtimes and syntax highlighters; complete public-entry measurements are listed separately. ```bash pnpm add @tanstack/markdown @@ -60,7 +57,7 @@ export function Article({ source }: { source: string }) @{ } ``` -TanStack Markdown targets controlled technical content. It supports the Markdown used by blogs and docs, then spends its remaining complexity budget on deterministic output, renderer parity, malformed-input resilience, and small entry points. It is intentionally not a complete CommonMark, GFM, MDX, or general content-processing implementation. +TanStack Markdown targets controlled technical content. It supports the Markdown used by blogs and docs, with deterministic output, renderer parity, malformed-input limits, and separately importable entry points. It is intentionally not a complete CommonMark, GFM, MDX, or general content-processing implementation. For syntax highlighting, use the tested [TanStack Highlight adapter](./docs/guides/syntax-highlighting.md#tanstack-highlight-adapter). It registers only the languages you choose and returns escaped token markup inside Markdown-owned code containers. diff --git a/docs/comparison.md b/docs/comparison.md index 166aebb..a926bd8 100644 --- a/docs/comparison.md +++ b/docs/comparison.md @@ -36,23 +36,9 @@ Markdown libraries optimize for different jobs. TanStack Markdown is designed fo ## Measured browser size -These repository benchmarks bundle representative browser entry points from pinned dependencies, minify them with esbuild, and compress them. They are reproducible with `pnpm run size`. +The repository's [generated size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) bundles representative browser entry points from pinned dependencies, minifies them with esbuild, and records gzip and Brotli bytes. Reproduce it with `pnpm run size`. -| Entry | Gzip | Brotli | -| --- | ---: | ---: | -| `@tanstack/markdown/parser` | 5.2 KB | 4.8 KB | -| `@tanstack/markdown/html` | 7.0 KB | 6.5 KB | -| `@tanstack/markdown/react` | 7.0 KB | 6.5 KB | -| React with streaming extension | 7.2 KB | 6.6 KB | -| `@tanstack/markdown/octane` | 7.0 KB | 6.4 KB | -| Marked | 12.5 KB | 11.5 KB | -| micromark | 15.4 KB | 13.7 KB | -| markdown-wasm JS + WASM | 31.3 KB | 26.4 KB | -| unified + remark + rehype | 36.8 KB | 32.7 KB | -| commonmark.js | 48.1 KB | 39.8 KB | -| markdown-it | 52.7 KB | 44.0 KB | - -The comparison does not represent equivalent feature sets. It shows the cost of each measured path for this repository’s rendering benchmark. See the generated [size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) for exact bytes and versions. +The report distinguishes selected-function imports from the complete namespace of every TanStack Markdown public entry. Framework runtimes are externalized from adapter measurements. Comparison packages have different feature sets, so the results describe those import profiles rather than equivalent capabilities or the final size of your application. ## Compatibility accounting @@ -62,4 +48,4 @@ Use [commonmark.js](https://github.com/commonmark/commonmark.js), micromark, or ## Performance -Across the maintained fixtures, TanStack Markdown is competitive with the JavaScript renderers in the suite, but it is not the fastest result in every fixture. Pre-parsed AST rendering is its cheapest path. The defensible advantage is the combined size, output contract, and focused feature set. See [Performance](./guides/performance.md) for methodology and current results. +Across the maintained fixtures, TanStack Markdown is competitive with the JavaScript renderers in the suite, but it is not the fastest result in every fixture. Pre-parsed AST rendering is its cheapest path. Compare the measured import sizes and supported syntax with the rendering and extension APIs your application needs. See [Performance](./guides/performance.md) for methodology and current results. diff --git a/docs/core-concepts/document-model.md b/docs/core-concepts/document-model.md index 1878cce..f15281e 100644 --- a/docs/core-concepts/document-model.md +++ b/docs/core-concepts/document-model.md @@ -84,8 +84,8 @@ await cache.set(key, JSON.stringify(document)) const html = renderHtml(document) ``` -Parsing once is useful for build pipelines, content indexes, multiple render targets, and high-traffic SSR paths. Rendering a pre-parsed AST is also the fastest measured path in the package. +Parsing once is useful for build pipelines, content indexes, multiple render targets, and high-traffic SSR paths. Passing a document to a renderer skips parsing the source again. ## Stability -The AST is public and typed, but the package is still pre-1.0. Pin versions when persisting documents across deployments, and regenerate cached AST data when upgrading across a release that changes node contracts. +The documented AST follows the [Version 1 compatibility policy](../project/version-one.md). Existing valid 1.x documents remain supported by later 1.x renderers. Record the producing package version with persisted documents, and rebuild caches to adopt parser fixes. Byte-for-byte rendered HTML is not a stable serialization contract, and untrusted JSON still requires validation before rendering. diff --git a/docs/core-concepts/syntax-profile.md b/docs/core-concepts/syntax-profile.md index 9d1b3af..0e3191d 100644 --- a/docs/core-concepts/syntax-profile.md +++ b/docs/core-concepts/syntax-profile.md @@ -75,4 +75,4 @@ The following are not project goals: - syntax highlighting, themes, or language grammars - asynchronous plugin pipelines -Unsupported input must still be deterministic, escaped by default, and bounded in runtime. New syntax requires evidence from real documentation, regression fixtures, renderer parity, and an accepted bundle cost. +Unsupported input must still be deterministic and escaped by default. Core parsing limits nesting and inline scans, but does not cap total input size or arbitrary extension work; see [Resource limits](./security.md#resource-limits). New syntax requires evidence from real documentation, regression fixtures, renderer parity, and an accepted bundle cost. diff --git a/docs/guides/ai-streaming.md b/docs/guides/ai-streaming.md index fb3d5fe..3329713 100644 --- a/docs/guides/ai-streaming.md +++ b/docs/guides/ai-streaming.md @@ -55,7 +55,7 @@ export default function Response() { ``` -The streaming extension suppresses empty trailing headings, blockquotes, and list items while a response is incomplete. It does not change completed paragraphs, lists, tables, quotes, or fenced code. An unclosed code fence renders all code accumulated after its opening fence. +The streaming extension suppresses empty trailing headings, blockquotes, and list items while a response is incomplete. This transform leaves other nodes unchanged, but the next update reparses the full source. Later text, such as a reference definition, can change how earlier content parses. An unclosed code fence renders all code accumulated after its opening fence. Keep the extension enabled after completion unless an intentionally empty final heading, quote, or list item is meaningful in your application. Without the extension, core parsing preserves those valid Markdown structures. @@ -65,7 +65,7 @@ Unclosed emphasis, code spans, links, and other inline delimiters remain literal The package reparses the accumulated response rather than maintaining parser state between updates. Batch very small transport tokens into normal UI updates when responses are unusually long or tokens arrive faster than the screen should repaint. -With the streaming extension enabled, React keeps completed groups of plain code lines in stable text nodes. Appending code updates the trailing group instead of replacing the entire block's text, which reduces browser layout work. Custom code components still receive string children, and highlighters keep their existing HTML rendering path. +With the streaming extension enabled, React keeps completed groups of plain code lines in stable text nodes. Appending code updates the trailing group instead of replacing the entire block's text, which avoids updating completed text groups on each append. Custom code components still receive string children, and highlighters keep their existing HTML rendering path. ## Security diff --git a/docs/guides/docs-preset.md b/docs/guides/docs-preset.md index b3c7e64..d1298f8 100644 --- a/docs/guides/docs-preset.md +++ b/docs/guides/docs-preset.md @@ -19,7 +19,7 @@ const html = renderHtml(document, { }) ``` -The preset is a separate 2.3 KB gzip entry and is not imported by the parser or renderers. +The preset is a separate entry and is not imported by the parser or renderers. See the [generated size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) for measured import profiles. ## Callouts diff --git a/docs/guides/octane.md b/docs/guides/octane.md index 96faf2b..f434887 100644 --- a/docs/guides/octane.md +++ b/docs/guides/octane.md @@ -12,7 +12,7 @@ The Octane adapter renders Markdown source or a pre-parsed document directly to pnpm add @tanstack/markdown octane ``` -The adapter requires `octane@0.1.12` or newer. Octane is an optional peer dependency and is not imported by any other package entry. +The adapter is tested with `octane@0.1.12`. Octane is an optional peer dependency and is not imported by any other package entry. Its peer range permits newer versions without guaranteeing compatibility with future pre-1.0 Octane changes; see [Version 1 compatibility](../project/version-one.md). ## Component usage diff --git a/docs/guides/performance.md b/docs/guides/performance.md index 651dc9f..9400d1c 100644 --- a/docs/guides/performance.md +++ b/docs/guides/performance.md @@ -8,20 +8,9 @@ TanStack Markdown’s performance strategy is architectural: keep parsing synchr ## Current measurements -The generated browser bundle report records: +The generated [size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) records minified, gzip, and Brotli bytes for browser bundles built with esbuild. It includes selected-function imports and, separately, the complete namespace of each public package entry. Those profiles can have different sizes; neither predicts every application's final bundle. -| Entry | Gzip | Brotli | -| --- | ---: | ---: | -| parser | 5.2 KB | 4.8 KB | -| HTML renderer | 7.0 KB | 6.5 KB | -| React adapter | 7.0 KB | 6.5 KB | -| Octane adapter | 7.0 KB | 6.4 KB | -| React adapter with streaming extension | 7.2 KB | 6.6 KB | -| Streaming extension | 0.3 KB | 0.3 KB | -| docs preset | 2.3 KB | 2.1 KB | -| callouts extension | 0.3 KB | 0.3 KB | - -Framework runtimes are externalized from their adapters, and the highlighter measurement uses only a callback stub. Exact generated bytes live in the [size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md). +Renderer measurements include the parser. Framework runtimes are externalized from their adapters, and the external-highlighter measurement uses only a callback stub. Check the report's generation date and reproduce it with `pnpm run size` when comparing a release. The maintained benchmark records parse, pre-parsed rendering, parse-and-render, external-highlighter, and progressive AI-response paths against pinned comparison packages. Timings vary by runtime, CPU, fixture shape, and dependency version, so the generated [benchmark report](https://github.com/TanStack/markdown/blob/main/reports/benchmarks.md) is the only source of current CPU results. @@ -40,7 +29,7 @@ Avoid UI adapters outside their matching framework. Import individual extensions ## Parse ahead of rendering -For content that changes less often than it is viewed, parse during ingestion or the build. Cache the serializable AST and render it for each target. The measured AST renderer is several times faster than parsing and rendering together. +For content that changes less often than it is viewed, parse during ingestion or the build. Cache the serializable AST and render it for each target. Passing an AST skips the parser on each render. The benchmark report compares that path with parsing and rendering together for each fixture. ## Keep highlighting external diff --git a/docs/installation.md b/docs/installation.md index 9ba1326..e2a73b1 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -49,7 +49,7 @@ React is a peer dependency and is not bundled into the adapter. ## Octane usage -The Octane adapter requires Octane 0.1.12 or newer: +The Octane adapter is tested with Octane 0.1.12: ```bash pnpm add @tanstack/markdown octane @@ -63,7 +63,7 @@ export function Article({ source }: { source: string }) @{ } ``` -Octane is an optional peer dependency and is not bundled into the adapter. +Octane is an optional peer dependency and is not bundled into the adapter. Its peer range permits newer releases without guaranteeing compatibility with future pre-1.0 Octane changes. See [Version 1 compatibility](./project/version-one.md) for tested versions and the React 19 requirement when both frameworks are installed. ## Runtime and module format diff --git a/docs/overview.md b/docs/overview.md index 0e7eb5a..4280f91 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -25,17 +25,17 @@ General Markdown processors optimize for broad conformance, plugin ecosystems, o - code metadata for documentation UI - a small browser bundle -TanStack Markdown spends its complexity budget on that path. It deliberately does not implement every CommonMark edge case, MDX evaluation, automatic linkification, or a general asynchronous processing ecosystem. +TanStack Markdown supports that workflow through separately importable renderers and optional docs extensions. It deliberately does not implement every CommonMark edge case, MDX evaluation, automatic linkification, or a general asynchronous processing ecosystem. ## Core properties ### Small entry points -Current minified browser bundles are 5.2 KB gzip for the parser, 7.0 KB for HTML rendering, and 7.0 KB for either UI adapter with its framework runtime externalized. The generated [bundle report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) is the source of truth. +Import the parser, HTML renderer, or matching UI adapter separately. The generated [bundle report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) records minified browser bundles with framework runtimes externalized, including both selected-function imports and complete public entry points. Reproduce it with `pnpm run size`; your application’s imports and bundler configuration determine its final size. ### Parse once, render many -`parseMarkdown` returns plain objects and arrays. The result can be serialized, cached, inspected, transformed, and passed to either renderer. +`parseMarkdown` returns plain objects and arrays. The result can be serialized, cached, inspected, transformed, and passed to the HTML, React, or Octane renderer. ### Safe defaults @@ -47,7 +47,7 @@ The supported contract is the [TanStack docs syntax profile](./core-concepts/syn ### AI streaming without parser state -The optional [AI streaming profile](./guides/ai-streaming.md) reparses accumulated response text and suppresses incomplete trailing block placeholders. It adds 0.2 KB gzip to the React path while leaving the core parser and renderers unchanged. +The optional [AI streaming profile](./guides/ai-streaming.md) reparses accumulated response text and suppresses incomplete trailing block placeholders. It is imported separately. Each update still parses the complete accumulated source; batch updates for long responses or very frequent transport tokens. ## Choose your starting point diff --git a/docs/project/architecture.md b/docs/project/architecture.md index 6a13889..c6a9447 100644 --- a/docs/project/architecture.md +++ b/docs/project/architecture.md @@ -48,7 +48,7 @@ The parser never imports a renderer. The renderers never import a highlighter. T The extension API handles the common need to add a docs block or derive metadata without paying for a general compiler pipeline. It intentionally does not implement async plugins, arbitrary virtual files, source maps, JSX evaluation, or cross-format compilation. -That boundary is the product: use a unified or MDX stack when those capabilities are central, and use TanStack Markdown when the controlled docs renderer is enough. +Use these hooks to recognize a custom block or inline marker, collect headings, or replace HTML for a node. Choose a unified or MDX stack when you need asynchronous transforms, compiler integration, or executable JSX. ## Release gates diff --git a/docs/project/faq.md b/docs/project/faq.md index f097ce1..2fa8ae8 100644 --- a/docs/project/faq.md +++ b/docs/project/faq.md @@ -6,7 +6,7 @@ title: FAQ ## Is TanStack Markdown CommonMark compliant? -No. It continuously measures CommonMark behavior and preserves established matches, but full conformance is not the product goal. The [generated compatibility report](https://github.com/TanStack/markdown/blob/main/reports/conformance.md) records 403 of 652 CommonMark 0.31.2 examples matching after output normalization in v0.0.14. +No. It continuously measures CommonMark behavior and preserves established matches, but full conformance is not the product goal. The [generated compatibility report](https://github.com/TanStack/markdown/blob/main/reports/conformance.md) records matches against all 652 CommonMark 0.31.2 examples after output normalization, with its generation date and preserved baseline. ## Is it GFM compliant? diff --git a/docs/reference/octane.md b/docs/reference/octane.md index dc730d5..432975f 100644 --- a/docs/reference/octane.md +++ b/docs/reference/octane.md @@ -4,7 +4,7 @@ title: Octane # Octane -The `@tanstack/markdown/octane` entry requires Octane 0.1.12 or newer. +The `@tanstack/markdown/octane` entry is tested with Octane 0.1.12. See [Version 1 compatibility](../project/version-one.md) for the supported adapter and runtime versions. ```ts import { diff --git a/skills/_artifacts/domain_map.yaml b/skills/_artifacts/domain_map.yaml index 0e69c60..f497e26 100644 --- a/skills/_artifacts/domain_map.yaml +++ b/skills/_artifacts/domain_map.yaml @@ -468,9 +468,9 @@ gaps: context: 'The repository documents and tests both adapters equally but contains no usage telemetry.' status: 'open' - skill: 'custom-extensions' - question: 'Which extension contracts are intended to remain stable before 1.0?' - context: 'The AST is explicitly public but pre-1.0; extension stability is not separately stated.' - status: 'open' + question: 'Which extension contracts remain stable within 1.x?' + context: 'docs/project/version-one.md documents extension ordering, callback inputs, and public AST contracts under semver.' + status: 'resolved' - skill: 'production-pipelines' question: 'Which recurring AI-generated mistakes occur in downstream applications beyond the regressions already captured in tests?' context: 'The repository has no issue history or maintainer interview evidence for downstream agent behavior.' diff --git a/skills/_artifacts/skill_spec.md b/skills/_artifacts/skill_spec.md index 9698411..d07500f 100644 --- a/skills/_artifacts/skill_spec.md +++ b/skills/_artifacts/skill_spec.md @@ -113,7 +113,7 @@ TanStack Markdown is a synchronous parser and renderer for controlled blog and d | Skill | Question | Status | | --- | --- | :---: | | react-rendering | How should React and Octane guidance be prioritized against real usage? | open | -| custom-extensions | Which extension contracts are intended to remain stable before 1.0? | open | +| custom-extensions | Which extension contracts remain stable within 1.x? | resolved | | production-pipelines | Which downstream AI mistakes recur beyond repository regressions? | open | ## Recommended Skill File Structure diff --git a/skills/render-markdown/SKILL.md b/skills/render-markdown/SKILL.md index 60ad700..5984211 100644 --- a/skills/render-markdown/SKILL.md +++ b/skills/render-markdown/SKILL.md @@ -152,8 +152,9 @@ lists, tables, frontmatter, reference definitions, and footnote definitions. - Footnotes render in first-reference order with collision-safe IDs and repeated-reference back links. - Code fence metadata is recorded in the AST; highlighting is external. -- The AST is public but pre-1.0. Regenerate persisted documents when an - upgrade changes node contracts. +- The documented AST follows the version 1 compatibility policy. Existing + valid 1.x documents remain supported by later 1.x renderers. Record the + producer version with persisted documents and rebuild caches to adopt fixes. ## Compatibility and Option Timing diff --git a/skills/render-markdown/references/ast-and-options.md b/skills/render-markdown/references/ast-and-options.md index 453690d..9e5d5f3 100644 --- a/skills/render-markdown/references/ast-and-options.md +++ b/skills/render-markdown/references/ast-and-options.md @@ -125,8 +125,10 @@ interface MarkdownDocument { - `frontmatter` contains the raw text between leading `---` delimiters. - `headings` is optional extension-derived data, not part of core parsing. - The object is plain and serializable. -- Because the package is pre-1.0, persisted ASTs should be regenerated after - an upgrade that changes node contracts. +- Existing valid 1.x ASTs remain supported by later 1.x renderers under + `docs/project/version-one.md`. Record the producing version with persisted + documents and rebuild caches to adopt parser fixes; rendered HTML bytes + are not a stable serialization contract. ## Block Nodes