Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,14 @@

A tiny, fast, deterministic Markdown parser and renderer for blogs and documentation.

- 5.2 KB gzip parser
- 7.0 KB gzip HTML renderer
- 7.0 KB gzip React adapter
- 7.0 KB gzip Octane adapter
- separately importable parser, HTML renderer, React adapter, and Octane adapter
- zero runtime dependencies
- serializable AST
- safe defaults for raw HTML and executable URLs
- optional docs extensions and external syntax highlighting
- optional AI streaming profile

Bundle sizes include the parser and exclude framework runtimes and syntax highlighters.
The [generated bundle report](./reports/sizes.md) records minified, gzip, and Brotli sizes for reproducible browser import profiles. Renderer measurements include the parser and exclude framework runtimes and syntax highlighters; complete public-entry measurements are listed separately.

```bash
pnpm add @tanstack/markdown
Expand Down Expand Up @@ -60,7 +57,7 @@ export function Article({ source }: { source: string }) @{
}
```

TanStack Markdown targets controlled technical content. It supports the Markdown used by blogs and docs, then spends its remaining complexity budget on deterministic output, renderer parity, malformed-input resilience, and small entry points. It is intentionally not a complete CommonMark, GFM, MDX, or general content-processing implementation.
TanStack Markdown targets controlled technical content. It supports the Markdown used by blogs and docs, with deterministic output, renderer parity, malformed-input limits, and separately importable entry points. It is intentionally not a complete CommonMark, GFM, MDX, or general content-processing implementation.

For syntax highlighting, use the tested [TanStack Highlight adapter](./docs/guides/syntax-highlighting.md#tanstack-highlight-adapter). It registers only the languages you choose and returns escaped token markup inside Markdown-owned code containers.

Expand Down
20 changes: 3 additions & 17 deletions docs/comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,23 +36,9 @@ Markdown libraries optimize for different jobs. TanStack Markdown is designed fo

## Measured browser size

These repository benchmarks bundle representative browser entry points from pinned dependencies, minify them with esbuild, and compress them. They are reproducible with `pnpm run size`.
The repository's [generated size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) bundles representative browser entry points from pinned dependencies, minifies them with esbuild, and records gzip and Brotli bytes. Reproduce it with `pnpm run size`.

| Entry | Gzip | Brotli |
| --- | ---: | ---: |
| `@tanstack/markdown/parser` | 5.2 KB | 4.8 KB |
| `@tanstack/markdown/html` | 7.0 KB | 6.5 KB |
| `@tanstack/markdown/react` | 7.0 KB | 6.5 KB |
| React with streaming extension | 7.2 KB | 6.6 KB |
| `@tanstack/markdown/octane` | 7.0 KB | 6.4 KB |
| Marked | 12.5 KB | 11.5 KB |
| micromark | 15.4 KB | 13.7 KB |
| markdown-wasm JS + WASM | 31.3 KB | 26.4 KB |
| unified + remark + rehype | 36.8 KB | 32.7 KB |
| commonmark.js | 48.1 KB | 39.8 KB |
| markdown-it | 52.7 KB | 44.0 KB |

The comparison does not represent equivalent feature sets. It shows the cost of each measured path for this repository’s rendering benchmark. See the generated [size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) for exact bytes and versions.
The report distinguishes selected-function imports from the complete namespace of every TanStack Markdown public entry. Framework runtimes are externalized from adapter measurements. Comparison packages have different feature sets, so the results describe those import profiles rather than equivalent capabilities or the final size of your application.

## Compatibility accounting

Expand All @@ -62,4 +48,4 @@ Use [commonmark.js](https://github.com/commonmark/commonmark.js), micromark, or

## Performance

Across the maintained fixtures, TanStack Markdown is competitive with the JavaScript renderers in the suite, but it is not the fastest result in every fixture. Pre-parsed AST rendering is its cheapest path. The defensible advantage is the combined size, output contract, and focused feature set. See [Performance](./guides/performance.md) for methodology and current results.
Across the maintained fixtures, TanStack Markdown is competitive with the JavaScript renderers in the suite, but it is not the fastest result in every fixture. Pre-parsed AST rendering is its cheapest path. Compare the measured import sizes and supported syntax with the rendering and extension APIs your application needs. See [Performance](./guides/performance.md) for methodology and current results.
4 changes: 2 additions & 2 deletions docs/core-concepts/document-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,8 @@ await cache.set(key, JSON.stringify(document))
const html = renderHtml(document)
```

Parsing once is useful for build pipelines, content indexes, multiple render targets, and high-traffic SSR paths. Rendering a pre-parsed AST is also the fastest measured path in the package.
Parsing once is useful for build pipelines, content indexes, multiple render targets, and high-traffic SSR paths. Passing a document to a renderer skips parsing the source again.

## Stability

The AST is public and typed, but the package is still pre-1.0. Pin versions when persisting documents across deployments, and regenerate cached AST data when upgrading across a release that changes node contracts.
The documented AST follows the [Version 1 compatibility policy](../project/version-one.md). Existing valid 1.x documents remain supported by later 1.x renderers. Record the producing package version with persisted documents, and rebuild caches to adopt parser fixes. Byte-for-byte rendered HTML is not a stable serialization contract, and untrusted JSON still requires validation before rendering.
2 changes: 1 addition & 1 deletion docs/core-concepts/syntax-profile.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,4 +75,4 @@ The following are not project goals:
- syntax highlighting, themes, or language grammars
- asynchronous plugin pipelines

Unsupported input must still be deterministic, escaped by default, and bounded in runtime. New syntax requires evidence from real documentation, regression fixtures, renderer parity, and an accepted bundle cost.
Unsupported input must still be deterministic and escaped by default. Core parsing limits nesting and inline scans, but does not cap total input size or arbitrary extension work; see [Resource limits](./security.md#resource-limits). New syntax requires evidence from real documentation, regression fixtures, renderer parity, and an accepted bundle cost.
4 changes: 2 additions & 2 deletions docs/guides/ai-streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ export default function Response() {

```

The streaming extension suppresses empty trailing headings, blockquotes, and list items while a response is incomplete. It does not change completed paragraphs, lists, tables, quotes, or fenced code. An unclosed code fence renders all code accumulated after its opening fence.
The streaming extension suppresses empty trailing headings, blockquotes, and list items while a response is incomplete. This transform leaves other nodes unchanged, but the next update reparses the full source. Later text, such as a reference definition, can change how earlier content parses. An unclosed code fence renders all code accumulated after its opening fence.

Keep the extension enabled after completion unless an intentionally empty final heading, quote, or list item is meaningful in your application. Without the extension, core parsing preserves those valid Markdown structures.

Expand All @@ -65,7 +65,7 @@ Unclosed emphasis, code spans, links, and other inline delimiters remain literal

The package reparses the accumulated response rather than maintaining parser state between updates. Batch very small transport tokens into normal UI updates when responses are unusually long or tokens arrive faster than the screen should repaint.

With the streaming extension enabled, React keeps completed groups of plain code lines in stable text nodes. Appending code updates the trailing group instead of replacing the entire block's text, which reduces browser layout work. Custom code components still receive string children, and highlighters keep their existing HTML rendering path.
With the streaming extension enabled, React keeps completed groups of plain code lines in stable text nodes. Appending code updates the trailing group instead of replacing the entire block's text, which avoids updating completed text groups on each append. Custom code components still receive string children, and highlighters keep their existing HTML rendering path.

## Security

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/docs-preset.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ const html = renderHtml(document, {
})
```

The preset is a separate 2.3 KB gzip entry and is not imported by the parser or renderers.
The preset is a separate entry and is not imported by the parser or renderers. See the [generated size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) for measured import profiles.

## Callouts

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/octane.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The Octane adapter renders Markdown source or a pre-parsed document directly to
pnpm add @tanstack/markdown octane
```

The adapter requires `octane@0.1.12` or newer. Octane is an optional peer dependency and is not imported by any other package entry.
The adapter is tested with `octane@0.1.12`. Octane is an optional peer dependency and is not imported by any other package entry. Its peer range permits newer versions without guaranteeing compatibility with future pre-1.0 Octane changes; see [Version 1 compatibility](../project/version-one.md).

## Component usage

Expand Down
17 changes: 3 additions & 14 deletions docs/guides/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,9 @@ TanStack Markdown’s performance strategy is architectural: keep parsing synchr

## Current measurements

The generated browser bundle report records:
The generated [size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) records minified, gzip, and Brotli bytes for browser bundles built with esbuild. It includes selected-function imports and, separately, the complete namespace of each public package entry. Those profiles can have different sizes; neither predicts every application's final bundle.

| Entry | Gzip | Brotli |
| --- | ---: | ---: |
| parser | 5.2 KB | 4.8 KB |
| HTML renderer | 7.0 KB | 6.5 KB |
| React adapter | 7.0 KB | 6.5 KB |
| Octane adapter | 7.0 KB | 6.4 KB |
| React adapter with streaming extension | 7.2 KB | 6.6 KB |
| Streaming extension | 0.3 KB | 0.3 KB |
| docs preset | 2.3 KB | 2.1 KB |
| callouts extension | 0.3 KB | 0.3 KB |

Framework runtimes are externalized from their adapters, and the highlighter measurement uses only a callback stub. Exact generated bytes live in the [size report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md).
Renderer measurements include the parser. Framework runtimes are externalized from their adapters, and the external-highlighter measurement uses only a callback stub. Check the report's generation date and reproduce it with `pnpm run size` when comparing a release.

The maintained benchmark records parse, pre-parsed rendering, parse-and-render, external-highlighter, and progressive AI-response paths against pinned comparison packages. Timings vary by runtime, CPU, fixture shape, and dependency version, so the generated [benchmark report](https://github.com/TanStack/markdown/blob/main/reports/benchmarks.md) is the only source of current CPU results.

Expand All @@ -40,7 +29,7 @@ Avoid UI adapters outside their matching framework. Import individual extensions

## Parse ahead of rendering

For content that changes less often than it is viewed, parse during ingestion or the build. Cache the serializable AST and render it for each target. The measured AST renderer is several times faster than parsing and rendering together.
For content that changes less often than it is viewed, parse during ingestion or the build. Cache the serializable AST and render it for each target. Passing an AST skips the parser on each render. The benchmark report compares that path with parsing and rendering together for each fixture.

## Keep highlighting external

Expand Down
4 changes: 2 additions & 2 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ React is a peer dependency and is not bundled into the adapter.

## Octane usage

The Octane adapter requires Octane 0.1.12 or newer:
The Octane adapter is tested with Octane 0.1.12:

```bash
pnpm add @tanstack/markdown octane
Expand All @@ -63,7 +63,7 @@ export function Article({ source }: { source: string }) @{
}
```

Octane is an optional peer dependency and is not bundled into the adapter.
Octane is an optional peer dependency and is not bundled into the adapter. Its peer range permits newer releases without guaranteeing compatibility with future pre-1.0 Octane changes. See [Version 1 compatibility](./project/version-one.md) for tested versions and the React 19 requirement when both frameworks are installed.

## Runtime and module format

Expand Down
8 changes: 4 additions & 4 deletions docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,17 +25,17 @@ General Markdown processors optimize for broad conformance, plugin ecosystems, o
- code metadata for documentation UI
- a small browser bundle

TanStack Markdown spends its complexity budget on that path. It deliberately does not implement every CommonMark edge case, MDX evaluation, automatic linkification, or a general asynchronous processing ecosystem.
TanStack Markdown supports that workflow through separately importable renderers and optional docs extensions. It deliberately does not implement every CommonMark edge case, MDX evaluation, automatic linkification, or a general asynchronous processing ecosystem.

## Core properties

### Small entry points

Current minified browser bundles are 5.2 KB gzip for the parser, 7.0 KB for HTML rendering, and 7.0 KB for either UI adapter with its framework runtime externalized. The generated [bundle report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) is the source of truth.
Import the parser, HTML renderer, or matching UI adapter separately. The generated [bundle report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) records minified browser bundles with framework runtimes externalized, including both selected-function imports and complete public entry points. Reproduce it with `pnpm run size`; your application’s imports and bundler configuration determine its final size.

### Parse once, render many

`parseMarkdown` returns plain objects and arrays. The result can be serialized, cached, inspected, transformed, and passed to either renderer.
`parseMarkdown` returns plain objects and arrays. The result can be serialized, cached, inspected, transformed, and passed to the HTML, React, or Octane renderer.

### Safe defaults

Expand All @@ -47,7 +47,7 @@ The supported contract is the [TanStack docs syntax profile](./core-concepts/syn

### AI streaming without parser state

The optional [AI streaming profile](./guides/ai-streaming.md) reparses accumulated response text and suppresses incomplete trailing block placeholders. It adds 0.2 KB gzip to the React path while leaving the core parser and renderers unchanged.
The optional [AI streaming profile](./guides/ai-streaming.md) reparses accumulated response text and suppresses incomplete trailing block placeholders. It is imported separately. Each update still parses the complete accumulated source; batch updates for long responses or very frequent transport tokens.

## Choose your starting point

Expand Down
2 changes: 1 addition & 1 deletion docs/project/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ The parser never imports a renderer. The renderers never import a highlighter. T

The extension API handles the common need to add a docs block or derive metadata without paying for a general compiler pipeline. It intentionally does not implement async plugins, arbitrary virtual files, source maps, JSX evaluation, or cross-format compilation.

That boundary is the product: use a unified or MDX stack when those capabilities are central, and use TanStack Markdown when the controlled docs renderer is enough.
Use these hooks to recognize a custom block or inline marker, collect headings, or replace HTML for a node. Choose a unified or MDX stack when you need asynchronous transforms, compiler integration, or executable JSX.

## Release gates

Expand Down
2 changes: 1 addition & 1 deletion docs/project/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ title: FAQ

## Is TanStack Markdown CommonMark compliant?

No. It continuously measures CommonMark behavior and preserves established matches, but full conformance is not the product goal. The [generated compatibility report](https://github.com/TanStack/markdown/blob/main/reports/conformance.md) records 403 of 652 CommonMark 0.31.2 examples matching after output normalization in v0.0.14.
No. It continuously measures CommonMark behavior and preserves established matches, but full conformance is not the product goal. The [generated compatibility report](https://github.com/TanStack/markdown/blob/main/reports/conformance.md) records matches against all 652 CommonMark 0.31.2 examples after output normalization, with its generation date and preserved baseline.

## Is it GFM compliant?

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/octane.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: Octane

# Octane

The `@tanstack/markdown/octane` entry requires Octane 0.1.12 or newer.
The `@tanstack/markdown/octane` entry is tested with Octane 0.1.12. See [Version 1 compatibility](../project/version-one.md) for the supported adapter and runtime versions.

```ts
import {
Expand Down
6 changes: 3 additions & 3 deletions skills/_artifacts/domain_map.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -468,9 +468,9 @@ gaps:
context: 'The repository documents and tests both adapters equally but contains no usage telemetry.'
status: 'open'
- skill: 'custom-extensions'
question: 'Which extension contracts are intended to remain stable before 1.0?'
context: 'The AST is explicitly public but pre-1.0; extension stability is not separately stated.'
status: 'open'
question: 'Which extension contracts remain stable within 1.x?'
context: 'docs/project/version-one.md documents extension ordering, callback inputs, and public AST contracts under semver.'
status: 'resolved'
- skill: 'production-pipelines'
question: 'Which recurring AI-generated mistakes occur in downstream applications beyond the regressions already captured in tests?'
context: 'The repository has no issue history or maintainer interview evidence for downstream agent behavior.'
Expand Down
2 changes: 1 addition & 1 deletion skills/_artifacts/skill_spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ TanStack Markdown is a synchronous parser and renderer for controlled blog and d
| Skill | Question | Status |
| --- | --- | :---: |
| react-rendering | How should React and Octane guidance be prioritized against real usage? | open |
| custom-extensions | Which extension contracts are intended to remain stable before 1.0? | open |
| custom-extensions | Which extension contracts remain stable within 1.x? | resolved |
| production-pipelines | Which downstream AI mistakes recur beyond repository regressions? | open |

## Recommended Skill File Structure
Expand Down
5 changes: 3 additions & 2 deletions skills/render-markdown/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,9 @@ lists, tables, frontmatter, reference definitions, and footnote definitions.
- Footnotes render in first-reference order with collision-safe IDs and
repeated-reference back links.
- Code fence metadata is recorded in the AST; highlighting is external.
- The AST is public but pre-1.0. Regenerate persisted documents when an
upgrade changes node contracts.
- The documented AST follows the version 1 compatibility policy. Existing
valid 1.x documents remain supported by later 1.x renderers. Record the
producer version with persisted documents and rebuild caches to adopt fixes.

## Compatibility and Option Timing

Expand Down
6 changes: 4 additions & 2 deletions skills/render-markdown/references/ast-and-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,10 @@ interface MarkdownDocument {
- `frontmatter` contains the raw text between leading `---` delimiters.
- `headings` is optional extension-derived data, not part of core parsing.
- The object is plain and serializable.
- Because the package is pre-1.0, persisted ASTs should be regenerated after
an upgrade that changes node contracts.
- Existing valid 1.x ASTs remain supported by later 1.x renderers under
`docs/project/version-one.md`. Record the producing version with persisted
documents and rebuild caches to adopt parser fixes; rendered HTML bytes
are not a stable serialization contract.

## Block Nodes

Expand Down
Loading