goldmark

About goldmark

This large README is shown as a plain-text preview. Read the full README at the source

# goldmark

[![https://pkg.go.dev/github.com/yuin/goldmark/v2](https://pkg.go.dev/badge/github.com/yuin/goldmark/v2.svg)](https://pkg.go.dev/github.com/yuin/goldmark/v2)
[![https://github.com/yuin/goldmark/actions?query=branch%3Av2+workflow%3Atest](https://github.com/yuin/goldmark/actions/workflows/test.yaml/badge.svg?branch=v2&event=push)](https://github.com/yuin/goldmark/actions?query=branch%3Av2+workflow%3Atest)
[![https://coveralls.io/github/yuin/goldmark?branch=v2](https://coveralls.io/repos/github/yuin/goldmark/badge.svg?branch=v2)](https://coveralls.io/github/yuin/goldmark?branch=v2)

> The Golden Swiss Army Knife for markdown processing. 100% CommonMark compliant, AST w/ CST, Extensible, Fast, CJK-friendly.

goldmark is compliant with CommonMark 0.31.2.

- [goldmark playground](https://yuin.github.io/goldmark/playground/v2/) : Try goldmark online. This playground is built with WASM(5-10MB).

There is also a Rust version of goldmark: [rushdown](https://github.com/yuin/rushdown)

## v2 status
v2 is still in the early stages of release. If you are using an extension that does not support v2, please use v1.

## v2 Motivation
goldmark was originally created with a focus on my personal goals. 

- Extensible. You can easily add your own syntax to Markdown.
- Performance. Prioritize performance over semantically perfect AST.
- Focus on the purpose of converting to HTML.

Unexpectedly, goldmark has been used by many people.

goldmark has become a major Markdown parser in Go ecosystem.

In such a situation, there have been many requests regarding use cases that were not emphasized at the time of initial creation.

- Semantic analysis of Markdown documents using AST
- Use cases that use more detailed position information, such as LSP servers

In particular, as Markdown documents have come to be used as Lingua franca for AI, there is an increasing need to analyze Markdown documents semantically. For the same reason, there are also increasing use cases for generating Markdown documents rather than parsing them. The use of CLI in AI agents is increasing, also a growing need to convert to formats other than HTML.

Breaking changes to an extensible library like goldmark have a huge impact, as third-party extensions will no longer work. Therefore, I have avoided making breaking changes for a long time.

It has been more than 7 years since goldmark was created, and technical debt has been accumulating. In the meantime, the Go language specification has changed significantly, including the introduction of generics. I believe that the changes in use cases, represented by AI, are a good opportunity to fundamentally review the design of goldmark, and I have decided to make breaking changes.

## v2 and v1 differences

- v2 focuses on building a more semantic AST.
- v2 uses generics.
- v2 clearly separates the parser and renderer. This makes it easier to implement rendering to formats other than HTML.
- v2 allows you to programmatically build an AST. And you can render the constructed AST to another format.
- v2 all nodes hold the start position. In the future, third-party extensions that support v2 are also expected to hold the start position.
- The core parsing algorithm is the same as v1. Third-party extensions must support v2, but the most complex parsing part can be used almost as it is.

## Maintenance policy
This project will maintain bug fixes, including security fixes, up to one major version prior to the latest major version.

## Migrating from v1 to v2
You can use LLMs to migrate your code from v1 to v2. 

Claude Code / Copilot CLI

```bash
/plugin marketplace add yuin/goldmark@v2
/plugin install migrate-goldmark-v1-to-v2@yuin-goldmark-v2
```

Migrating your goldmark extension projects:

```
/migrate-goldmark-v1-to-v2:migrate-goldmark-extension-v1-to-v2
```

Migrating your applications using goldmark:

```
/migrate-goldmark-v1-to-v2:migrate-goldmark-app-v1-to-v2
```

These skills will create a migration plan for your project and execute the migration plan to update your code to be compatible with goldmark v2.

Of course, even you can migrate manually if they understand these contents :)
See `.agent-plugins` directory for the implementation of these skills.

## Features

- **Standards-compliant** : goldmark is fully compliant with the latest [CommonMark](https://commonmark.org/) specification.
- **Extensible** : Do you want to add a `@username` mention syntax to Markdown?
  You can easily do so in goldmark. You can add your AST nodes, parsers for block-level elements, parsers for inline-level elements,
  transformers for paragraphs, transformers for the whole AST structure, and renderers.
- **Performance** : goldmark is one of the fastest CommonMark-compliant Markdown parsers in Go.
- **Robust** : goldmark is tested with `go test --fuzz`.
- **Built-in extensions** : goldmark ships with common extensions like tables, strikethrough,
  task lists, and definition lists.
- **Semantically clean AST** :  goldmark builds a clean AST structure that is easy to analyze and transform.
- **CST support** : goldmark keeps the original source information for each node, so you can write it back to Markdown format without losing less/any information. goldmark keeps which syntax was used for headings (ATX or Setext), whether `<>` was used to enclose links like `<http://example.com>`, and other such information. goldmark can distinguish between `you & me` and `you &amp; me`. Other markdown libraries only keep the decoded value, so they cannot distinguish these.
- **CJK-friendly** : goldmark is CJK-friendly (its author is Japanese :) ). The CommonMark specification is primarily designed with Western languages in mind, and many CommonMark implementations likewise assume Western-language text, especially English.
- **Depends only on standard libraries.**

## Installation

```bash
$ go get github.com/yuin/goldmark/v2
```


## Usage

Convert Markdown documents with the CommonMark-compliant mode:

```go
import (
    "bytes"
    "github.com/yuin/goldmark/v2/parser"
    "github.com/yuin/goldmark/v2/renderer/html"
)

source := []byte("こんにちは、 **世界** 。")

var buf bytes.Buffer
p := parser.New()
r := html.New()

doc := p.Parse(source)
if err := r.Render(&buf, source, doc); err != nil {
    panic(err)
}
if "<p>こんにちは、 <strong>世界</strong> 。</p>\n" != buf.String() {
    panic("unexpected output:" + buf.String())
}
```

Build an AST and render it to HTML:

```go
import (
    "bytes"
    "github.com/yuin/goldmark/v2/ast"
    "github.com/yuin/goldmark/v2/text"
    "github.com/yuin/goldmark/v2/renderer/html"
)

doc := ast.N(ast.NewDocument(),
    ast.N(ast.NewParagraph(),
        "こんにちは、",
        ast.N(ast.NewEmphasis(),
            "世界",
        ),
        "。",
    ),
    ast.N(func() ast.Node {
        n := ast.NewParagraph()
        n.SetAttribute("class", text.NewMultiLineValue("greeting", text.IdentityDecoder))
        return n
    }(), "Hello, world."),
)

var buf bytes.Buffer
r := html.New()
if err := r.Render(&buf, nil, doc); err != nil {
    panic(err)
}
if "<p>こんにちは、<em>世界</em>。</p>\n<p class=\"greeting\">Hello, world.</p>\n" != buf.String() {
    panic("unexpected output:" + buf.String())
}
```

Query the AST: [goldmark-astquery](https://github.com/yuin/goldmark-astquery)

```go
import (
    "github.com/yuin/goldmark/v2/parser"
    "github.com/yuin/goldmark/v2/ast"
    "github.com/yuin/goldmark-astquery"
)

p := parser.New()
source := []byte(`
# Heading 1
    
> - [link1](http://www.example.com)
>
> [link2](https://www2.example.com)
>
> hoge
`)
doc := p.Parse(source)

result := astquery.Match(doc, source,
    astquery.ForEachDescendant(
        astquery.Bind("link-parents",
            astquery.HasChild(
                astquery.NodeKind(ast.KindLink,
                    astquery.MatchesProperty("Destination", astquery.Pattern(`https://.*`)),
                ),
            ),
        ),
    ),
)
link := result["link-parents"][0].FirstChild().(*ast.Link)
if link.Destination.Value(source) != "https://www2.example.com" {
    panic("unexpected result")
}
```

## Custom parser and renderer

```go
import (
    "bytes"
    "github.com/yuin/goldmark/v2/extension"
    "github.com/yuin/goldmark/v2/parser"
    "github.com/yuin/goldmark/v2/renderer/html"
)

source := []byte("こんにちは、 ~~世界~~ 。")

p := parser.New(parser.WithAttribute(), parser.WithExtensions(extension.StrikethroughParser))
r := html.New(html.WithXHTML(), html.WithUnsafe(), html.WithExtensions(extension.StrikethroughHTMLRenderer))

var buf bytes.Buffer
doc := p.Parse(source)
if err := r.Render(&buf, source, doc); err != nil {
    panic(err)
}
if "<p>こんにちは、 <del>世界</del> 。</p>\n" != buf.String() {
    panic("unexpected output:" + buf.String())
}

```

### Parser options

| Functional option | Type | Description |
| ----------------- | ---- | ----------- |
| `parser.WithBlockParsers` | `[]util.PrioritizedValue[parser.BlockParser]` | Parsers for parsing block level elements. |
| `parser.WithInlineParsers` | `[]util.PrioritizedValue[parser.InlineParser]` | Parsers for parsing inline level elements. |
| `parser.WithParagraphTransformers` | `[]util.PrioritizedValue[parser.ParagraphTransformer]` | Transformers for transforming paragraph nodes. |
| `parser.WithASTTransformers` | `[]util.PrioritizedValue[parser.ASTTransformer]` | Transformers for transforming an AST. |
| `parser.WithAutoHeadingID` | `-` | Enables auto heading ids. |
| `parser.WithAttribute` | `-` | Enables custom attributes. Currently only headings supports attributes. |
| `parser.WithIDGenerator` | `parser.IDGenerator` |  Generator for heading ids. |
| `parser.WithDefaultParsers` | `bool` | Enables default parsers. Default is true. |
| `parser.WithEscapedSpace` | `-` | Enables escaped space. This is useful for CJK users. |
| `parser.WithParseDelimiterFunc` | `parser.ParseDelimiterFunc` | Function that determines whether a delimiter can be parsed. |
| `parser.WithExtensions` | `[]parser.Extension` | Enables parser extensions. |

### Parse options

| Functional option | Type | Description |
| ----------------- | ---- | ----------- |
| `parser.WithContext` | `parser.Context` | Context for parsing. |
| `parser.WithPrettyPrint` | `[]ast.PrettyPrintOption` | Prints the parsed AST tree to stdout (or a custom `io.Writer` via `ast.PrettyPrintOption`) for debugging. |


### HTML Renderer options

| Functional option | Type | Description |
| ----------------- | ---- | ----------- |
| `html.WithLineBreakStrategy` | `html.LineBreakStrategy` | Soft line breaks are rendered as a newline. Some asian users will see it as an unnecessary space. With this option, you can change the behavior. |
| `html.WithHardWraps` | `-` | Render newlines as `<br>`.|
| `html.WithIsInTightBlockFunc` | `html.IsInTightBlockFunc` | Function that determines whether a node is in a tight block. |
| `html.WithNodeRenderer` | `ast.NodeKind`, `html.NodeRenderer` | Add a node renderer for a specific node kind. |
| `html.WithNodeRenderers` | `map[ast.NodeKind]html.NodeRenderer` | Add node renderers for specific node kinds. |
| `html.WithNodeRendererDecorator` | `ast.NodeKind`, `html.NodeRendererDecorator` | Add a decorator for a node renderer. |
| `html.WithNodeRendererDecorators` | `map[ast.NodeKind]html.NodeRendererDecorator` | Add decorators for node renderers. |
| `html.WithXHTML` | `-` | Render as XHTML. |
| `html.WithUnsafe` | `-` | By default, goldmark does not render raw HTML or potentially dangerous links. With this option, goldmark renders such content as written. |
| `html.WithExtensions` | `[]html.Extension` | Enables parser extensions. |

#### Defined line break strategies <a name="defined-line-break-strategies"></a>

| Style | Description |
| ----- | ----------- |
| `SimpleEastAsianLineBreakStrategy` | Soft line breaks are ignored if both sides of the break are east asian wide character. This behavior is the same as [`east_asian_line_breaks`](https://pandoc.org/MANUAL.html#extension-east_asian_line_breaks) in Pandoc. |
| `CSSText3LineBreakStrategy` | This option implements CSS text level3 [Segment Break Transformation Rules](https://drafts.csswg.org/css-text-3/#line-break-transform) with [some enhancements](https://github.com/w3c/csswg-drafts/issues/5086). |

**Example of `SimpleEastAsianLineBreakStrategy`**

Input Markdown:

```md
私はプログラマーです。
東京の会社に勤めています。
GoでWebアプリケーションを開発しています。
```

Output:

```html
<p>私はプログラマーです。東京の会社に勤めています。\nGoでWebアプリケーションを開発しています。</p>
```

**Example of `CSSText3LineBreakStrategy`**

Input Markdown:

```md
私はプログラマーです。
東京の会社に勤めています。
GoでWebアプリケーションを開発しています。
```

Output:

```html
<p>私はプログラマーです。東京の会社に勤めています。GoでWebアプリケーションを開発しています。</p>
```

### Render options

| Functional option | Type | Description |
| ----------------- | ---- | ----------- |
| `renderer.WithContext` | `renderer.Context` | Context for rendering. Passed to `Renderer[W].Render` as a `RenderOption`. |

### Notes for CJK users
CommonMark is primarily designed with Western languages in mind. CommonMark spec doc only contains Western languages.
CJK languages, which do not separate words with spaces, have reported various problems.

These issues have been discussed for many years, but even now, more than 10 years after the announcement of CommonMark in 2014, the CommonMark spec can not fully take CJK into account.

goldmark provides its own feature extensions to address these issues.

#### Emphasis
CJK users are most dissatisfied with the emphasis specification in CommonMark.

```
これは**「重要」**です。
```

This kind of emphasis is very natural in CJK, and AI also uses it frequently.
However, the CommonMark spec cannot recognize this as emphasis because there is no space before and after the emphasis.
In contrast, the original Markdown recognizes this sentence as emphasis.

In goldmark, you can address this issue in two ways.

**1: `parser.WithParseDelimiterFunc`**

You can customize the emphasis parsing algorithm using the `parser.WithParseDelimiterFunc` option.

- `parser.ParseDelimiter` : Algorithm that is compliant with the CommonMark spec(default)
- `parser.ParseDelimiterSimple`: Algorithm that is close to the original Markdown

CommonMark specifies a complex algorithm in the algorithm for "whether emphasis can be started."

`parser.ParseDelimiterSimple` performs the following simple algorithms.

- Delimiters cannot start emphasis if there is a space after the delimiter
- Delimiters cannot end emphasis if there is a space before the delimiter

This algorithm allows for natural emphasis even in CJK.

**2: `parser.WithEscapedSpace`**

`parser.WithEscapedSpace` option allows for natural emphasis in CJK while keeping the emphasis parsing algorithm compliant with CommonMark.

```
これは\ **「重要」**\ です。
```

You can achieve natural emphasis in CJK by adding "invisible spaces" before and after the emphasis.
This technique is also adopted in reStructuredText, the standard documentation format for Python, and has a proven track record.

#### Line breaks
In many cases, soft line breaks in CommonMark are treated as unnecessary spaces in CJK.

In goldmark, you can use [LineBreakStrategy](#defined-line-break-strategies) to remove unnecessary spaces in CJK.

#### Issues in major extensions
Unfortunately, many of the well-known feature extensions, represented by Github Flavored Markdown(GFM), are also designed with Western languages in mind.

For example, the GFM alert notation is written as follows.

```
> [!NOTE]
> This is a note.
```

We can not change the title to `情報` or `参考` even if we write the document in Japanese.

In goldmark's built-in extensions, the [Linkify](#linkify-extension) specification has issues regarding CJK.

```
こちらhttp://example.comが参考になります
```

GFM interprets this as `こちら<a href="http://example.comが参考になります">http://example.comが参考になります</a>`.
You can achieve natural linkification in CJK by using the `parser.WithEscapedSpace` option.

```
こちらhttp://example.com\ が参考になります。
```


### Built-in extensions

Each extension is a pair of a parser extension and an HTML renderer extension. 

- `Table(Parser|HTMLRenderer)`
    - [GitHub Flavored Markdown: Tables](https://github.github.com/gfm/#tables-extension-)
- `Strikethrough(Parser|HTMLRenderer)`
    - [GitHub Flavored Markdown: Strikethrough](https://github.github.com/gfm/#strikethrough-extension-)
- `LinkifyParser`
    - [GitHub Flavored Markdown: Autolinks](https://github.github.com/gfm/#autolinks-extension-)
    - This extension only affects parsing; autolinks render through the same `ast.AutoLink` renderer as CommonMark autolinks, so it has no HTML renderer half.
- `TaskList(Parser|HTMLRenderer)`
    - [GitHub Flavored Markdown: Task list items](https://github.github.com/gfm/#task-list-items-extension-)
- `GFM(Parser|HTMLRenderer)`
    - This extension enables Table, Strikethrough, Linkify and TaskList.
    - This extension does not filter tags defined in [6.11: Disallowed Raw HTML (extension)](https://github.github.com/gfm/#disallowed-raw-html-extension-).
      If you need to filter HTML tags, see [Security](#security).
    - If you need to parse github emojis, you can use [goldmark-emoji](https://github.com/yuin/goldmark-emoji) extension.
    - [goldmark-github-slugger](https://github.com/yuin/goldmark-github-slugger) generates the same heading ids as GitHub.
    - [goldmark-alert](https://github.com/yuin/goldmark-alert) adds support for GitHub-style alert blocks.
- `DefinitionList(Parser|HTMLRenderer)`
    - [PHP Markdown Extra: Definition lists](https://michelf.ca/projects/php-markdown/extra/#def-list)
- `Footnote(Parser|HTMLRenderer)`
    - [PHP Markdown Extra: Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes)
- `TypographerParser`
    - This extension substitutes punctuations with typographic entities like [smartypants](https://daringfireball.net/projects/smartypants/).
    - This extension only affects parsing (it emits already-substituted text), so it has no HTML renderer half.

### Attributes
The `parser.WithAttribute` option allows you to define attributes on some elements.

Currently only headings support attributes.

**Attributes are being discussed in the
[CommonMark forum](https://talk.commonmark.org/t/consistent-attribute-syntax/272).
This syntax may possibly change in the future.**


#### Headings

```
## heading ## {#id .className attrName=attrValue class="class1 class2"}

## heading {#id .className attrName=attrValue class="class1 class2"}
```

```
heading {#id .className attrName=attrValue}
============
```

Attributes specification is almost the same as HTML attributes.

- `"` or `'` quoted strings can contain any character except the quote character itself. HTML entity references are also allowed.
- Unquoted values cannot contain any whitespace characters or `}`.

In addition to the HTML attribute specification, there is a special syntax for IDs and class names.

- `#`-prefixed strings are interpreted as ID attributes.
- `.`-prefixed strings are interpreted as class names.

Like other CommonMark attribute values (e.g., FencedCodeBlock language, link title), attribute values can contain entity references and symbol escapes with `\`.


### Table extension
The Table extension implements [Table(extension)](https://github.github.com/gfm/#tables-extension-), as
defined in [GitHub Flavored Markdown Spec](https://github.github.com/gfm/).

Specs are defined for XHTML, so specs use some deprecated attributes for HTML5.

You can override alignment rendering method via options.

| Functional option | Type | Description |
| ----------------- | ---- | ----------- |
| `extension.WithTableCellAlignMethod` | `extension.TableCellAlignMethod` | Option indicates how are table cells aligned. |

### Typographer extension

The Typographer extension translates plain ASCII punctuation characters into typographic-punctuation HTML entities.

Default substitutions are:

| Punctuation | Default entity |
| ------------ | ---------- |
| `'`           | `&lsquo;`, `&rsquo;` |
| `"`           | `&ldquo;`, `&rdquo;` |

Frequently Asked Questions