cbor
Small, safe, and easy CBOR encoding and decoding library
About cbor
This large README is shown as a plain-text preview. Read the full README at the source
<h1>CBOR Codec <a href="https://pkg.go.dev/github.com/fxamacker/cbor/v2"><img src="https://raw.githubusercontent.com/fxamacker/images/refs/heads/master/cbor/go-logo-blue.svg" alt="Go logo" style="height: 1em;" align="right"></a></h1>
[fxamacker/cbor](https://github.com/fxamacker/cbor) is a library for encoding and decoding [CBOR](https://www.rfc-editor.org/info/std94) and [CBOR Sequences](https://www.rfc-editor.org/rfc/rfc8742.html).
CBOR is a [trusted alternative](https://www.rfc-editor.org/rfc/rfc8949.html#name-comparison-of-other-binary-) to JSON, MessagePack, Protocol Buffers, etc. CBOR is an Internet Standard defined by [IETF STD 94 (RFC 8949)](https://www.rfc-editor.org/info/std94) and is designed to be relevant for decades.
`fxamacker/cbor` is used in projects by Arm Ltd., EdgeX Foundry, Flow Foundation, Fraunhofer‑AISEC, IBM, Kubernetes[*](https://github.com/search?q=org%3Akubernetes%20fxamacker%2Fcbor&type=code), Let's Encrypt, Linux Foundation, Microsoft, Oasis Protocol, Red Hat[*](https://github.com/search?q=org%3Aopenshift+fxamacker%2Fcbor&type=code), Tailscale[*](https://github.com/search?q=org%3Atailscale+fxamacker%2Fcbor&type=code), Veraison[*](https://github.com/search?q=org%3Averaison+fxamacker%2Fcbor&type=code), [etc](https://github.com/fxamacker/cbor#who-uses-fxamackercbor).
See [Quick Start](#quick-start) and [Releases](https://github.com/fxamacker/cbor/releases/). š `UnmarshalFirst` and `DiagnoseFirst` can decode CBOR Sequences. `MarshalToBuffer` and `UserBufferEncMode` accepts user-specified buffer.
## fxamacker/cbor
[](https://github.com/fxamacker/cbor/actions?query=workflow%3Aci)
[](https://github.com/fxamacker/cbor/actions?query=workflow%3A%22cover+%E2%89%A597%25%22)
[](https://github.com/fxamacker/cbor/actions/workflows/codeql-analysis.yml)
[](#fuzzing-and-code-coverage)
[](https://goreportcard.com/report/github.com/fxamacker/cbor)
[](https://github.com/fxamacker/cbor#fuzzing-and-code-coverage)
`fxamacker/cbor` is a CBOR codec in full conformance with [IETF STD 94 (RFC 8949)](https://www.rfc-editor.org/info/std94). It also supports CBOR Sequences ([RFC 8742](https://www.rfc-editor.org/rfc/rfc8742.html)) and Extended Diagnostic Notation ([Appendix G of RFC 8610](https://www.rfc-editor.org/rfc/rfc8610.html#appendix-G)).
Features include full support for CBOR tags, [Core Deterministic Encoding](https://www.rfc-editor.org/rfc/rfc8949.html#name-core-deterministic-encoding), duplicate map key detection, etc.
API is mostly same as `encoding/json`, plus interfaces that simplify concurrency and CBOR options.
Design balances trade-offs between security, speed, concurrency, encoded data size, usability, etc.
<details><summary> š Highlights</summary><p/>
__š Speed__
Encoding and decoding is fast without using Go's `unsafe` package. Slower settings are opt-in. Default limits allow very fast and memory efficient rejection of malformed CBOR data.
__š Security__
Decoder has configurable limits that defend against malicious inputs. Duplicate map key detection is supported. By contrast, `encoding/gob` is [not designed to be hardened against adversarial inputs](https://pkg.go.dev/encoding/gob#hdr-Security).
Codec passed multiple confidential security assessments in 2022. No vulnerabilities found in subset of codec in a [nonconfidential security assessment](https://github.com/veraison/go-cose/blob/v1.0.0-rc.1/reports/NCC_Microsoft-go-cose-Report_2022-05-26_v1.0.pdf) prepared by NCC Group for Microsoft Corporation.
__šļø Data Size__
Struct tag options (`toarray`, `keyasint`, `omitempty`, `omitzero`) and field tag "-" automatically reduce size of encoded structs. Encoding optionally shrinks float64ā32ā16 when values fit.
__:jigsaw: Usability__
API is mostly same as `encoding/json` plus interfaces that simplify concurrency for CBOR options. Encoding and decoding modes can be created at startup and reused by any goroutines.
Presets include Core Deterministic Encoding, Preferred Serialization, CTAP2 Canonical CBOR, etc.
__š Extensibility__
Features include CBOR [extension points](https://www.rfc-editor.org/rfc/rfc8949.html#section-7.1) (e.g. CBOR tags) and extensive settings. API has interfaces that allow users to create custom encoding and decoding without modifying this library.
<hr/>
</details>
### Secure Decoding with Configurable Settings
`fxamacker/cbor` has configurable limits, etc. that defend against malicious CBOR data.
Notably, `fxamacker/cbor` is fast at rejecting malformed CBOR data.
> [!NOTE]
> Benchmarks rejecting 10 bytes of malicious CBOR data decoding to `[]byte`:
>
> | Codec | Speed (ns/op) | Memory | Allocs |
> | :---- | ------------: | -----: | -----: |
> | fxamacker/cbor 2.7.0 | 47 Ā± 7% | 32 B/op | 2 allocs/op |
> | ugorji/go 1.2.12 | 5878187 Ā± 3% | 67111556 B/op | 13 allocs/op |
>
> Faster hardware (overclocked DDR4 or DDR5) can reduce speed difference.
>
> <details><summary> š Benchmark details </summary><p/>
>
> Latest comparison for decoding CBOR data to Go `[]byte`:
> - Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
> - go1.22.7, linux/amd64, i5-13600K (DDR4-2933, disabled e-cores)
> - go test -bench=. -benchmem -count=20
>
> #### Prior comparisons
>
> | Codec | Speed (ns/op) | Memory | Allocs |
> | :---- | ------------: | -----: | -----: |
> | fxamacker/cbor 2.5.0-beta2 | 44.33 Ā± 2% | 32 B/op | 2 allocs/op |
> | fxamacker/cbor 0.1.0 - 2.4.0 | ~44.68 Ā± 6% | 32 B/op | 2 allocs/op |
> | ugorji/go 1.2.10 | 5524792.50 Ā± 3% | 67110491 B/op | 12 allocs/op |
> | ugorji/go 1.1.0 - 1.2.6 | š„ runtime: | out of memory: | cannot allocate |
>
> - Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
> - go1.19.6, linux/amd64, i5-13600K (DDR4)
> - go test -bench=. -benchmem -count=20
>
> </details>
In contrast, some codecs can crash or use excessive resources while decoding bad data.
> [!WARNING]
> Go's `encoding/gob` is [not designed to be hardened against adversarial inputs](https://pkg.go.dev/encoding/gob#hdr-Security).
>
> <details><summary> š gob fatal error (out of memory) š„ decoding 181 bytes</summary><p/>
>
> ```Go
> // Example of encoding/gob having "fatal error: runtime: out of memory"
> // while decoding 181 bytes (all Go versions as of Dec. 8, 2024).
> package main
> import (
> "bytes"
> "encoding/gob"
> "encoding/hex"
> "fmt"
> )
>
> // Example data is from https://github.com/golang/go/issues/24446
> // (shortened to 181 bytes).
> const data = "4dffb503010102303001ff30000109010130010800010130010800010130" +
> "01ffb80001014a01ffb60001014b01ff860001013001ff860001013001ff" +
> "860001013001ff860001013001ffb80000001eff850401010e3030303030" +
> "30303030303030303001ff3000010c0104000016ffb70201010830303030" +
> "3030303001ff3000010c000030ffb6040405fcff00303030303030303030" +
> "303030303030303030303030303030303030303030303030303030303030" +
> "30"
>
> type X struct {
> J *X
> K map[string]int
> }
>
> func main() {
> raw, _ := hex.DecodeString(data)
> decoder := gob.NewDecoder(bytes.NewReader(raw))
>
> var x X
> decoder.Decode(&x) // fatal error: runtime: out of memory
> fmt.Println("Decoding finished.")
> }
> ```
>
>
> </details>
### Smaller Encodings with Struct Tag Options
Struct tags automatically reduce encoded size of structs and improve speed.
We can write less code by using struct tag options:
- `toarray`: encode without field names (decode back to original struct)
- `keyasint`: encode field names as integers (decode back to original struct)
- `omitempty`: omit empty field when encoding
- `omitzero`: omit zero-value field when encoding
As a special case, struct field tag "-" omits the field.
NOTE: When a struct uses `toarray`, the encoder will ignore `omitempty` and `omitzero` to prevent position of encoded array elements from changing. This allows decoder to match encoded elements to their Go struct field.
> [!NOTE]
> `fxamacker/cbor` can encode a 3-level nested Go struct to 1 byte!
> - `encoding/json`: 18 bytes of JSON
> - `fxamacker/cbor`: 1 byte of CBOR
>
> <details><summary> š Encoding 3-level nested Go struct with omitempty</summary><p/>
>
> https://go.dev/play/p/YxwvfPdFQG2
>
> ```Go
> // Example encoding nested struct (with omitempty tag)
> // - encoding/json: 18 byte JSON
> // - fxamacker/cbor: 1 byte CBOR
>
> package main
>
> import (
> "encoding/hex"
> "encoding/json"
> "fmt"
>
> "github.com/fxamacker/cbor/v2"
> )
>
> type GrandChild struct {
> Quux int `json:",omitempty"`
> }
>
> type Child struct {
> Baz int `json:",omitempty"`
> Qux GrandChild `json:",omitempty"`
> }
>
> type Parent struct {
> Foo Child `json:",omitempty"`
> Bar int `json:",omitempty"`
> }
>
> func cb() {
> results, _ := cbor.Marshal(Parent{})
> fmt.Println("hex(CBOR): " + hex.EncodeToString(results))
>
> text, _ := cbor.Diagnose(results) // Diagnostic Notation
> fmt.Println("DN: " + text)
> }
>
> func js() {
> results, _ := json.Marshal(Parent{})
> fmt.Println("hex(JSON): " + hex.EncodeToString(results))
>
> text := string(results) // JSON
> fmt.Println("JSON: " + text)
> }
>
> func main() {
> cb()
> fmt.Println("-------------")
> js()
> }
> ```
>
> Output (DN is Diagnostic Notation):
> ```
> hex(CBOR): a0
> DN: {}
> -------------
> hex(JSON): 7b22466f6f223a7b22517578223a7b7d7d7d
> JSON: {"Foo":{"Qux":{}}}
> ```
>
> </details>
## Quick Start
__Install__: `go get github.com/fxamacker/cbor/v2` and `import "github.com/fxamacker/cbor/v2"`.
> [!TIP]
>
> TinyGo users can try beta/experimental branch [feature/cbor-tinygo-beta](https://github.com/fxamacker/cbor/tree/feature/cbor-tinygo-beta).
>
> <details><summary> š More about TinyGo feature branch</summary>
>
> ### TinyGo
>
> :warning: The `feature/cbor-tinygo-beta` branch is not currently fuzz tested.
>
> Branch [feature/cbor-tinygo-beta](https://github.com/fxamacker/cbor/tree/feature/cbor-tinygo-beta) tracks fxamacker/cbor master branch and it is updated periodically. It can be compiled with both tinygo-org/tinygo (0.42.0-dev-801bd484 or newer) and golang/go compilers.
>
> This branch requires TinyGo 0.42.0-dev-801bd484 (or newer) because TinyGo v0.41.1 does not include a required fix ([tinygo-org/tinygo#5508](https://github.com/tinygo-org/tinygo/pull/5508)).
>
> This branch passes unit tests (with both TinyGo and Go compilers) and it is considered experimental.
>
> Changes in this feature branch only affect TinyGo compiled software. Summary of changes:
> - Default `DecOptions.MaxNestedLevels` is reduced to 16 (was 32). User can specify a higher limit when running with a larger stack.
> - Encoding error message can be different when encoding function type. For details, see [tinygo-org/tinygo#4458](https://github.com/tinygo-org/tinygo/issues/4458).
>
> NOTE: An older version of this branch [tinygo-beta/v2.7.0-tinygo0.33](https://github.com/fxamacker/cbor/tree/tinygo-beta/v2.7.0-tinygo0.33) (based on fxamacker/cbor v2.7.0) can be used with TinyGo v0.33. It disables decoding CBOR tag data to Go interface values due to [tinygo-org/tinygo#4277](https://github.com/tinygo-org/tinygo/issues/4277).
>
> </details>
### Key Points
This library can encode and decode CBOR (RFC 8949) and CBOR Sequences (RFC 8742).
- __CBOR data item__ is a single piece of CBOR data and its structure may contain 0 or more nested data items.
- __CBOR sequence__ is a concatenation of 0 or more encoded CBOR data items.
Configurable limits and options can be used to balance trade-offs.
- Encoding and decoding modes are created from options (settings).
- Modes can be created at startup and reused.
- Modes are safe for concurrent use.
### Default Mode
Package level functions only use this library's default settings.
They provide the "default mode" of encoding and decoding.
```go
// API matches encoding/json for Marshal, Unmarshal, Encode, Decode, etc.
b, err = cbor.Marshal(v) // encode v to []byte b
err = cbor.Unmarshal(b, &v) // decode []byte b to v
decoder = cbor.NewDecoder(r) // create decoder with io.Reader r
err = decoder.Decode(&v) // decode a CBOR data item to v
// v2.7.0 added MarshalToBuffer() and UserBufferEncMode interface.
err = cbor.MarshalToBuffer(v, b) // encode v to b instead of using built-in buf pool.
// v2.5.0 added new functions that return remaining bytes.
// UnmarshalFirst decodes first CBOR data item and returns remaining bytes.
rest, err = cbor.UnmarshalFirst(b, &v) // decode []byte b to v
// DiagnoseFirst translates first CBOR data item to text and returns remaining bytes.
text, rest, err = cbor.DiagnoseFirst(b) // decode []byte b to Diagnostic Notation text
// NOTE: Unmarshal() returns ExtraneousDataError if there are remaining bytes, but
// UnmarshalFirst() and DiagnoseFirst() allow trailing bytes.
```
> [!IMPORTANT]
> CBOR settings allow trade-offs between speed, security, encoding size, etc.
>
> - Different CBOR libraries may use different default settings.
> - CBOR-based formats or protocols usually require specific settings.
>
> For example, WebAuthn uses "CTAP2 Canonical CBOR" which is available as a preset.
### Presets
Presets can be used as-is or as a starting point for custom settings.
```go
// EncOptions is a struct of encoder settings.
func CoreDetEncOptions() EncOptions // RFC 8949 Core Deterministic Encoding
func PreferredUnsortedEncOptions() EncOptions // RFC 8949 Preferred Serialization
func CanonicalEncOptions() EncOptions // RFC 7049 Canonical CBOR
func CTAP2EncOptions() EncOptions // FIDO2 CTAP2 Canonical CBOR
```
Presets are used to create custom modes.
### Custom Modes
Modes are created from settings. Once created, modes have immutable settings.
š” Create the mode at startup and reuse it. It is safe for concurrent use.
```Go
// Create encoding mode.
opts := cbor.CoreDetEncOptions() // use preset options as a starting point
opts.Time = cbor.TimeUnix // change any settings if needed
em, err := opts.EncMode() // create an immutable encoding mode
// Reuse the encoding mode. It is safe for concurrent use.
// API matches encoding/json.
b, err := em.Marshal(v) // encode v to []byte b
encoder := em.NewEncoder(w) // create encoder with io.Writer w
err := encoder.Encode(v) // encode v to io.Writer w
```
Default mode and custom modes automatically apply struct tags.
### User Specified Buffer for Encoding (v2.7.0)
`UserBufferEncMode` interface extends `EncMode` interface to add `MarshalToBuffer()`. It accepts a user-specified buffer instead of using built-in buffer pool.
```Go
em, err := myEncOptions.UserBufferEncMode() // create UserBufferEncMode mode
var buf bytes.Buffer
err = em.MarshalToBuffer(v, &buf) // encode v to provided buf
```
### Struct Tags
Struct tag options (`toarray`, `keyasint`, `omitempty`, `omitzero`) reduce encoded size of structs.
As a special case, struct field tag "-" omits the field.
<details><summary> š Example encoding with struct field tag "-"</summary><p/>
https://go.dev/play/p/aWEIFxd7InX
```Go
// https://github.com/fxamacker/cbor/issues/652
package main
import (
"encoding/json"
"fmt"
"github.com/fxamacker/cbor/v2"
)
// The `cbor:"-"` tag omits the Type field when encoding to CBOR.
type Entity struct {
_ struct{} `cbor:",toarray"`
ID uint64 `json:"id"`
Type string `cbor:"-" json:"typeOf"`
Name string `json:"name"`
}
func main() {
entity := Entity{
ID: 1,
Type: "int64",
Name: "Identifier",
}
c, _ := cbor.Marshal(entity)
diag, _ := cbor.Diagnose(c)
fmt.Printf("CBOR in hex: %x\n", c)
fmt.Printf("CBOR in edn: %s\n", diag)
j, _ := json.Marshal(entity)
fmt.Printf("JSON: %s\n", string(j))
fmt.Printf("JSON encoding is %d bytes\n", len(j))
fmt.Printf("CBOR encoding is %d bytes\n", len(c))
// Output:
// CBOR in hex: 82016a4964656e746966696572
// CBOR in edn: [1, "Identifier"]
// JSON: {"id":1,"typeOf":"int64","name":"Identifier"}
// JSON encoding is 45 bytes
// CBOR encoding is 13 bytes
}
```
</details>
<details><summary> š Example encoding 3-level nested Go struct to 1 byte CBOR</summary><p/>
https://go.dev/play/p/YxwvfPdFQG2
```Go
// Example encoding nested struct (with omitempty tag)
// - encoding/json: 18 byte JSON
// - fxamacker/cbor: 1 byte CBOR
package main
import (
"encoding/hex"
"encoding/json"
"fmt"
"github.com/fxamacker/cbor/v2"
)
type GrandChild struct {
Quux int `json:",omitempty"`
}
type Child struct {
Baz int `json:",omitempty"`
Qux GrandChild `json:",omitempty"`
}
type Parent struct {
Foo Child `json:",omitempty"`
Bar int `json:",omitempty"`
}
func cb() {
results, _ := cbor.Marshal(Parent{})
fmt.Println("hex(CBOR): " + hex.EncodeToString(results))
text, _ := cbor.Diagnose(results) // Diagnostic Notation
fmt.Println("DN: " + text)
}
func js() {
results, _ := json.Marshal(Parent{})
fmt.Println("hex(JSON): " + hex.EncodeToString(results))
text := string(results) // JSON
fmt.Println("JSON: " + text)
}
func main() {
cb()
fmt.Println("-------------")
js()
}
```
Output (DN is Diagnostic Notation):
```
hex(CBOR): a0
DN: {}
-------------
hex(JSON): 7b22466f6f223a7b22517578223a7b7d7d7d
JSON: {"Foo":{"Qux":{}}}
```
<hr/>
</details>
<details><summary> š Example using struct tag options</summary><p/>
</details>
Struct tag options simplify use of CBOR-based protocols that require CBOR arrays or maps with integer keys.
### CBOR Tags
CBOR tags are specified in a `TagSet`.
Custom modes can be created with a `TagSet` to handle CBOR tags.
```go
em, err := opts.EncMode() // no CBOR tags
em, err := opts.EncModeWithTags(ts) // immutable CBOR tags
em, err := opts.EncModeWithSharedTags(ts) // mutable shared CBOR tags
```
`TagSet` and modes using it are safe for concurrent use. Equivalent API is available for `DecMode`.
<details><summary> š Example using TagSet and TagOptions</summary><p/>
```go
// Use signedCWT struct defined in "Decoding CWT" example.
// Create TagSet (safe for concurrency).
tags := cbor.NewTagSet()
// Register tag COSE_Sign1 18 with signedCWT type.
tags.Add(
cbor.TagOptions{EncTag: cbor.EncTagRequired, DecTag: cbor.DecTagRequired},
reflect.TypeOf(signedCWT{}),
18)
// Create DecMode with immutable tags.
dm, _ := cbor.DecOptions{}.DecModeWithTags(tags)
// Unmarshal to signedCWT with tag support.
var v signedCWT
if err := dm.Unmarshal(data, &v); err != nil {
return err
}
// Create EncMode with immutable tags.
em, _ := cbor.EncOptions{}.EncModeWithTags(tags)
// Marshal signedCWT with tag number.
if data, err := em.Marshal(v); err != nil {
return err
}
```
</details>
š `fxamacker/cbor` allows user apps to use almost any current or future CBOR tag number by implementing `cbor.Marshaler` and `cbor.Unmarshaler` interfaces.
Basically, `MarshalCBOR` and `UnmarshalCBOR` functions can be implemented by user apps and those functions will automatically be called by this CBOR codec's `Marshal`, `Unmarshal`, etc.
Frequently Asked Questions
What is cbor?
cbor is a Serialization library for the Go programming language. Small, safe, and easy CBOR encoding and decoding library
How many GitHub stars does cbor have?
cbor has 1,094 GitHub stars in the directory's latest synchronization.
How do I install cbor?
Install cbor with the Go module system using `go get fxamacker/cbor`. Check the repository for the current installation instructions.
What category does cbor belong to?
cbor is listed under Serialization, specifically Serialization.