XQuery lets you extract data from HTML/XML documents using XPath expression.

xquery
====
[![Build Status](https://travis-ci.org/antchfx/xquery.svg?branch=master)](https://travis-ci.org/antchfx/xquery)
[![Coverage Status](https://coveralls.io/repos/github/antchfx/xquery/badge.svg?branch=master)](https://coveralls.io/github/antchfx/xquery?branch=master)
[![GoDoc](https://godoc.org/github.com/antchfx/xquery?status.svg)](https://godoc.org/github.com/antchfx/xquery)
[![Go Report Card](https://goreportcard.com/badge/github.com/antchfx/xquery)](https://goreportcard.com/report/github.com/antchfx/xquery)

> NOTE: This package is deprecated. Recommends use [htmlquery](https://github.com/antchfx/htmlquery) and [xmlquery](https://github.com/antchfx/xmlquery) package, get latest version to fixed some issues.

Overview
===

Golang package, lets you extract data from HTML/XML documents using XPath expression.

List of supported XPath functions you can found here [XPath Package](https://github.com/antchfx/xpath).

Installation
====

> go get github.com/antchfx/xquery


HTML Query [![GoDoc](https://godoc.org/github.com/antchfx/xquery/html?status.svg)](https://godoc.org/github.com/antchfx/xquery/html)
===

Extract data from HTML document.

```go
package main

import (
	"github.com/antchfx/xpath"
	"github.com/antchfx/xquery/html"
)

func main() {
	// Load HTML file.
	f, err := os.Open(`./examples/test.html`)
	if err != nil {
		panic(err)
	}
	// Parse HTML document.
	doc, err := htmlquery.Parse(f)
	if err != nil{
		panic(err)
	}

	// Option 1: using xpath's expr to matches nodes.
	expr := xpath.MustCompile("count(//div[@class='article'])")
	fmt.Printf("%f \n", expr.Evaluate(htmlquery.CreateXPathNavigator(doc)).(float64))

	expr = xpath.MustCompile("//a/@href")
	iter := expr.Evaluate(htmlquery.CreateXPathNavigator(doc)).(*xpath.NodeIterator)
	for iter.MoveNext() {
		fmt.Printf("%s \n", iter.Current().Value()) // output href
	}

	// Option 2: using build-in functions Find() to matches nodes.
	for _, n := range htmlquery.Find(doc, "//a/@href") {
		fmt.Printf("%s \n", htmlquery.SelectAttr(n, "href")) // output href
	}
}
```

XML Query [![GoDoc](https://godoc.org/github.com/antchfx/xquery/xml?status.svg)](https://godoc.org/github.com/antchfx/xquery/xml)
===
Extract data from XML document.

```go
package main

import (
	"github.com/antchfx/xpath"
	"github.com/antchfx/xquery/xml"
)

func main() {
	// Load XML document from file.
	f, err := os.Open(`./examples/test.xml`)
	if err != nil {
		panic(err)
	}
	// Parse XML document.
	doc, err := xmlquery.Parse(f)
	if err != nil{
		panic(err)
	}

	// Option 1: using xpath's expr to matches nodes.

	// sum all book's price via Evaluate()
	expr, err := xpath.Compile("sum(//book/price)")
	if err != nil {
		panic(err)
	}
	fmt.Printf("total price: %f\n", expr.Evaluate(xmlquery.CreateXPathNavigator(doc)).(float64))

	for _, n := range xmlquery.Find(doc, "//book") {
		fmt.Printf("%s : %s \n", n.SelectAttr("id"), xmlquery.FindOne(n, "title").InnerText())
	}
	
	// Option 2: using build-in functions FindOne() to matches node.
	n := xmlquery.FindOne(doc, "//book[@id='bk104']")
	fmt.Printf("%s \n", n.OutputXML(true))
}
```


xquery

Simple command line XML comparer that generates diffs of folders, files and tags

[![LICENSE](https://img.shields.io/badge/license-MIT-orange.svg)](LICENSE)
[![GoDoc](https://godoc.org/github.com/XML-Comp/XML-Comp?status.png)](https://godoc.org/github.com/XML-Comp/XML-Comp)
[![Go Report Card](https://goreportcard.com/badge/github.com/XML-Comp/XML-Comp)](https://goreportcard.com/report/github.com/XML-Comp/XML-Comp)
[![codebeat badge](https://codebeat.co/badges/1600adbb-27a3-4c3b-803e-818e1834b51a)](https://codebeat.co/projects/github-com-xml-comp-xml-comp)


[![Top Level Coverage](https://coveralls.io/repos/github/XML-Comp/XML-Comp/badge.svg?branch=master)](https://coveralls.io/github/XML-Comp/XML-Comp?branch=master)
[![Travis Build Status](https://api.travis-ci.org/XML-Comp/XML-Comp.svg?branch=master)](https://travis-ci.org/XML-Comp/XML-Comp)
[![DOI](https://zenodo.org/badge/71943139.svg)](https://zenodo.org/badge/latestdoi/71943139)


![Awesome](https://raw.githubusercontent.com/istio/fortio/master/docs/mentioned-badge.svg?sanitize=true)

[![Donate](https://www.paypalobjects.com/pt_BR/BR/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=arxdsilva%40gmail%2ecom&lc=BR&item_name=xml%2dcomp&currency_code=USD&bn=PP%2dDonationsBF%3abtn_donateCC_LG%2egif%3aNonHosted)


## Menu
1. [What is XML-Comp?](https://github.com/xml-comp/xml-comp#what-is-XML-Comp)
2. [Features](https://github.com/xml-comp/xml-comp#features)
3. [Installing](https://github.com/xml-comp/xml-comp#installing)
4. [Running](https://github.com/xml-comp/xml-comp#running)
5. [How this works?](https://github.com/xml-comp/xml-comp#how-this-works)
6. [Comparing any kind of document](https://github.com/xml-comp/xml-comp#comparing-any-kind-of-document)
7. [Contributing](https://github.com/xml-comp/xml-comp#contributing)
8. [To Do](https://github.com/xml-comp/xml-comp#to-do---check-our-issues--milestones)
9. [Using only the comparer package](https://github.com/xml-comp/xml-comp#using-only-the-comparer-package)

## What is XML-Comp
This is a command line tool and a package that together they provide the capability of comparing two directories and appending to files the differences between the directories, also creates possible files or folders that are missing. It was made to help [RimWorld](http://rimworldgame.com/)'s [community translators](https://github.com/ludeon)(1) to know what was modified on the last XML updates and to let them keep in track of what they need to add/remove from what has been done.

(1) and maybe other indie games that uses XML

### Features
- [x] [Compare two given directories](https://github.com/XML-Comp/XML-Comp/issues/7)
- [x] [Append missing tags to the respective file](https://github.com/XML-Comp/XML-Comp/issues/8)
- [x] [Creates missing files on the compared directory](https://github.com/XML-Comp/XML-Comp/issues/9)
- [x] [Creates missing folders on the compared directory](https://github.com/XML-Comp/XML-Comp/issues/32)
- [x] [Compare one directory with multiple directories](https://github.com/XML-Comp/XML-Comp/issues/48)
- [x] [Compare automatically the game's english version with the given translation](https://github.com/XML-Comp/XML-Comp/issues?q=is%3Aissue+is%3Aclosed)
- [ ] Translate tag content
- [ ] Expose untracked files
- [ ] Expose untracked directories
- [ ] Expose untracked tags

### Installing
```
$ go get github.com/XML-Comp/XML-Comp
```

### Running
```shell
$ XML-Comp -translation /path/to/language/translation
```

### How this works?
You need the path that is called "translation", which are described bellow:
- **`"translation"`**: Full path directory of your RimWorld ~Language~ folder cloned from [GitHub](https://github.com/ludeon).

My "translation" path: **`/Users/arthur/Github/RimWorld-PortugueseBrazilian`**

With this path in hand, running `xml-comp -translation your/path/to/translation` will let you know in every file of your project what is missing by adding lines to it with what is needed to translate! That simple!

#### [RIMWORLD not installed in standard path]
If by any reason you did not install the game on Steam's standard path or want to use a different one, It's possible to use the `original` flag that exposes your customized path to the game as shows: 

- **`"original"`**: Full path directory of your RimWorld English folder (optional).
My "original" path (optional): **`/Users/arthur/Library/Application Support/Steam/steamapps/common/RimWorld/RimWorldMac.app/Mods/Core/Languages/English`**

In order to use this flag you'll need to parse it with the `translation` flag as in: `xml-comp -original path/... -translation path/...`

#### Comparing any kind of document
To compare any kind of files, all you need is to use the flag `-doc <type name>`, eg `-doc html`. This will use the paths that you gave only to compare the specified type of document. Another example:

```shell
$ XML-Comp -doc html -original path/to/It -translation path/to/It
```

OBS: This is not required, by default It's comparing all `.xml` files that are encountered.

#### [Contributing](https://github.com/XML-Comp/XML-Comp/blob/master/Contributing.md)

#### [Join our Gitter](https://gitter.im/XML-Comparer/Lobby)
#### To Do - Check our [Issues](https://github.com/XML-Comp/XML-Comp/issues) & [Milestones](https://github.com/XML-Comp/XML-Comp/milestones)

## Using only the comparer package
```go
// Import the package
import "github.com/XML-Comp/XML-Comp/comparer"
// Set document type variable to the desired document
// without the "." | eg: "xml" or "html"
comparer.DocType = "html"
// Start the main function with the full paths to compare
// the firstPath is always what will be used as model
func main() {
    err := comparer.Compare(firstPath, comparingPath)
    if err != nil {
        log.Fatal(err)
    }
}
```


XML-Comp

[![Build Status](https://travis-ci.org/sbabiv/xml2map.svg?branch=master)](https://travis-ci.org/sbabiv/xml2map)
[![Coverage Status](https://coveralls.io/repos/github/sbabiv/xml2map/badge.svg?branch=master)](https://coveralls.io/github/sbabiv/xml2map?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/sbabiv/xml2map)](https://goreportcard.com/report/github.com/sbabiv/xml2map)
[![GoDoc](https://godoc.org/github.com/sbabiv/xml2map?status.svg)](https://godoc.org/github.com/sbabiv/xml2map)
[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/avelino/awesome-go#xml)

# xml2map
XML to MAP converter written Golang

Sometimes there is a need for the representation of previously unknown structures. Such a universal representation is usually a string in the form of JSON, XML, or the structure of data map. similar to the map[string]interface{} or map[interface{}]interface{}.

This is a converter from the old XML format to map[string]interface{} Golang

For example, the map[string]interface{} can be used as a universal type in template generation. Golang "text/template" and etc.

## Getting started

#### 1. install 

``` sh
go get -u github.com/sbabiv/xml2map
```

Or, using dep:

``` sh
dep ensure -add github.com/sbabiv/xml2map
```


#### 2. use it

```go

func main() {
	data := `<container uid="FA6666D9-EC9F-4DA3-9C3D-4B2460A4E1F6" lifetime="2019-10-10T18:00:11">
				<cats>
					<cat>
						<id>CDA035B6-D453-4A17-B090-84295AE2DEC5</id>
						<name>moritz</name>
						<age>7</age> 	
						<items>
							<n>1293</n>
							<n>1255</n>
							<n>1257</n>
						</items>
					</cat>
					<cat>
						<id>1634C644-975F-4302-8336-1EF1366EC6A4</id>
						<name>oliver</name>
						<age>12</age>
					</cat>
					<dog color="gray">hello</dog>
				</cats>
				<color>white</color>
				<city>NY</city>
			</container>`

	decoder := xml2map.NewDecoder(strings.NewReader(data))
	result, err := decoder.Decode()

	if err != nil {
		fmt.Printf("%v\n", err)
	} else {
		fmt.Printf("%v\n", result)
	}
	
	v := result["container"].
		(map[string]interface{})["cats"].
			(map[string]interface{})["cat"].
				([]map[string]interface{})[0]["items"].
					(map[string]interface{})["n"].([]string)[1]
					
	fmt.Printf("n[1]: %v\n", v)
}

```
if you want to use your custom prefixes use the 

```
NewDecoderWithPrefix(reader io.Reader, attrPrefix, textPrefix string) *Decoder
```
[Go Playground](https://play.golang.org/p/_n35DRTxTYF)

## Output

```go
map[container:map[@uid:FA6666D9-EC9F-4DA3-9C3D-4B2460A4E1F6 @lifetime:2019-10-10T18:00:11 cats:map[cat:[map[id:CDA035B6-D453-4A17-B090-84295AE2DEC5 name:moritz age:7 items:map[n:[1293 1255 1257]]] map[id:1634C644-975F-4302-8336-1EF1366EC6A4 name:oliver age:12]] dog:map[@color:gray #text:hello]] color:white city:NY]]

result: 1255
```

## Benchmark


```go
$ go test -bench=. -benchmem
goos: darwin
goarch: amd64
pkg: github.com/sbabiv/xml2map
BenchmarkDecoder-8         50000             29773 ns/op           15032 B/op        261 allocs/op
PASS
ok      github.com/sbabiv/xml2map       1.805s
```

## Licence
[MIT](https://opensource.org/licenses/MIT)

## Author 
Babiv Sergey


xml2map

# XPath

[![GoDoc](https://godoc.org/github.com/antchfx/xpath?status.svg)](https://godoc.org/github.com/antchfx/xpath)
[![Coverage Status](https://coveralls.io/repos/github/antchfx/xpath/badge.svg?branch=master)](https://coveralls.io/github/antchfx/xpath?branch=master)
[![Build Status](https://github.com/antchfx/xpath/actions/workflows/testing.yml/badge.svg)](https://github.com/antchfx/xpath/actions/workflows/testing.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/antchfx/xpath)](https://goreportcard.com/report/github.com/antchfx/xpath)

XPath is Go package provides selecting nodes from XML, HTML or other documents using XPath expression.

# Implementation

- [htmlquery](https://github.com/antchfx/htmlquery) - an XPath query package for HTML document

- [xmlquery](https://github.com/antchfx/xmlquery) - an XPath query package for XML document.

- [jsonquery](https://github.com/antchfx/jsonquery) - an XPath query package for JSON document

# Supported Features

#### The basic XPath patterns.

> The basic XPath patterns cover 90% of the cases that most stylesheets will need.

- `node` : Selects all child elements with nodeName of node.

- `*` : Selects all child elements.

- `@attr` : Selects the attribute attr.

- `@*` : Selects all attributes.

- `node()` : Matches an org.w3c.dom.Node.

- `text()` : Matches a org.w3c.dom.Text node.

- `comment()` : Matches a comment.

- `.` : Selects the current node.

- `..` : Selects the parent of current node.

- `/` : Selects the document node.

- `a[expr]` : Select only those nodes matching a which also satisfy the expression expr.

- `a[n]` : Selects the nth matching node matching a When a filter's expression is a number, XPath selects based on position.

- `a/b` : For each node matching a, add the nodes matching b to the result.

- `a//b` : For each node matching a, add the descendant nodes matching b to the result.

- `//b` : Returns elements in the entire document matching b.

- `a|b` : All nodes matching a or b, union operation(not boolean or).

- `(a, b, c)` : Evaluates each of its operands and concatenates the resulting sequences, in order, into a single result sequence

- `(a/b)` : Selects all matches nodes as grouping set.

#### Node Axes

- `child::*` : The child axis selects children of the current node.

  - `child::node()`: Selects all the children of the context node.
  - `child::text()`: Selects all text node children of the context node.

- `descendant::*` : The descendant axis selects descendants of the current node. It is equivalent to '//'.

- `descendant-or-self::*` : Selects descendants including the current node.

- `attribute::*` : Selects attributes of the current element. It is equivalent to @\*

- `following-sibling::*` : Selects nodes after the current node.

- `preceding-sibling::*` : Selects nodes before the current node.

- `following::*` : Selects the first matching node following in document order, excluding descendants.

- `preceding::*` : Selects the first matching node preceding in document order, excluding ancestors.

- `parent::*` : Selects the parent if it matches. The '..' pattern from the core is equivalent to 'parent::node()'.

- `ancestor::*` : Selects matching ancestors.

- `ancestor-or-self::*` : Selects ancestors including the current node.

- `self::*` : Selects the current node. '.' is equivalent to 'self::node()'.

#### Expressions

The gxpath supported three types: number, boolean, string.

- `path` : Selects nodes based on the path.

- `a = b` : Standard comparisons.

  - `a = b` : True if a equals b.
  - `a != b` : True if a is not equal to b.
  - `a < b` : True if a is less than b.
  - `a <= b` : True if a is less than or equal to b.
  - `a > b` : True if a is greater than b.
  - `a >= b` : True if a is greater than or equal to b.

- `a + b` : Arithmetic expressions.

  - `- a` Unary minus
  - `a + b` : Addition
  - `a - b` : Subtraction
  - `a * b` : Multiplication
  - `a div b` : Division
  - `a mod b` : Modulus (division remainder)

- `a or b` : Boolean `or` operation.

- `a and b` : Boolean `and` operation.

- `(expr)` : Parenthesized expressions.

- `fun(arg1, ..., argn)` : Function calls:

| Function                | Supported |
| ----------------------- | --------- |
| `boolean()`             | ✓         |
| `ceiling()`             | ✓         |
| `choose()`              | ✗         |
| `concat()`              | ✓         |
| `contains()`            | ✓         |
| `count()`               | ✓         |
| `current()`             | ✗         |
| `document()`            | ✗         |
| `element-available()`   | ✗         |
| `ends-with()`           | ✓         |
| `false()`               | ✓         |
| `floor()`               | ✓         |
| `format-number()`       | ✗         |
| `function-available()`  | ✗         |
| `generate-id()`         | ✗         |
| `id()`                  | ✗         |
| `key()`                 | ✗         |
| `lang()`                | ✗         |
| `last()`                | ✓         |
| `local-name()`          | ✓         |
| `lower-case()`[^1]      | ✓         |
| `matches()`             | ✓         |
| `name()`                | ✓         |
| `namespace-uri()`       | ✓         |
| `normalize-space()`     | ✓         |
| `not()`                 | ✓         |
| `number()`              | ✓         |
| `position()`            | ✓         |
| `replace()`             | ✓         |
| `reverse()`             | ✓         |
| `round()`               | ✓         |
| `starts-with()`         | ✓         |
| `string()`              | ✓         |
| `string-join()`[^1]     | ✓         |
| `string-length()`       | ✓         |
| `substring()`           | ✓         |
| `substring-after()`     | ✓         |
| `substring-before()`    | ✓         |
| `sum()`                 | ✓         |
| `system-property()`     | ✗         |
| `translate()`           | ✓         |
| `true()`                | ✓         |
| `unparsed-entity-url()` | ✗         |

[^1]: XPath-2.0 expression


xpath

# zek

Zek is a **prototype** for creating a Go
[struct](https://golang.org/ref/spec#Struct_types) from an XML document. The
resulting struct works best for *reading* XML (see also
[#14](https://github.com/miku/zek/issues/14)), to create XML, you might want to
use [something else](https://github.com/avelino/awesome-go#xml).

It was developed at [Leipzig University Library](https://github.com/ubleipzig) to shorten the
time to go from raw XML to a struct that allows to access XML data in Go
programs.

> Skip the fluff, just the code.

Given some [XML](https://raw.githubusercontent.com/miku/zek/master/fixtures/e.xml), run:

```go
$ curl -s https://raw.githubusercontent.com/miku/zek/master/fixtures/e.xml | zek -e
// Rss was generated 2018-08-30 20:24:14 by tir on sol.
type Rss struct {
    XMLName xml.Name `xml:"rss"`
    Text    string   `xml:",chardata"`
    Rdf     string   `xml:"rdf,attr"`
    Dc      string   `xml:"dc,attr"`
    Geoscan string   `xml:"geoscan,attr"`
    Media   string   `xml:"media,attr"`
    Gml     string   `xml:"gml,attr"`
    Taxo    string   `xml:"taxo,attr"`
    Georss  string   `xml:"georss,attr"`
    Content string   `xml:"content,attr"`
    Geo     string   `xml:"geo,attr"`
    Version string   `xml:"version,attr"`
    Channel struct {
        Text          string `xml:",chardata"`
        Title         string `xml:"title"`         // ESS New Releases (Display...
        Link          string `xml:"link"`          // http://tinyurl.com/ESSNew...
        Description   string `xml:"description"`   // New releases from the Ear...
        LastBuildDate string `xml:"lastBuildDate"` // Mon, 27 Nov 2017 00:06:35...
        Item          []struct {
            Text        string `xml:",chardata"`
            Title       string `xml:"title"`       // Surficial geology, Aberde...
            Link        string `xml:"link"`        // https://geoscan.nrcan.gc....
            Description string `xml:"description"` // Geological Survey of Cana...
            Guid        struct {
                Text        string `xml:",chardata"` // 304279, 306212, 306175, 3...
                IsPermaLink string `xml:"isPermaLink,attr"`
            } `xml:"guid"`
            PubDate       string   `xml:"pubDate"`      // Fri, 24 Nov 2017 00:00:00...
            Polygon       []string `xml:"polygon"`      // 64.0000 -98.0000 64.0000 ...
            Download      string   `xml:"download"`     // https://geoscan.nrcan.gc....
            License       string   `xml:"license"`      // http://data.gc.ca/eng/ope...
            Author        string   `xml:"author"`       // Geological Survey of Cana...
            Source        string   `xml:"source"`       // Geological Survey of Cana...
            SndSeries     string   `xml:"SndSeries"`    // Bedford Institute of Ocea...
            Publisher     string   `xml:"publisher"`    // Natural Resources Canada,...
            Edition       string   `xml:"edition"`      // prelim., surficial data m...
            Meeting       string   `xml:"meeting"`      // Geological Association of...
            Documenttype  string   `xml:"documenttype"` // serial, open file, serial...
            Language      string   `xml:"language"`     // English, English, English...
            Maps          string   `xml:"maps"`         // 1 map, 5 maps, Publicatio...
            Mapinfo       string   `xml:"mapinfo"`      // surficial geology, surfic...
            Medium        string   `xml:"medium"`       // on-line; digital, digital...
            Province      string   `xml:"province"`     // Nunavut, Northwest Territ...
            Nts           string   `xml:"nts"`          // 066B, 095J; 095N; 095O; 0...
            Area          string   `xml:"area"`         // Aberdeen Lake, Mackenzie ...
            Subjects      string   `xml:"subjects"`
            Program       string   `xml:"program"`       // GEM2: Geo-mapping for Ene...
            Project       string   `xml:"project"`       // Rae Province Project Mana...
            Projectnumber string   `xml:"projectnumber"` // 340521, 343202, 340557, 3...
            Abstract      string   `xml:"abstract"`      // This new surficial geolog...
            Links         string   `xml:"links"`         // Online - En ligne (PDF, 9...
            Readme        string   `xml:"readme"`        // readme | https://geoscan....
            PPIid         string   `xml:"PPIid"`         // 34532, 35096, 35438, 2563...
        } `xml:"item"`
    } `xml:"channel"`
}
```

## Online

* try online via WASM: [https://xml-to-go.github.io/](https://xml-to-go.github.io/), thanks [YaroslavPodorvanov](https://github.com/YaroslavPodorvanov)!
* try it online at [https://blog.kowalczyk.info/tools/xmltogo/](https://blog.kowalczyk.info/tools/xmltogo/) -- thanks, [kjk](https://github.com/kjk)!

## About

[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)

Upsides:

* it works fine for non-recursive structures,
* does not need XSD or DTD,
* it is relatively convenient to access attributes, children and text,
* will generate a single struct, which make for a quite compact representation,
* simple user interface,
* comments with examples,
* schema inference across multiple files.

Downsides:

* experimental, early, buggy, unstable prototype,
* no support for recursive types (similar to *Russian Doll* strategy, [[1](https://medbiq.org/std_specs/techguidelines/xmldesignguidelines.pdf#page=7)])
* no type inference, everything is accessible as string (without a schema, type inference may fail if the type *guess* is wrong)

Bugs:

> Mapping between XML elements and data structures is inherently flawed: an XML
> element is an order-dependent collection of anonymous values, while a data
> structure is an order-independent collection of named values.

https://golang.org/pkg/encoding/xml/#pkg-note-BUG

Related projects:

* https://github.com/bemasher/JSONGen
* https://github.com/dutchcoders/XMLGen
* https://github.com/gnewton/chidley
* https://github.com/twpayne/go-xmlstruct

And other [awesome XML utilities](https://github.com/avelino/awesome-go#xml).

Presentations:

* [Lightning talk about struct generators](https://gist.github.com/miku/39e4273d15abfd7e4297071338da3349) at [GoLab 2018](https://www.golab.io/)

## Install

```
$ go install github.com/miku/zek/cmd/zek@latest
```

Debian and RPM packages:

* https://github.com/miku/zek/releases

It's in [AUR](https://aur.archlinux.org/packages/zek-bin/), too.

![](https://github.com/miku/zek/blob/master/docs/114391.png)

## Usage

```shell
$ zek -h
  -B    use a fixed banner string (e.g. for CI)
  -C    emit less compact struct
  -F    skip formatting
  -I    use verbatim innerxml instead of chardata
  -P string
        if set, write out struct within a package with the given name
  -S int
        read at most this many tags, approximately (0=unlimited)
  -c    emit more compact struct (noop, as this is the default since 0.1.7)
  -d    debug output
  -e    add comments with example
  -j    add JSON tags
  -m    omit empty Text fields
  -max-examples int
        limit number of examples (default 10)
  -n string
        use a different name for the top-level struct
  -o string
        if set, write to output file, not stdout
  -p    write out an example program
  -s    strict parsing and writing
  -t string
        emit struct for tag matching this name
  -u    filter out duplicated examples
  -version
        show version
  -x int
        max chars for example (default 25)
```

Examples:

```xml
$ cat fixtures/a.xml
<a></a>

$ zek -C < fixtures/a.xml
type A struct {
    XMLName xml.Name `xml:"a"`
    Text    string   `xml:",chardata"`
}
```

Debug output dumps the internal tree as JSON to stdout.

```json
$ zek -d < fixtures/a.xml
{"name":{"Space":"","Local":"a"}}
```

Example program:

```go
package main

import (
	"encoding/json"
	"encoding/xml"
	"fmt"
	"log"
	"os"
)

// A was generated 2017-12-05 17:35:21 by tir on apollo.
type A struct {
	XMLName xml.Name `xml:"a"`
	Text    string   `xml:",chardata"`
}

func main() {
	dec := xml.NewDecoder(os.Stdin)
	var doc A
	if err := dec.Decode(&doc); err != nil {
		log.Fatal(err)
	}
	b, err := json.Marshal(doc)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(b))
}

$ zek -C -p < fixtures/a.xml > sample.go && go run sample.go < fixtures/a.xml | jq . && rm sample.go
{
  "XMLName": {
    "Space": "",
    "Local": "a"
  },
  "Text": ""
}
```

More complex example:

```go
$ zek < fixtures/d.xml
// Root was generated 2019-06-11 16:27:04 by tir on hayiti.
type Root struct {
        XMLName xml.Name `xml:"root"`
        Text    string   `xml:",chardata"`
        A       []struct {
                Text string `xml:",chardata"`
                B    []struct {
                        Text string `xml:",chardata"`
                        C    string `xml:"c"`
                        D    string `xml:"d"`
                } `xml:"b"`
        } `xml:"a"`
}

$ zek -p < fixtures/d.xml > sample.go && go run sample.go < fixtures/d.xml | jq . && rm sample.go
{
  "XMLName": {
    "Space": "",
    "Local": "root"
  },
  "Text": "\n\n\n\n",
  "A": [
    {
      "Text": "\n  \n  \n",
      "B": [
        {
          "Text": "\n    \n  ",
          "C": "Hi",
          "D": ""
        },
        {
          "Text": "\n    \n    \n  ",
          "C": "World",
          "D": ""
        }
      ]
    },
    {
      "Text": "\n  \n",
      "B": [
        {
          "Text": "\n    \n  ",
          "C": "Hello",
          "D": ""
        }
      ]
    },
    {
      "Text": "\n  \n",
      "B": [
        {
          "Text": "\n    \n  ",
          "C": "",
          "D": "World"
        }
      ]
    }
  ]
}
```

Annotate with comments:

```go
$ zek -e < fixtures/l.xml
// Records was generated 2019-06-11 16:29:35 by tir on hayiti.
type Records struct {
        XMLName xml.Name `xml:"Records"`
        Text    string   `xml:",chardata"` // \n
        Xsi     string   `xml:"xsi,attr"`
        Record  []struct {
                Text   string `xml:",chardata"`
                Header struct {
                        Text       string `xml:",chardata"`
                        Status     string `xml:"status,attr"`
                        Identifier string `xml:"identifier"` // oai:ojs.localhost:article...
                        Datestamp  string `xml:"datestamp"`  // 2009-06-24T14:48:23Z, 200...
                        SetSpec    string `xml:"setSpec"`    // eppp:ART, eppp:ART, eppp:...
                } `xml:"header"`
                Metadata struct {
                        Text    string `xml:",chardata"`
                        Rfc1807 struct {
                                Text           string   `xml:",chardata"`
                                Xmlns          string   `xml:"xmlns,attr"`
                                Xsi            string   `xml:"xsi,attr"`
                                SchemaLocation string   `xml:"schemaLocation,attr"`
                                BibVersion     string   `xml:"bib-version"`  // v2, v2, v2...
                                ID             string   `xml:"id"`           // http://jou...
                                Entry          string   `xml:"entry"`        // 2009-06-24...
                                Organization   []string `xml:"organization"` // Proceeding...
                                Title          string   `xml:"title"`        // Introducti...
                                Type           string   `xml:"type"`
                                Author         []string `xml:"author"`       // KRAMPEN, G..
                                Copyright      string   `xml:"copyright"`    // Das Urhebe...
                                OtherAccess    string   `xml:"other_access"` // url:http:/...
                                Keyword        string   `xml:"keyword"`
                                Period         []string `xml:"period"`
                                Monitoring     string   `xml:"monitoring"`
                                Language       string   `xml:"language"` // en, en, en, e...
                                Abstract       string   `xml:"abstract"` // After a short...
                                Date           string   `xml:"date"`     // 2009-06-22 12...
                        } `xml:"rfc1807"`
                } `xml:"metadata"`
                About string `xml:"about"`
        } `xml:"Record"`
}
```

## Only consider a nested element

```go
$ zek -t metadata fixtures/z.xml
// Metadata was generated 2019-06-11 16:33:26 by tir on hayiti.
type Metadata struct {
        XMLName xml.Name `xml:"metadata"`
        Text    string   `xml:",chardata"`
        Dc      struct {
                Text  string `xml:",chardata"`
                Xmlns string `xml:"xmlns,attr"`
                Title struct {
                        Text  string `xml:",chardata"`
                        Xmlns string `xml:"xmlns,attr"`
                } `xml:"title"`
                Identifier struct {
                        Text  string `xml:",chardata"`
                        Xmlns string `xml:"xmlns,attr"`
                } `xml:"identifier"`
                Rights struct {
                        Text  string `xml:",chardata"`
                        Xmlns string `xml:"xmlns,attr"`
                        Lang  string `xml:"lang,attr"`
                } `xml:"rights"`
                AccessRights struct {
                        Text  string `xml:",chardata"`
                        Xmlns string `xml:"xmlns,attr"`
                } `xml:"accessRights"`
        } `xml:"dc"`
}
```

## Inference across files

```go
$ zek fixtures/a.xml fixtures/b.xml fixtures/c.xml
// A was generated 2017-12-05 17:40:14 by tir on apollo.
type A struct {
	XMLName xml.Name `xml:"a"`
	Text    string   `xml:",chardata"`
	B       []struct {
		Text string `xml:",chardata"`
	} `xml:"b"`
}
```

This is also useful, if you deal with archives containing XML files:

```shell
$ unzip -p 4082359.zip '*.xml' | zek -e
```

Given a directory full of zip files, you can combined find, unzip and zek:

```shell
$ for i in $(find ftp/b571 -type f -name "*zip"); do unzip -p $i '*xml'; done | zek -e
```

Another example (tarball with thousands of XML files, seemingly MARC):

```shell
$ tar -xOzf /tmp/20180725.125255.tar.gz | zek -e
// OAIPMH was generated 2018-09-26 15:03:29 by tir on sol.
type OAIPMH struct {
        XMLName        xml.Name `xml:"OAI-PMH"`
        Text           string   `xml:",chardata"`
        Xmlns          string   `xml:"xmlns,attr"`
        Xsi            string   `xml:"xsi,attr"`
        SchemaLocation string   `xml:"schemaLocation,attr"`
        ListRecords    struct {
                Text   string `xml:",chardata"`
                Record struct {
                        Text   string `xml:",chardata"`
                        Header struct {
                                Text       string `xml:",chardata"`
                                Identifier struct {
                                        Text string `xml:",chardata"` // aleph-pub:000000001, ...
                                } `xml:"identifier"`
                        } `xml:"header"`
                        Metadata struct {
                                Text   string `xml:",chardata"`
                                Record struct {
                                        Text           string `xml:",chardata"`
                                        Xmlns          string `xml:"xmlns,attr"`
                                        Xsi            string `xml:"xsi,attr"`
                                        SchemaLocation string `xml:"schemaLocation,attr"`
                                        Leader         struct
                                                Text string `xml:",chardata"` // 00001nM2.01200024
                                        } `xml:"leader"`
                                        Controlfield []struct {
                                                Text string `xml:",chardata"` // 00001nM2.01200024
                                                Tag  string `xml:"tag,attr"`
                                        } `xml:"controlfield"`
                                        Datafield []struct {
                                                Text     string `xml:",chardata"`
                                                Tag      string `xml:"tag,attr"`
                                                Ind1     string `xml:"ind1,attr"`
                                                Ind2     string `xml:"ind2,attr"`
                                                Subfield []struct {
                                                        Text string `xml:",chardata"` // KM0000002
                                                        Code string `xml:"code,attr"`
                                                } `xml:"subfield"`
                                        } `xml:"datafield"`
                                } `xml:"record"`
                        } `xml:"metadata"`
                } `xml:"record"`
        } `xml:"ListRecords"`
}
```

## Generate a package

If you want in include generated file in the build process, e.g. with [go
generate](https://go.dev/blog/generate), you may find `-P` and `-o`
helpful.

```sh
$ cat fixtures/b.xml
<a><b></b></a>
```

Run on the command line or via *go generate*:

```sh
$ zek -P mypkg -o data.go < fixtures/b.xml
```

This would write out the following in `data.go` file:

```go
// Code generated by zek; DO NOT EDIT.

package mypkg

import "encoding/xml"

// A was generated 2021-09-16 11:23:06 by tir on trieste.
type A struct {
        XMLName xml.Name `xml:"a"`
        Text    string   `xml:",chardata"`
        B       string   `xml:"b"`
}
```

Note that any existing file will be overwritten, without any warning.

## Use innerxml instead of chardata

You may want `chardata` or `innerxml` tag. Default is `chardata`, to use `innerxml` use the `-I` flag.

```sh
$ zek -B -I fixtures/d.xml
// Root was generated automatically by zek 0.1.24. DO NOT EDIT.
type Root struct {
        XMLName xml.Name `xml:"root"`
        Text    string   `xml:",innerxml"`
        A       []struct {
                Text string `xml:",innerxml"`
                B    []struct {
                        Text string `xml:",innerxml"`
                        C    string `xml:"c"`
                        D    string `xml:"d"`
                } `xml:"b"`
        } `xml:"a"`
}
```

## Misc

As a side effect, zek seems to be a useful for debugging. Example:

* https://git.io/vbUNa

This record is emitted from a typical [OAI](https://www.openarchives.org/)
server ([OJS](https://pkp.sfu.ca/ojs/), not even uncommon), yet one can quickly
spot the flaw in the structure.

Over 30 different struct generated manually in the course of a few hours
(around five minutes per source): https://git.io/vbTDo.

-- Current extent leader: [1532](https://github.com/miku/zek/blob/master/fixtures/r.txt) lines struct

Using zek:

* [Finding Aids Unleashed: Iterative Development of a Portable Publication System](https://ital.corejournals.org/index.php/ital/article/download/17328/11945), New York University Libraries



xmlquery is Golang XPath package for XML query

# xmlquery

[![Build Status](https://github.com/antchfx/xmlquery/actions/workflows/testing.yml/badge.svg)](https://github.com/antchfx/xmlquery/actions/workflows/testing.yml)
[![GoDoc](https://godoc.org/github.com/antchfx/xmlquery?status.svg)](https://godoc.org/github.com/antchfx/xmlquery)
[![Go Report Card](https://goreportcard.com/badge/github.com/antchfx/xmlquery)](https://goreportcard.com/report/github.com/antchfx/xmlquery)

# Overview

`xmlquery` is an XPath query package for XML documents, allowing you to extract
data or evaluate from XML documents with an XPath expression.

`xmlquery` has a built-in query object caching feature that caches recently used
XPATH query strings. Enabling caching can avoid recompile XPath expression for
each query.

You can visit this page to learn about the supported XPath(1.0/2.0) syntax. https://github.com/antchfx/xpath

[htmlquery](https://github.com/antchfx/htmlquery) - Package for the HTML document query.

[xmlquery](https://github.com/antchfx/xmlquery) - Package for the XML document query.

[jsonquery](https://github.com/antchfx/jsonquery) - Package for the JSON document query.

# Installation

```
 $ go get github.com/antchfx/xmlquery
```

# Quick Starts

```go
import (
	"github.com/antchfx/xmlquery"
)

func main(){
	s := `<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0">
<channel>
  <title>W3Schools Home Page</title>
  <link>https://www.w3schools.com</link>
  <description>Free web building tutorials</description>
  <item>
    <title>RSS Tutorial</title>
    <link>https://www.w3schools.com/xml/xml_rss.asp</link>
    <description>New RSS tutorial on W3Schools</description>
  </item>
  <item>
    <title>XML Tutorial</title>
    <link>https://www.w3schools.com/xml</link>
    <description>New XML tutorial on W3Schools</description>
  </item>
</channel>
</rss>`

	doc, err := xmlquery.Parse(strings.NewReader(s))
	if err != nil {
		panic(err)
	}
	channel := xmlquery.FindOne(doc, "//channel")
	if n := channel.SelectElement("title"); n != nil {
		fmt.Printf("title: %s\n", n.InnerText())
	}
	if n := channel.SelectElement("link"); n != nil {
		fmt.Printf("link: %s\n", n.InnerText())
	}
	for i, n := range xmlquery.Find(doc, "//item/title") {
		fmt.Printf("#%d %s\n", i, n.InnerText())
	}
}
```

# Getting Started

### Find specified XPath query.

```go
list, err := xmlquery.QueryAll(doc, "a")
if err != nil {
	panic(err)
}
```

#### Parse an XML from URL.

```go
doc, err := xmlquery.LoadURL("http://www.example.com/sitemap.xml")
```

#### Parse an XML from string.

```go
s := `<?xml version="1.0" encoding="utf-8"?><rss version="2.0"></rss>`
doc, err := xmlquery.Parse(strings.NewReader(s))
```

#### Parse an XML from io.Reader.

```go
f, err := os.Open("../books.xml")
doc, err := xmlquery.Parse(f)
```

#### Parse an XML in a stream fashion (simple case without elements filtering).

```go
f, _ := os.Open("../books.xml")
p, err := xmlquery.CreateStreamParser(f, "/bookstore/book")
for {
	n, err := p.Read()
	if err == io.EOF {
		break
	}
	if err != nil {
		panic(err)
	}
	fmt.Println(n)
}
```

Notes: `CreateStreamParser()` used for saving memory if your had a large XML file to parse.

#### Parse an XML in a stream fashion (simple case advanced element filtering).

```go
f, _ := os.Open("../books.xml")
p, err := xmlquery.CreateStreamParser(f, "/bookstore/book", "/bookstore/book[price>=10]")
for {
	n, err := p.Read()
	if err == io.EOF {
		break
	}
	if err != nil {
		panic(err)
	}
	fmt.Println(n)
}
```

#### Find authors of all books in the bookstore.

```go
list := xmlquery.Find(doc, "//book//author")
// or
list := xmlquery.Find(doc, "//author")
```

#### Find the second book.

```go
book := xmlquery.FindOne(doc, "//book[2]")
```

#### Find the last book.

```go
book := xmlquery.FindOne(doc, "//book[last()]")
```

#### Find all book elements and only get `id` attribute. 

```go
list := xmlquery.Find(doc,"//book/@id")
fmt.Println(list[0].InnerText) // outout @id value
```

#### Find all books with id `bk104`.

```go
list := xmlquery.Find(doc, "//book[@id='bk104']")
```

#### Find all books with price less than 5.

```go
list := xmlquery.Find(doc, "//book[price<5]")
```

#### Evaluate total price of all books.

```go
expr, err := xpath.Compile("sum(//book/price)")
price := expr.Evaluate(xmlquery.CreateXPathNavigator(doc)).(float64)
fmt.Printf("total price: %f\n", price)
```

#### Count the number of books.

```go
expr, err := xpath.Compile("count(//book)")
count := expr.Evaluate(xmlquery.CreateXPathNavigator(doc)).(float64)
```

#### Calculate the total price of all book prices.

```go
expr, err := xpath.Compile("sum(//book/price)")
price := expr.Evaluate(xmlquery.CreateXPathNavigator(doc)).(float64)
```

# Advanced Features

### Parse `UTF-16` XML file with `ParseWithOptions()`.

```go
f, _ := os.Open(`UTF-16.XML`)
// Convert UTF-16 XML to UTF-8
utf16ToUtf8Transformer := unicode.UTF16(unicode.LittleEndian, unicode.IgnoreBOM).NewDecoder()
utf8Reader := transform.NewReader(f, utf16ToUtf8Transformer)
// Sets `CharsetReader`
options := xmlquery.ParserOptions{
	Decoder: &xmlquery.DecoderOptions{
		CharsetReader: func(charset string, input io.Reader) (io.Reader, error) {
			return input, nil
		},
	},
}
doc, err := xmlquery.ParseWithOptions(utf8Reader, options)
```

### Query with custom namespace prefix.

```go
s := `<?xml version="1.0" encoding="UTF-8"?>
<pd:ProcessDefinition xmlns:pd="http://xmlns.xyz.com/process/2003" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<pd:activity name="Invoke Request-Response Service">
<pd:type>RequestReplyActivity</pd:type>
<pd:resourceType>OpClientReqActivity</pd:resourceType>
<pd:x>300</pd:x>
<pd:y>80</pd:y>
</pd:activity>
</pd:ProcessDefinition>`
nsMap := map[string]string{
	"q": "http://xmlns.xyz.com/process/2003",
	"r": "http://www.w3.org/1999/XSL/Transform",
	"s": "http://www.w3.org/2001/XMLSchema",
}
expr, _ := xpath.CompileWithNS("//q:activity", nsMap)
node := xmlquery.QuerySelector(doc, expr)
```

#### Create XML document without call `xml.Marshal`.

```go
doc := &xmlquery.Node{
	Type: xmlquery.DeclarationNode,
	Data: "xml",
	Attr: []xml.Attr{
		xml.Attr{Name: xml.Name{Local: "version"}, Value: "1.0"},
	},
}
root := &xmlquery.Node{
	Data: "rss",
	Type: xmlquery.ElementNode,
}
doc.FirstChild = root
channel := &xmlquery.Node{
	Data: "channel",
	Type: xmlquery.ElementNode,
}
root.FirstChild = channel
title := &xmlquery.Node{
	Data: "title",
	Type: xmlquery.ElementNode,
}
title_text := &xmlquery.Node{
	Data: "W3Schools Home Page",
	Type: xmlquery.TextNode,
}
title.FirstChild = title_text
channel.FirstChild = title

fmt.Println(doc.OutputXML(true))
fmt.Println(doc.OutputXMLWithOptions(WithOutputSelf()))
```

Output:

```xml
<?xml version="1.0"?><rss><channel><title>W3Schools Home Page</title></channel></rss>
```

# FAQ

#### `Find()` vs `QueryAll()`, which is better?

`Find` and `QueryAll` both do the same thing: searches all of matched XML nodes.
`Find` panics if provided with an invalid XPath query, while `QueryAll` returns
an error.

#### Can I save my query expression object for the next query?

Yes, you can. We provide `QuerySelector` and `QuerySelectorAll` methods; they
accept your query expression object.

Caching a query expression object avoids recompiling the XPath query
expression, improving query performance.

# Questions

Please let me know if you have any questions


xmlquery

Procedural XML generation API based on libxml2's xmlwriter module

xmlwriter
=========

> [!WARNING]
> This repo has [moved to sourcehut](https://git.sr.ht/~shabbyrobe/xmlwriter). The new import path is
> go.shabbyrobe.org/xmlwriter. This library is feature complete, so only bugfixes will be accepted
> in the new repo. The old import path of github.com/shabbyrobe/xmlwriter will be marked as deprecated
> at some point in 2026.

[![GoDoc](https://godoc.org/github.com/shabbyrobe/xmlwriter?status.svg)](https://godoc.org/github.com/shabbyrobe/xmlwriter)

xmlwriter is a pure-Go library providing a procedural XML generation API based
on libxml2's xmlwriter module. The package is extensively documented at
[GoDoc](https://godoc.org/github.com/shabbyrobe/xmlwriter).

Quick example:

```go
func main() {
    b := &bytes.Buffer{}
    w := xmlwriter.Open(b)
    ec := &xmlwriter.ErrCollector{}
    defer ec.Panic()

    ec.Do(
        w.StartDoc(xmlwriter.Doc{})
        w.StartElem(xmlwriter.Elem{Name: "foo"})
        w.WriteAttr(xmlwriter.Attr{Name: "a1", Value: "val1"})
        w.WriteAttr(xmlwriter.Attr{Name: "a2", Value: "val2"})
        w.WriteComment(xmlwriter.Comment{"hello"})
        w.StartElem(xmlwriter.Elem{Name: "bar"})
        w.WriteAttr(xmlwriter.Attr{Name: "a1", Value: "val1"})
        w.WriteAttr(xmlwriter.Attr{Name: "a2", Value: "val2"})
        w.StartElem(xmlwriter.Elem{Name: "baz"})
        w.EndAllFlush()
    )
    fmt.Println(b.String())
}
```

xmlwriter is about twice as quick as using the stdlib's `encoding/xml` and
offers total control of the output. If you don't require that level of control,
it's probably better to stick with `encoding/xml`

    BenchmarkWriterHuge-8     	     165	   7189290 ns/op	    4944 B/op	       4 allocs/op
    BenchmarkWriterSmall-8    	  299679	      4035 ns/op	    4944 B/op	       4 allocs/op
    BenchmarkGolangHuge-8      	      52	  21770422 ns/op	 4324496 B/op	   60008 allocs/op
    BenchmarkGolangSmall-8    	  139767	      8828 ns/op	    5936 B/op	      28 allocs/op

xmlwriter is exhaustively tested using a fairly insane mess of C scripts you
can find in the `tester/` directory.


License
-------

xmlwriter uses the Apache License 2.0. I pulled in about 60 lines of code from
the `xml/encoding` package in the Go sources and retained the copyright. Not sure 
the exact implications, IANAL. Please file an issue if I've done something wrong.



The Best Go Libraries For XML (7)

xquery

XML-Comp

xml2map

xpath

zek

xmlquery

xmlwriter