Query and update data structures using selectors from the command line. Comparable to jq/yq but supports JSON, YAML, TOML and XML with zero runtime dependencies

[![Gitbook](https://badges.aleen42.com/src/gitbook_1.svg)](https://daseldocs.tomwright.me)
[![Go Report Card](https://goreportcard.com/badge/github.com/tomwright/dasel/v3)](https://goreportcard.com/report/github.com/tomwright/dasel/v3)
[![PkgGoDev](https://pkg.go.dev/badge/github.com/tomwright/dasel)](https://pkg.go.dev/github.com/tomwright/dasel/v3)
![Test](https://github.com/TomWright/dasel/workflows/Test/badge.svg)
![Build](https://github.com/TomWright/dasel/workflows/Build/badge.svg)
[![codecov](https://codecov.io/gh/TomWright/dasel/branch/master/graph/badge.svg)](https://codecov.io/gh/TomWright/dasel)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)
![GitHub Downloads](https://img.shields.io/github/downloads/TomWright/dasel/total)
![Homebrew Formula Downloads](https://img.shields.io/homebrew/installs/dy/dasel?label=brew%20installs)
![GitHub License](https://img.shields.io/github/license/TomWright/dasel)
[![GitHub tag (latest by date)](https://img.shields.io/github/v/tag/TomWright/dasel?label=latest%20release)](https://github.com/TomWright/dasel/releases/latest)
[![Homebrew tag (latest by date)](https://img.shields.io/homebrew/v/dasel)](https://formulae.brew.sh/formula/dasel)

<div align="center">
 <img src="./daselgopher.png" alt="Dasel mascot" width="250"/>
</div>

# Dasel

Dasel (short for **Data-Select**) is a command-line tool and library for querying, modifying, and transforming data structures such as JSON, YAML, TOML, XML, and CSV.

It provides a consistent, powerful syntax to traverse and update data — making it useful for developers, DevOps, and data wrangling tasks.

---

## Features

* **Multi-format support**: JSON, YAML, TOML, XML, CSV, HCL, INI.
* **Unified query syntax**: Access data in any format with the same selectors.
* **Query & search**: Extract values, lists, or structures with intuitive syntax.
* **Modify in place**: Update, insert, or delete values directly in structured files.
* **Convert between formats**: Seamlessly transform data from JSON → YAML, TOML → JSON, etc.
* **Script-friendly**: Simple CLI integration for shell scripts and pipelines.
* **Library support**: Import and use in Go projects.

---

## Installation

### Homebrew (macOS/Linux)

```sh
brew install dasel
```

### Go Install

```sh
go install github.com/tomwright/dasel/v3/cmd/dasel@master
```

### Prebuilt Binaries

Prebuilt binaries are available on the [Releases](https://github.com/TomWright/dasel/releases) page for Linux, macOS, and Windows.

### None of the above?

See the [installation docs](https://daseldocs.tomwright.me/getting-started/installation) for more options.

---

## Basic Usage

### Selecting Values

By default, Dasel evaluates the final selector and prints the result.

```sh
echo '{"foo": {"bar": "baz"}}' | dasel -i json 'foo.bar'
# Output: "baz"
```

### Modifying Values

Update values inline:

```sh
echo '{"foo": {"bar": "baz"}}' | dasel -i json 'foo.bar = "bong"'
# Output: "bong"
```

Use `--root` to output the full document after modification:

```sh
echo '{"foo": {"bar": "baz"}}' | dasel -i json --root 'foo.bar = "bong"'
# Output:
{
 "foo": {
 "bar": "bong"
 }
}
```

Update values based on previous value:

```sh
echo '[1,2,3,4,5]' | dasel -i json --root 'each($this = $this*2)'
# Output:
[
 2,
 4,
 6,
 8,
 10
]
```

### Format Conversion

```sh
cat data.json | dasel -i json -o yaml
```

### Recursive Descent (`..`)

Searches all nested objects and arrays for a matching key or index.

```sh
echo '{"foo": {"bar": "baz"}}' | dasel -i json '..bar'
# Output:
[
 "baz"
]

```

### Search (`search`)

Finds all values matching a condition anywhere in the structure.

```sh
echo '{"foo": {"bar": "baz"}}' | dasel -i json 'search(bar == "baz")'
# Output:
[
 {
 "bar": "baz"
 }
]

```

---

## Documentation

Full documentation is available at [daseldocs.tomwright.me](https://daseldocs.tomwright.me).

---

## Contributing

Contributions are welcome! Please see the [CONTRIBUTING.md](./CONTRIBUTING.md) for details.

---

## License

MIT License. See [LICENSE](./LICENSE) for details.

## Stargazers over time

[![Stargazers over time](https://starchart.cc/TomWright/dasel.svg)](https://starchart.cc/TomWright/dasel)

dasel

A GraphQL field builder utilizing Go generics with extra utilities and features

# grapher [![Go Reference](https://pkg.go.dev/badge/github.com/reaganiwadha/grapher.svg)](https://pkg.go.dev/github.com/reaganiwadha/grapher) [![Go Report Card](https://goreportcard.com/badge/github.com/reaganiwadha/grapher)](https://goreportcard.com/report/github.com/reaganiwadha/grapher) [![codecov](https://codecov.io/gh/reaganiwadha/grapher/branch/trunk/graph/badge.svg)](https://codecov.io/gh/reaganiwadha/grapher)

A GraphQL field builder utilizing Go generics with extra utilities and features.
Depends on [graphql-go](https://github.com/graphql-go/graphql).

## Examples
### Without grapher
```go
type PostsQuery struct {
	Limit int         `json:"limit"`
	Query null.String `json:"query"`
	Tags  []string    `json:"tags"`
}

type Post struct {
	ID   int    `json:"id"`
	Body string `json:"body"`
}

func buildField() graphql.Fields {
	return graphql.Fields{
		"GetPosts": &graphql.Field{
			Description : "Returns posts",
			Type: graphql.NewList(graphql.NewObject(
				graphql.ObjectConfig{
					Name:        "Post",
					Fields:      graphql.Fields{
						"id" : &graphql.Field{
							Type: graphql.Int,
						},
						"body" : &graphql.Field{
							Type: graphql.String,
						},
					},
				},
			)),
			Args: map[string]*graphql.ArgumentConfig{
				"limit": {
					Type: graphql.NewNonNull(graphql.Int),
				},
				"query": {
					Type: graphql.String,
				},
				"tags": {
					Type: graphql.NewList(graphql.String),
				},
			},
			Resolve: func(p graphql.ResolveParams) (ret interface{}, err error) {
				var q PostsQuery
				if err = mapstructure.Decode(p.Args, &q); err != nil {
					return
				}

				/// resolver logic
				return
			},
		},
	}
}
```

### With grapher
```go
type PostsQuery struct {
	Limit int         `json:"limit"`
	Query null.String `json:"query"`
	Tags  []string    `json:"tags"`
}

type Post struct {
	ID   int    `json:"id"`
	Body string `json:"body"`
}

func buildFieldGrapher() graphql.Fields{
	return graphql.Fields{
		"GetPosts" : grapher.NewFieldBuilder[PostsQuery, []Post]().
			WithDescription("Returns posts").
			WithResolver(func(p graphql.ResolveParams, query PostsQuery) (ret []Post, err error) {
				// resolver logic
				return 
			}).
			MustBuild(),
	}
}
```

## v1 Development
This package is still a work-in-progress. The feature candidates for the v1 release is :
- [x] Struct to `graphql.Object`/`graphql.InputObject` translator utilizing struct tags
- [x] Struct to `map[string]*graphql.ArgumentConfig{}`
- [x] Configurable struct tags translator
- [x] Schema field builder utilizing go 1.18 Generics feature ensuring type safety
- [x] Resolver middlewares
- [x] Resolver arg validator
- [x] Mux-like schema building with `MutationQueryCollector`

Most features are done, but needs a little bit of polishing.

If you have any feature ideas, do not hesitate to tell and open a new issue in the Issues tab!

## Contributing
Contributions in any way, shape, or form is super welcomed! If you have any issues, ideas or even a question just open a new issue, or make a pull request. 

## License
MIT


grapher

# jsonql
JSON query expression library in Golang.

This library enables query against JSON. Currently supported operators are: (precedences from low to high)

```
||
&&
= != > < >= <= ~= !~= is isnot contains
+ -
* / %
^
( )
```

The following are the operator mappings to SQL:
* `AND` to `&&`
* `OR`	to `||`
* `RLIKE` to `~=`
* `NOT RLIKE` to `!~=`

## Install
`go get -u github.com/elgs/jsonql`

## Example
```go
package main

import (
	"fmt"

	"github.com/elgs/jsonql"
)

var jsonString = `
[
 {
 "name": "elgs",
 "gender": "m",
 "age": 35,
 "skills": [
 "Golang",
 "Java",
 "C"
 ]
 },
 {
 "name": "enny",
 "gender": "f",
 "age": 36,
 "hobby": null,
 "skills": [
 "IC",
 "Electric design",
 "Verification"
 ]
 },
 {
 "name": "sam",
 "gender": "m",
 "age": 1,
 "hobby": "dancing",
 "skills": [
 "Eating",
 "Sleeping",
 "Crawling"
 ]
 }
]
`

func main() {
	parser, err := jsonql.NewStringQuery(jsonString)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println(parser.Query("name='elgs'"))
	//[map[skills:[Golang Java C] name:elgs gender:m age:35]] <nil>

	fmt.Println(parser.Query("name='elgs' && gender='f'"))
	//[] <nil>

	fmt.Println(parser.Query("age<10 || (name='enny' && gender='f')"))
	// [map[hobby:<nil> skills:[IC Electric design Verification] name:enny gender:f age:36] map[name:sam gender:m age:1 hobby:dancing skills:[Eating Sleeping Crawling]]] <nil>

	fmt.Println(parser.Query("age<10"))
	// [map[gender:m age:1 hobby:dancing skills:[Eating Sleeping Crawling] name:sam]] <nil>

	fmt.Println(parser.Query("1=0"))
	//[] <nil>

	fmt.Println(parser.Query("age=(2*3)^2"))
	//[map[skills:[IC Electric design Verification] name:enny gender:f age:36 hobby:<nil>]] <nil>

	fmt.Println(parser.Query("name ~= 'e.*'"))
	// [map[name:elgs gender:m age:35 skills:[Golang Java C]] map[hobby:<nil> skills:[IC Electric design Verification] name:enny gender:f age:36]] <nil>

	fmt.Println(parser.Query("name='el'+'gs'"))
	fmt.Println(parser.Query("age=30+5.0"))
	fmt.Println(parser.Query("age=40.0-5"))
	fmt.Println(parser.Query("age=70-5*7"))
	fmt.Println(parser.Query("age=70.0/2.0"))
	fmt.Println(parser.Query("age=71%36"))
	// [map[name:elgs gender:m age:35 skills:[Golang Java C]]] <nil>

	fmt.Println(parser.Query("hobby is defined"))
	// [map[name:enny gender:f age:36 hobby:<nil> skills:[IC Electric design Verification]] map[name:sam gender:m age:1 hobby:dancing skills:[Eating Sleeping Crawling]]] <nil>

	fmt.Println(parser.Query("hobby isnot defined"))
	// [map[name:sam gender:m age:1 skills:[Eating Sleeping Crawling]]] <nil>

	fmt.Println(parser.Query("hobby is null"))
	// [map[hobby:<nil> skills:[IC Electric design Verification] name:enny gender:f age:36]] <nil>

	fmt.Println(parser.Query("hobby isnot null"))
	// [map[name:sam gender:m age:1 hobby:dancing skills:[Eating Sleeping Crawling]]] <nil>

	fmt.Println(parser.Query("skills contains 'Eating'"))
	// [map[age:1 gender:m hobby:dancing name:sam skills:[Eating Sleeping Crawling]]] <nil>
}
```

## Query Expressions
For details of query expressions, please read: https://github.com/elgs/gojq

jsonql

# api-fu ![GitHub Actions](https://github.com/ccbrown/api-fu/workflows/Build/badge.svg) [![Go Report Card](https://goreportcard.com/badge/github.com/ccbrown/api-fu)](https://goreportcard.com/report/github.com/ccbrown/api-fu) [![codecov](https://codecov.io/gh/ccbrown/api-fu/branch/master/graph/badge.svg)](https://codecov.io/gh/ccbrown/api-fu) [![Documentation](https://godoc.org/github.com/ccbrown/api-fu?status.svg)](https://godoc.org/github.com/ccbrown/api-fu) [![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)

**api-fu** (noun)
  1. (informal) Mastery of APIs. 💪

## Packages

* The top level `apifu` package is an opinionated library that aims to make it as easy as possible to build APIs that conform to API-fu's ideals. See the examples directory for example usage.
* The `graphql` package is an unopinionated library for building GraphQL APIs. If you agree with API-fu's ideals, you should use `apifu` instead, but if you want something lower level, the `graphql` package is still an excellent standalone GraphQL library. It fully supports all features of the [June 2018 spec](https://graphql.github.io/graphql-spec/June2018/).
* The `graphql/transport` directory contains unopinionated libraries for using various transport protocols. For example, it contains transport implementations that allow you to serve your GraphQL API via WebSockets and provide subscription functionality.
* The `cmd/gql-client-gen` package provides a CLI tool that can be used to generate types for use by client code. THis allows your client's queries to be type-safe and fully validated at compile time.

### Experimental Packages

The above packages are mature and have been thoroughly proven in real-world, production deployments. The following packages have not yet seen such rigorous real-world testing and are thus considered experimental. They are fully functional and well unit tested, but may change at any time and are not yet subject to any compatibility guarantees.

* The `jsonapi` package is a library for building [JSON:API](https://jsonapi.org) APIs. It's somewhat high level, but is no more opinionated than JSON:API itself is. However, it does hold some of those opinions more strongly (i.e. it doesn't support violating many of the JSON:API spec's recommendations and "SHOULD"s).

## Usage

API-fu builds GraphQL APIs with code. To begin, you need a config that at least defines a query field:

```go
var fuCfg apifu.Config

fuCfg.AddQueryField("foo", &graphql.FieldDefinition{
    Type: graphql.StringType,
    Resolve: func(ctx *graphql.FieldContext) (interface{}, error) {
        return "bar", nil
    },
})
```

From there, you can build the API:

```go
fu, err := apifu.NewAPI(&fuCfg)
if err != nil {
    panic(err)
}
```

And serve it:

```go
fu.ServeGraphQL(w, r)
```

API-fu also has first-class support for common patterns such as nodes that are queryable using global ids. See the examples directory for more complete example code.

## Features

### ✅ Supports all features of the [latest GraphQL spec](https://spec.graphql.org/June2018/).

This includes null literals, error extensions, subscriptions, and directives.

### 🚅 Fast!

The graphql package is over twice as fast and several times more memory efficient than its inspiration ([graphql-go/graphql](https://github.com/graphql-go/graphql)).

```
pkg: github.com/ccbrown/api-fu/graphql/benchmarks
BenchmarkAPIFu
BenchmarkAPIFu-16        	     765	   1553517 ns/op	  890575 B/op	   22587 allocs/op
BenchmarkGraphQLGo
BenchmarkGraphQLGo-16    	     315	   3753681 ns/op	 3990220 B/op	   45952 allocs/op
```

### ⚡️ Supports efficient batching and concurrency without the use of goroutines.

The `graphql` package supports virtually any batching or concurrency pattern using low level primitives.

The `apifu` package provides high level ways to use them.

For example, you can define a resolver like this to do work in a goroutine:

```go
fuCfg.AddQueryField("myField", &graphql.FieldDefinition{
    Type: graphql.IntType,
    Resolve: func(ctx *graphql.FieldContext) (interface{}, error) {
        return Go(ctx.Context, func() (interface{}, error) {
            return doSomethingComplex(), nil
        }), nil
    },
})
```

Or you can define a resolver like this to batch up queries, allowing you to minimize round trips to your database:

```go
fuCfg.AddQueryField("myField", &graphql.FieldDefinition{
    Type: graphql.IntType,
    Resolve: Batch(func(ctx []*graphql.FieldContext) []graphql.ResolveResult {
        return resolveABunchOfTheseAtOnce(ctx)
    },
})
```

### 💡 Provides implementations for commonly used scalar types.

For example, the `apifu` package provides date-time and long (but JavaScript safe) integers.

### 📡 Implements handlers for HTTP, the [Apollo graphql-ws protocol](https://github.com/apollographql/subscriptions-transport-ws), and the [newer graphql-transport-ws protocol](https://github.com/enisdenjo/graphql-ws).

Once you've built your API, all you have to do is:

```go
fu.ServeGraphQL(w, r)
```

Or:

```go
fu.ServeGraphQLWS(w, r)
```

### 📖 Provides easy-to-use helpers for creating connections adhering to the [Relay Cursor Connections Specification](https://facebook.github.io/relay/graphql/connections.htm).

Just provide a name, cursor constructor, edge fields, and edge getter:

```go
{
    "messagesConnection": apifu.TimeBasedConnection(&apifu.TimeBasedConnectionConfig{
        NamePrefix: "ChannelMessages",
        EdgeCursor: func(edge interface{}) apifu.TimeBasedCursor {
            message := edge.(*model.Message)
            return apifu.NewTimeBasedCursor(message.Time, string(message.Id))
        },
        EdgeFields: map[string]*graphql.FieldDefinition{
            "node": &graphql.FieldDefinition{
                Type: graphql.NewNonNullType(messageType),
                Resolve: func(ctx *graphql.FieldContext) (interface{}, error) {
                    return ctx.Object, nil
                },
            },
        },
        EdgeGetter: func(ctx *graphql.FieldContext, minTime time.Time, maxTime time.Time, limit int) (interface{}, error) {
            return ctxSession(ctx.Context).GetMessagesByChannelIdAndTimeRange(ctx.Object.(*model.Channel).Id, minTime, maxTime, limit)
        },
    }),
}
```

### 🛠 Can generate Apollo-like client-side type definitions and validate queries in source code.

The `gql-client-gen` tool can be used to generate types for use in client-side code as well as validate queries at compile-time. The generated types intelligently unmarshal inline fragments and fragment spreads based on `__typename` values.

See [cmd/gql-client-gen](cmd/gql-client-gen) for details.

### 🚔 Calculates operation costs during validation for rate limiting and metering

During validation, you can specify a max operation cost or get the actual cost
of an operation using customizable cost definitions:

```go
doc, errs := graphql.ParseAndValidate(req.Query, req.Schema, req.ValidateCost(maxCost, &actualCost))
```

## API Design Guidelines

The following are guidelines that are recommended for all new GraphQL APIs. API-fu aims to make it easy to conform to these for robust and future-proof APIs:

* All mutations should resolve to result types. No mutations should simply resolve to a node. For example, a `createUser` mutation should resolve to a `CreateUserResult` object with a `user` field rather than simply resolving to a `User`. This is necessary to keep mutations extensible. Likewise, subscriptions should not resolve directly to node types. For example, a subscription for messages in a chat room (`chatRoomMessages`) should resolve to a `ChatRoomMessagesEvent` type.
* Nodes with 1-to-many relationships should make related nodes available via [Relay Cursor Connections](https://facebook.github.io/relay/graphql/connections.htm). Nodes should not have fields that simply resolve to lists of related nodes. Additionally, all connections must require a `first` or `last` argument that specifies the upper bound on the number of nodes returned by that connection. This makes it possible to determine an upper bound on the number of nodes returned by a query before that query begins execution, e.g. using rules similar to [GitHub's](https://developer.github.com/v4/guides/resource-limitations/).
* Mutations that modify nodes should always include the updated version of that node in the result. This makes it easy for clients to maintain up-to-date state and tolerate eventual consistency (If a client updates a resource, then immediately requests it in a subsequent query, the server may provide a version of the resource that was cached before the update.).
* Nodes should provide revision numbers. Each time a node is modified, the revision number must increment. This helps clients maintain up-to-date state and enables simultaneous change detection.
* It should be easy for clients to query historical data and subscribe to real-time data without missing anything due to race conditions. The most transparent and fool-proof way to facilitate this is to make subscriptions immediately push a small history of events to clients as soon as they're started. The pushed history should generally only need to cover a few seconds' worth of events. If queries use eventual consistency, the pushed history should be at least as large as the query cache's TTL.

## Versioning and Compatibility Guarantees

This library is not versioned. However, one guarantee is made: Unless otherwise noted, backwards-incompatible changes made will break your build at compile-time. If your application compiles after updating API-fu, you're good to go.


api-fu

A drop-in query language for any database schema

[![Go Reference](https://pkg.go.dev/badge/github.com/seldonio/goven.svg)](https://pkg.go.dev/github.com/seldonio/goven)
[![Go Report Card](https://goreportcard.com/badge/github.com/seldonio/goven)](https://goreportcard.com/report/github.com/seldonio/goven)
[![codecov](https://codecov.io/gh/seldonio/goven/branch/master/graph/badge.svg?token=ZBCTOI896Y)](https://codecov.io/gh/seldonio/goven)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go) 

# Goven 🧑‍🍳

Goven (go-oven) is a go library that allows you to have a drop-in query language for your database schema. 

* Take any gorm database object and with a few lines of code you can make it searchable.
* [Safe query language not exposing SQL to your users](https://imgs.xkcd.com/comics/exploits_of_a_mom.png).
* Easily extensible to support more advanced queries for your own schema.
* Basic grammar that allows for powerful queries.

Like a real life oven, it takes something raw (your database struct + a query input) and produces something specific (the schema specific parser + SQL output). We call the adaptors "recipes" that Goven can make. Currently Goven only supports a SQL adaptor, but the AST produced by the lexer/parser can easily be extended to other query languages.

## Recipes

### Basic Example

You can make a basic query using gorm against your database, something like this: 

```go
reflection := reflect.ValueOf(&User{})
queryAdaptor, err := sql_adaptor.NewDefaultAdaptorFromStruct(reflection)
if err != nil {
 return nil, err
}

dbQuery := db.WithContext(ctx)
parsedQuery, err := queryAdaptor.Parse("(name=james AND age > 11) OR email=fred@gmail.com")
if err != nil {
	return nil, err
}
dbQuery = query.Model(User{}).Where(parsedQuery.Raw, sql_adaptor.StringSliceToInterfaceSlice(parsedQuery.Values)...)
err = dbQuery.Find(&users).Error
```

The values are interpolated to prevent injection attacks.

### Extension Example

You can also extend the basic query language with regex matchers. An example would be having a Tag struct on your User schema.

```go
type User struct {
	gorm.Model
	name string
	tags []Tag
}

type Tag struct {
	gorm.Model
	Key string
	Value string
}
```

You can make this searchable my defining a regex and a custom matcher.

e.g if we want `tags[key]=value` to work then we can add the following matcher when creating the adaptor.

```go
KeyValueRegex = `(tags)\[(.+)\]`

// keyValueMatcher is a custom matcher for and tags[y].
func keyValueMatcher(ex *goven_parser.Expression) (*goven_sql.SqlResponse, error) {
	reg, err := regexp.Compile(KeyValueRegex)
	if err != nil {
		return nil, err
	}
	slice := reg.FindStringSubmatch(ex.Field)
	if slice == nil {
		return nil, errors.New("didn't match regex expression")
	}
	if len(slice) < 3 {
		return nil, errors.New("regex match slice is too short")
	}
	sq := goven_sql.SqlResponse{
		Raw: fmt.Sprintf("id IN (SELECT user_id FROM %s WHERE key=? AND value%s?)", slice[1], ex.Comparator),
		Values: []string{slice[2], ex.Value},
	}
	return &sq, nil
}
```

### Protecting Fields

Sometimes we may not want particular fields to be searchable by end users. You can protect them by removing them from the fields mapping when creating your adaptor.

```go
defaultFields := goven_sql.FieldParseValidatorFromStruct(gorm)
delete(defaultFields, "fieldname")
```

## Grammar

Goven has a simple syntax that allows for powerful queries.

Fields can be compared using the following operators: 

`=`, `!=`, `>=`, `<=`, `<`, `>`, `%`

The `%` operator allows you to do partial string matching using LIKE.

Multiple queries can be combined using `AND`, `OR`.

Together this means you can build up a query like this:

`model_name=iris AND version>=2.0`

More advanced queries can be built up using bracketed expressions:

`(model_name=iris AND version>=2.0) OR artifact_type=TENSORFLOW`

goven

![gqlgen](https://user-images.githubusercontent.com/980499/133180111-d064b38c-6eb9-444b-a60f-7005a6e68222.png)

# gqlgen [![Integration](https://github.com/99designs/gqlgen/actions/workflows/integration.yml/badge.svg)](https://github.com/99designs/gqlgen/actions) [![Coverage Status](https://coveralls.io/repos/github/99designs/gqlgen/badge.svg?branch=master)](https://coveralls.io/github/99designs/gqlgen?branch=master) [![Go Report Card](https://goreportcard.com/badge/github.com/99designs/gqlgen)](https://goreportcard.com/report/github.com/99designs/gqlgen) [![Go Reference](https://pkg.go.dev/badge/github.com/99designs/gqlgen.svg)](https://pkg.go.dev/github.com/99designs/gqlgen) [![Read the Docs](https://badgen.net/badge/docs/available/green)](http://gqlgen.com/)

## What is gqlgen?

[gqlgen](https://github.com/99designs/gqlgen) is a Go library for building GraphQL servers without any fuss. 

- **gqlgen is based on a Schema first approach** — You get to Define your API using the GraphQL [Schema Definition Language](http://graphql.org/learn/schema/).
- **gqlgen prioritizes Type safety** — You should never see `map[string]interface{}` here.
- **gqlgen enables Codegen** — We generate the boring bits, so you can focus on building your app quickly.

Still not convinced enough to use **gqlgen**? Compare **gqlgen** with other Go graphql [implementations](https://gqlgen.com/feature-comparison/)

## Quick start

1. [Initialise a new go module](https://golang.org/doc/tutorial/create-module)

```shell
mkdir example
cd example
go mod init example
```

2. Add `github.com/99designs/gqlgen` to your project, as a [tool dependency](https://go.dev/doc/modules/managing-dependencies#tools)

```shell
go get -tool github.com/99designs/gqlgen
```

3. Initialise gqlgen config and generate models

```shell
go tool gqlgen init
```

4. Start the graphql server

```shell
go run server.go
```

More help to get started:

- [Getting started tutorial](https://gqlgen.com/getting-started/) - a comprehensive guide to help you get started
- [Real-world examples](https://github.com/99designs/gqlgen/tree/master/_examples) show how to create GraphQL applications
- [Reference docs](https://pkg.go.dev/github.com/99designs/gqlgen) for the APIs

## Reporting Issues

If you think you've found a bug, or something isn't behaving the way you think it should, please raise an [issue](https://github.com/99designs/gqlgen/issues) on GitHub.

## Contributing

We welcome contributions, Read our [Contribution Guidelines](https://github.com/99designs/gqlgen/blob/master/CONTRIBUTING.md) to learn more about contributing to **gqlgen**

## Frequently asked questions

### How do I prevent fetching child objects that might not be used?

When you have nested or recursive schema like this:

```graphql
type User {
	id: ID!
	name: String!
	friends: [User!]!
}
```

You need to tell gqlgen that it should only fetch friends if the user requested it. There are two ways to do this:

### Using Custom Models

Write a custom model that omits the friends field:

```go
type User struct {
 ID int
 Name string
}
```

And reference the model in `gqlgen.yml`:

```yaml
# gqlgen.yml
models:
 User:
 model: github.com/you/pkg/model.User # go import path to the User struct above
```

### Using Explicit Resolvers

If you want to keep using the generated model, mark the field as requiring a resolver explicitly in `gqlgen.yml` like this:

```yaml
# gqlgen.yml
models:
 User:
 fields:
 friends:
 resolver: true # force a resolver to be generated
```

After doing either of the above and running generate we will need to provide a resolver for friends:

```go
func (r *userResolver) Friends(ctx context.Context, obj *User) ([]*User, error) {
 // select * from user where friendid = obj.ID
 return friends, nil
}
```

You can also use inline config with directives to achieve the same result

```graphql
directive @goModel(
	model: String
	models: [String!]
) on OBJECT | INPUT_OBJECT | SCALAR | ENUM | INTERFACE | UNION

directive @goField(
	forceResolver: Boolean
	name: String
	omittable: Boolean
	type: String
 autoBindGetterHaser: Boolean
 forceGenerate: Boolean
	batch: Boolean
) on INPUT_FIELD_DEFINITION | FIELD_DEFINITION

type User @goModel(model: "github.com/you/pkg/model.User") {
	id: ID! @goField(name: "todoId")
	friends: [User!]! @goField(forceResolver: true)
	# When omit_resolver_fields is enabled, this field will still be generated in the struct
	data: String! @goField(forceResolver: true, forceGenerate: true)
}
```

The field resolvers will be executed concurrently in separate goroutines. The degree of concurrency can be customized with the [`worker_limit` configuration attribute](https://gqlgen.com/config/).

### Can I change the type of the ID from type String to Type Int?

Yes! You can by remapping it in config as seen below:

```yaml
models:
 ID: # The GraphQL type ID is backed by
 model:
 - github.com/99designs/gqlgen/graphql.IntID # a go integer
 - github.com/99designs/gqlgen/graphql.ID # or a go string
 - github.com/99designs/gqlgen/graphql.UintID # or a go uint
```

This means gqlgen will be able to automatically bind to strings or ints for models you have written yourself, but the
first model in this list is used as the default type and it will always be used when:

- Generating models based on schema
- As arguments in resolvers

There isn't any way around this, gqlgen has no way to know what you want in a given context.

### Why do my interfaces have getters? Can I disable these?

These were added in v0.17.14 to allow accessing common interface fields without casting to a concrete type.
However, certain fields, like Relay-style Connections, cannot be implemented with simple getters.

If you'd prefer to not have getters generated in your interfaces, you can add the following in your `gqlgen.yml`:

```yaml
# gqlgen.yml
omit_getters: true
```

## Other Resources

- [Christopher Biscardi @ Gophercon UK 2018](https://youtu.be/FdURVezcdcw)
- [Introducing gqlgen: a GraphQL Server Generator for Go](https://99designs.com.au/blog/engineering/gqlgen-a-graphql-server-generator-for-go/)
- [Dive into GraphQL by Iván Corrales Solera](https://medium.com/@ivan.corrales.solera/dive-into-graphql-9bfedf22e1a)
- [Sample Project built on gqlgen with Postgres by Oleg Shalygin](https://github.com/oshalygin/gqlgen-pg-todo-example)
- [Hackernews GraphQL Server with gqlgen by Shayegan Hooshyari](https://www.howtographql.com/graphql-go/0-introduction/)

gqlgen

A simple Go package to Query over JSON Data

![gojsonq-logo](gojsonq.png)

[![Build Status](https://travis-ci.org/thedevsaddam/gojsonq.svg?branch=master)](https://travis-ci.org/thedevsaddam/gojsonq)
[![Project status](https://img.shields.io/badge/version-v2-green.svg)](https://github.com/thedevsaddam/gojsonq/releases)
[![Go Report Card](https://goreportcard.com/badge/github.com/thedevsaddam/gojsonq)](https://goreportcard.com/report/github.com/thedevsaddam/gojsonq)
[![Coverage Status](https://coveralls.io/repos/github/thedevsaddam/gojsonq/badge.svg?branch=master)](https://coveralls.io/github/thedevsaddam/gojsonq)
[![GoDoc](https://godoc.org/github.com/thedevsaddam/gojsonq?status.svg)](https://pkg.go.dev/github.com/thedevsaddam/gojsonq/v2)
[![License](https://img.shields.io/dub/l/vibe-d.svg)](LICENSE.md)

A simple Go package to Query over JSON Data. It provides [simple](https://github.com/thedevsaddam/gojsonq/wiki/Queries#jsonstringjson), [elegant](https://github.com/thedevsaddam/gojsonq/wiki/Queries#selectproperties) and [fast](https://github.com/thedevsaddam/gojsonq/wiki/Benchmark) [ODM](https://github.com/thedevsaddam/gojsonq/wiki/Queries#frompath) like API to access, query JSON document

### Installation

Install the package using
```go
$ go get github.com/thedevsaddam/gojsonq/v2
```

### Usage

To use the package import it in your `*.go` code
```go
import "github.com/thedevsaddam/gojsonq/v2"
```

Let's see a quick example:

[See in playground](https://play.golang.org/p/UiqyllP2vkn)

```go
package main

import gojsonq "github.com/thedevsaddam/gojsonq/v2"

func main() {
	const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`
	name := gojsonq.New().FromString(json).Find("name.first")
	println(name.(string)) // Tom
}
```

Another example:

[See in playground](https://play.golang.org/p/QLVxpi6nVbi)

```go
package main

import (
	"fmt"

	gojsonq "github.com/thedevsaddam/gojsonq/v2"
)

func main() {
	const json = `{"city":"dhaka","type":"weekly","temperatures":[30,39.9,35.4,33.5,31.6,33.2,30.7]}`
	avg := gojsonq.New().FromString(json).From("temperatures").Avg()
	fmt.Printf("Average temperature: %.2f", avg) // 33.471428571428575
}
```

You can query your document using the various query methods such as **[Find](https://github.com/thedevsaddam/gojsonq/wiki/Queries#findpath)**, **[First](https://github.com/thedevsaddam/gojsonq/wiki/Queries#first)**, **[Nth](https://github.com/thedevsaddam/gojsonq/wiki/Queries#nthindex)**, **[Pluck](https://github.com/thedevsaddam/gojsonq/wiki/Queries#pluckproperty)**, **[Where](https://github.com/thedevsaddam/gojsonq/wiki/Queries#wherekey-op-val)**, **[OrWhere](https://github.com/thedevsaddam/gojsonq/wiki/Queries#orwherekey-op-val)**, **[WhereIn](https://github.com/thedevsaddam/gojsonq/wiki/Queries#whereinkey-val)**, **[WhereStartsWith](https://github.com/thedevsaddam/gojsonq/wiki/Queries#wherestartswithkey-val)**, **[WhereEndsWith](https://github.com/thedevsaddam/gojsonq/wiki/Queries#whereendswithkey-val)**, **[WhereContains](https://github.com/thedevsaddam/gojsonq/wiki/Queries#wherecontainskey-val)**, **[Sort](https://github.com/thedevsaddam/gojsonq/wiki/Queries#sortorder)**, **[GroupBy](https://github.com/thedevsaddam/gojsonq/wiki/Queries#groupbyproperty)**, **[SortBy](https://github.com/thedevsaddam/gojsonq/wiki/Queries#sortbyproperty-order)** and so on. Also you can aggregate data after query using **[Avg](https://github.com/thedevsaddam/gojsonq/wiki/Queries#avgproperty)**, **[Count](https://github.com/thedevsaddam/gojsonq/wiki/Queries#count)**, **[Max](https://github.com/thedevsaddam/gojsonq/wiki/Queries#maxproperty)**, **[Min](https://github.com/thedevsaddam/gojsonq/wiki/Queries#minproperty)**, **[Sum](https://github.com/thedevsaddam/gojsonq/wiki/Queries#sumproperty)** etc.

## Find more query API in [Wiki page](https://github.com/thedevsaddam/gojsonq/wiki/Queries)

## Bugs and Issues

If you encounter any bugs or issues, feel free to [open an issue at
github](https://github.com/thedevsaddam/gojsonq/issues).

Also, you can shoot me an email to
<mailto:thedevsaddam@gmail.com> for hugs or bugs.

## Credit

Special thanks to [Nahid Bin Azhar](https://github.com/nahid) for the inspiration and guidance for the package. Thanks to [Ahmed Shamim Hasan Shaon](https://github.com/me-shaon) for his support from the very beginning.

## Contributors
* [Lenin Hasda](https://github.com/leninhasda)
* [Sadlil Rhythom](https://github.com/sadlil)
* [See contributors list here](https://github.com/thedevsaddam/gojsonq/graphs/contributors)

## Contribution
If you are interested to make the package better please send pull requests or create an issue so that others can fix.
[Read the contribution guide here](CONTRIBUTING.md)

## License
The **gojsonq** is an open-source software licensed under the [MIT License](LICENSE.md).

gojsonq

GraphQL server with a focus on ease of use

# graphql-go [![Sourcegraph](https://sourcegraph.com/github.com/graph-gophers/graphql-go/-/badge.svg)](https://sourcegraph.com/github.com/graph-gophers/graphql-go?badge) [![Go](https://github.com/graph-gophers/graphql-go/actions/workflows/go.yml/badge.svg)](https://github.com/graph-gophers/graphql-go/actions/workflows/go.yml) [![Go Report](https://goreportcard.com/badge/github.com/graph-gophers/graphql-go)](https://goreportcard.com/report/github.com/graph-gophers/graphql-go) [![GoDoc](https://godoc.org/github.com/graph-gophers/graphql-go?status.svg)](https://godoc.org/github.com/graph-gophers/graphql-go)

<img src="docs/img/logo.png" width="300">

The goal of this project is to provide full support of the [October 2021 GraphQL specification](https://spec.graphql.org/October2021/) with a set of idiomatic, easy to use Go packages.

While still under development (`internal` APIs are almost certainly subject to change), this library is safe for production use.

## Features

- minimal API
- support for `context.Context`
- support for the `OpenTelemetry` and `OpenTracing` standards
- schema type-checking against resolvers
- resolvers are matched to the schema based on method sets (can resolve a GraphQL schema with a Go interface or Go struct).
- handles panics in resolvers
- parallel execution of resolvers
- inspect the selected fields and their args to prefetch data and avoid the N+1 query problem
- subscriptions
 - [sample WS transport](https://github.com/graph-gophers/graphql-transport-ws)

## (Some) Documentation [![GoDoc](https://godoc.org/github.com/graph-gophers/graphql-go?status.svg)](https://godoc.org/github.com/graph-gophers/graphql-go)

### Getting started

In order to run a simple GraphQL server locally create a `main.go` file with the following content:
```go
package main

import (
	"log"
	"net/http"

	graphql "github.com/graph-gophers/graphql-go"
	"github.com/graph-gophers/graphql-go/relay"
)

type query struct{}

func (query) Hello() string { return "Hello, world!" }

func main() {
	s := `
 type Query {
 hello: String!
 }
 `
	schema := graphql.MustParseSchema(s, &query{})
	http.Handle("/query", &relay.Handler{Schema: schema})
	log.Fatal(http.ListenAndServe(":8080", nil))
}

```
Then run the file with `go run main.go`. To test:
	 
```sh
curl -XPOST -d '{"query": "{ hello }"}' localhost:8080/query
```
For more realistic usecases check our [examples section](https://github.com/graph-gophers/graphql-go/wiki/Examples).

### Resolvers

A resolver must have one method or field for each field of the GraphQL type it resolves. The method or field name has to be [exported](https://golang.org/ref/spec#Exported_identifiers) and match the schema's field's name in a non-case-sensitive way.
You can use struct fields as resolvers by using `SchemaOpt: UseFieldResolvers()`. For example,
```
opts := []graphql.SchemaOpt{graphql.UseFieldResolvers()}
schema := graphql.MustParseSchema(s, &query{}, opts...)
``` 

When using `UseFieldResolvers` schema option, a struct field will be used *only* when:
- there is no method for a struct field
- a struct field does not implement an interface method
- a struct field does not have arguments

The method has up to two arguments:

- Optional `context.Context` argument.
- Mandatory `*struct { ... }` argument if the corresponding GraphQL field has arguments. The names of the struct fields have to be [exported](https://golang.org/ref/spec#Exported_identifiers) and have to match the names of the GraphQL arguments in a non-case-sensitive way.

The method has up to two results:

- The GraphQL field's value as determined by the resolver.
- Optional `error` result.

Example for a simple resolver method:

```go
func (r *helloWorldResolver) Hello() string {
	return "Hello world!"
}
```

The following signature is also allowed:

```go
func (r *helloWorldResolver) Hello(ctx context.Context) (string, error) {
	return "Hello world!", nil
}
```

### Separate resolvers for different operations
This feature was released in `v1.6.0`.

The GraphQL specification allows for fields with the same name defined in different query types. For example, the schema below is a valid schema definition:
```graphql
schema {
 query: Query
 mutation: Mutation
}

type Query {
 hello: String!
}

type Mutation {
 hello: String!
}
```
The above schema would result in name collision if we use a single resolver struct because fields from both operations correspond to methods in the root resolver (the same Go struct). In order to resolve this issue, the library allows resolvers for query, mutation and subscription operations to be separated using the `Query`, `Mutation` and `Subscription` methods of the root resolver. These special methods are optional and if defined return the resolver for each opeartion. For example, the following is a resolver corresponding to the schema definition above. Note that there is a field named `hello` in both the query and the mutation definitions:

```go
type RootResolver struct{}
type QueryResolver struct{}
type MutationResolver struct{}

func(r *RootResolver) Query() *QueryResolver {
 return &QueryResolver{}
}

func(r *RootResolver) Mutation() *MutationResolver {
 return &MutationResolver{}
}

func (*QueryResolver) Hello() string {
	return "Hello query!"
}

func (*MutationResolver) Hello() string {
	return "Hello mutation!"
}

schema := graphql.MustParseSchema(sdl, &RootResolver{}, nil)
...
```

### Schema Options

- `UseStringDescriptions()` enables the usage of double quoted and triple quoted. When this is not enabled, comments are parsed as descriptions instead.
- `UseFieldResolvers()` specifies whether to use struct field resolvers.
- `MaxDepth(n int)` specifies the maximum field nesting depth in a query. The default is 0 which disables max depth checking.
- `MaxParallelism(n int)` specifies the maximum number of resolvers per request allowed to run in parallel. The default is 10.
- `MaxPooledBufferCap(n int)` specifies the maximum buffer capacity of buffers stored in the internal memory pool. Defaults to 16KB. Buffers larger than this limit are discarded instead of pooled.
- `Tracer(tracer trace.Tracer)` is used to trace queries and fields. It defaults to `noop.Tracer`.
- `Logger(logger log.Logger)` is used to log panics during query execution. It defaults to `exec.DefaultLogger`.
- `PanicHandler(panicHandler errors.PanicHandler)` is used to transform panics into errors during query execution. It defaults to `errors.DefaultPanicHandler`.
- `DisableIntrospection()` disables introspection queries.
- `DisableFieldSelections()` disables capturing child field selections used by helper APIs (see below).
- `DisableMemoryPooling()` disables internal execution-path memory pooling. Pooling is enabled by default; this option is intended for diagnostics and benchmark comparisons.
- `OverlapValidationLimit(n int)` sets a hard cap on examined overlap pairs during validation; exceeding it emits `OverlapValidationLimitExceeded` error.

### Field Selection Inspection Helpers

Resolvers can introspect which immediate child fields were requested using:

```go
graphql.SelectedFieldNames(ctx) // []string of direct child schema field names
graphql.HasSelectedField(ctx, "name") // bool
graphql.SortedSelectedFieldNames(ctx) // sorted copy
```

Use cases include building projection lists for databases or conditionally avoiding expensive sub-fetches. The helpers are intentionally shallow (only direct children) and fragment spreads / inline fragments are flattened with duplicates removed; meta fields (e.g. `__typename`) are excluded.

Performance: selection data is computed lazily only when a helper is called. If you never call them there is effectively no additional overhead. To remove even the small context value insertion you can opt out with `DisableFieldSelections()`; helpers then return empty results.

For more detail and examples see the [docs](https://godoc.org/github.com/graph-gophers/graphql-go).

### Custom Errors

Errors returned by resolvers can include custom extensions by implementing the `ResolverError` interface:

```go
type ResolverError interface {
	error
	Extensions() map[string]interface{}
}
```

Example of a simple custom error:

```go
type droidNotFoundError struct {
	Code string `json:"code"`
	Message string `json:"message"`
}

func (e droidNotFoundError) Error() string {
	return fmt.Sprintf("error [%s]: %s", e.Code, e.Message)
}

func (e droidNotFoundError) Extensions() map[string]interface{} {
	return map[string]interface{}{
		"code": e.Code,
		"message": e.Message,
	}
}
```

Which could produce a GraphQL error such as:

```go
{
 "errors": [
 {
 "message": "error [NotFound]: This is not the droid you are looking for",
 "path": [
 "droid"
 ],
 "extensions": {
 "code": "NotFound",
 "message": "This is not the droid you are looking for"
 }
 }
 ],
 "data": null
}
```

### Tracing

By default the library uses `noop.Tracer`. If you want to change that you can use the OpenTelemetry or the OpenTracing implementations, respectively:

```go
// OpenTelemetry tracer
package main

import (
	"github.com/graph-gophers/graphql-go"
	"github.com/graph-gophers/graphql-go/example/starwars"
	otelgraphql "github.com/graph-gophers/graphql-go/trace/otel"
	"github.com/graph-gophers/graphql-go/trace/tracer"
)
// ...
_, err := graphql.ParseSchema(starwars.Schema, nil, graphql.Tracer(otelgraphql.DefaultTracer()))
// ...
```
Alternatively you can pass an existing trace.Tracer instance:
```go
tr := otel.Tracer("example")
_, err = graphql.ParseSchema(starwars.Schema, nil, graphql.Tracer(&otelgraphql.Tracer{Tracer: tr}))
```


```go
// OpenTracing tracer
package main

import (
	"github.com/graph-gophers/graphql-go"
	"github.com/graph-gophers/graphql-go/example/starwars"
	"github.com/graph-gophers/graphql-go/trace/opentracing"
	"github.com/graph-gophers/graphql-go/trace/tracer"
)
// ...
_, err := graphql.ParseSchema(starwars.Schema, nil, graphql.Tracer(opentracing.Tracer{}))

// ...
```

If you need to implement a custom tracer the library would accept any tracer which implements the interface below:
```go
type Tracer interface {
 TraceQuery(ctx context.Context, queryString string, operationName string, variables map[string]interface{}, varTypes map[string]*introspection.Type) (context.Context, func([]*errors.QueryError))
 TraceField(ctx context.Context, label, typeName, fieldName string, trivial bool, args map[string]interface{}) (context.Context, func(*errors.QueryError))
 TraceValidation(context.Context) func([]*errors.QueryError)
}
```


### [Examples](https://github.com/graph-gophers/graphql-go/wiki/Examples)

graphql

# graphql [![CircleCI](https://circleci.com/gh/graphql-go/graphql/tree/master.svg?style=svg)](https://circleci.com/gh/graphql-go/graphql/tree/master) [![Go Reference](https://pkg.go.dev/badge/github.com/graphql-go/graphql.svg)](https://pkg.go.dev/github.com/graphql-go/graphql) [![Coverage Status](https://coveralls.io/repos/github/graphql-go/graphql/badge.svg?branch=master)](https://coveralls.io/github/graphql-go/graphql?branch=master) [![Join the chat at https://gitter.im/graphql-go/graphql](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/graphql-go/graphql?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

An implementation of GraphQL in Go. Follows the official reference implementation [`graphql-js`](https://github.com/graphql/graphql-js).

Supports: queries, mutations & subscriptions.

### Documentation

godoc: https://pkg.go.dev/github.com/graphql-go/graphql

### Contribute Back

Friendly reminder links are available in case you would like to contribute back into our commitment with Go and open-source.

| Author        |  PayPal Link  |
|:-------------:|:-------------:|
| [Hafiz Ismail](https://github.com/sogko) | Not available yet. |
| [Chris Ramón](https://github.com/chris-ramon) | https://www.paypal.com/donate/?hosted_button_id=WHUQQYEMTRQBJ |

### Getting Started

To install the library, run:
```bash
go get github.com/graphql-go/graphql
```

The following is a simple example which defines a schema with a single `hello` string-type field and a `Resolve` method which returns the string `world`. A GraphQL query is performed against this schema with the resulting output printed in JSON format.

```go
package main

import (
	"encoding/json"
	"fmt"
	"log"

	"github.com/graphql-go/graphql"
)

func main() {
	// Schema
	fields := graphql.Fields{
		"hello": &graphql.Field{
			Type: graphql.String,
			Resolve: func(p graphql.ResolveParams) (interface{}, error) {
				return "world", nil
			},
		},
	}
	rootQuery := graphql.ObjectConfig{Name: "RootQuery", Fields: fields}
	schemaConfig := graphql.SchemaConfig{Query: graphql.NewObject(rootQuery)}
	schema, err := graphql.NewSchema(schemaConfig)
	if err != nil {
		log.Fatalf("failed to create new schema, error: %v", err)
	}

	// Query
	query := `
		{
			hello
		}
	`
	params := graphql.Params{Schema: schema, RequestString: query}
	r := graphql.Do(params)
	if len(r.Errors) > 0 {
		log.Fatalf("failed to execute graphql operation, errors: %+v", r.Errors)
	}
	rJSON, _ := json.Marshal(r)
	fmt.Printf("%s \n", rJSON) // {"data":{"hello":"world"}}
}
```
For more complex examples, refer to the [examples/](https://github.com/graphql-go/graphql/tree/master/examples/) directory and [graphql_test.go](https://github.com/graphql-go/graphql/blob/master/graphql_test.go).

### Third Party Libraries
| Name          | Author        | Description  |
|:-------------:|:-------------:|:------------:|
| [graphql-go-handler](https://github.com/graphql-go/graphql-go-handler) | [Hafiz Ismail](https://github.com/sogko) | Middleware to handle GraphQL queries through HTTP requests. |
| [graphql-relay-go](https://github.com/graphql-go/graphql-relay-go) | [Hafiz Ismail](https://github.com/sogko) | Lib to construct a graphql-go server supporting react-relay. |
| [golang-relay-starter-kit](https://github.com/sogko/golang-relay-starter-kit) | [Hafiz Ismail](https://github.com/sogko) | Barebones starting point for a Relay application with Golang GraphQL server. |
| [dataloader](https://github.com/nicksrandall/dataloader) | [Nick Randall](https://github.com/nicksrandall) | [DataLoader](https://github.com/facebook/dataloader) implementation in Go. |

### Blog Posts
- [Golang + GraphQL + Relay](https://wehavefaces.net/learn-golang-graphql-relay-1-e59ea174a902)



graphql-go

Apollos' "GraphQL over Websocket" client and server implementation

[![Go Report Card](https://goreportcard.com/badge/github.com/zaba505/gws)](https://goreportcard.com/report/github.com/zaba505/gws)
[![GoDoc](https://godoc.org/github.com/Zaba505/gws?status.svg)](https://pkg.go.dev/github.com/zaba505/gws?tab=doc)
[![build](https://github.com/Zaba505/gws/workflows/build/badge.svg)](https://github.com/Zaba505/gws/actions)
[![codecov](https://codecov.io/gh/Zaba505/gws/branch/master/graph/badge.svg)](https://codecov.io/gh/Zaba505/gws)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)

# gws
A WebSocket client and server for GraphQL.

`gws` implements Apollos' "GraphQL over WebSocket" protocol,
which can be found in PROTOCOL.md. It provides an RPC-like API for both
client and server usage.

## Getting Started

`gws` provides both a client and server implementation for the
"GraphQL over Websocket" protocol.

### Client

To create a client, you must first dial the server like:
```go
conn, err := gws.Dial(context.TODO(), "ws://example.com")
// Don't forget to handle the error
```

`gws.Dial` will return high-level abstraction around the
underlying connection. This is very similar to gRPCs'
behaviour. After dialing the server, now simply wrap the
return conn and begin performing queries, like:
```go
client := gws.NewClient(conn)

resp, err := client.Query(context.TODO(), &gws.Request{Query: "{ hello { world } }"})
// Don't forget to handle the error

var data struct{
  Hello struct{
    World string
  }
}
err = json.Unmarshal(resp.Data, &data)
```

### Server

The server implementation provided isn't actually a standalone
server but instead just a specially configured `http.Handler`.
This is because the underlying protocol is simply WebSocket,
which runs on top of HTTP. Creating a bare bones handler is
as simple as the following:
```go
msgHandler := func(s *Stream, req *Request) error {
  // Handle request
  return nil
}

http.ListenAndServe(":8080", gws.NewHandler(gws.HandlerFunc(msgHandler)))
```


A query library for retrieving part of JSON based on JSONPath syntax

# AsaiYusuke/JSONPath

[![Test](https://github.com/AsaiYusuke/jsonpath/actions/workflows/test.yml/badge.svg)](https://github.com/AsaiYusuke/jsonpath/actions/workflows/test.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/AsaiYusuke/jsonpath/v2)](https://goreportcard.com/report/github.com/AsaiYusuke/jsonpath/v2)
[![Coverage Status](https://coveralls.io/repos/github/AsaiYusuke/jsonpath/badge.svg?branch=main)](https://coveralls.io/github/AsaiYusuke/jsonpath?branch=main)
[![Go Reference](https://pkg.go.dev/badge/github.com/AsaiYusuke/jsonpath/v2.svg)](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2)
[![Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go#query-language)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

![AsaiYusuke/JSONPath](assets/logo.svg)

This [Go](https://golang.org/) library allows you to extract parts of a JSON object using the _JSONPath_ query syntax.

The core JSONPath syntax supported by this library is based on:

- [Stefan Gössner's JSONPath - XPath for JSON](https://goessner.net/articles/JsonPath/)
- [Christoph Burgmer's json-path-comparison](https://github.com/cburgmer/json-path-comparison)
- [JSONPath Internet Draft Development](https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-jsonpath)

## Note

For details on syntax compatibility with other libraries, see [:memo: my comparison results](https://asaiyusuke.github.io/jsonpath/cburgmer-json-path-comparison/docs/index.html).

## Table of Contents

- [Getting started](#getting-started)
- [Basic design](#basic-design)
- [How to use](#how-to-use)
 - [Retrieve one-time or repeated](#-retrieve-one-time-or-repeatedly)
 - [Error handling](#-error-handling)
 - [Function syntax](#-function-syntax)
 - [Accessing JSON](#-accessing-json)
- [Differences](#differences)
- [Benchmarks](#benchmarks)
- [Project progress](#project-progress)

## Getting started

### Install

```bash
go get github.com/AsaiYusuke/jsonpath/v2
```

### Simple example

```go
package main

import (
 "encoding/json"
 "fmt"

 "github.com/AsaiYusuke/jsonpath/v2"
)

func main() {
 jsonPath, srcJSON := `$.key`, `{"key":"value"}`
 var src any
 json.Unmarshal([]byte(srcJSON), &src)
 output, _ := jsonpath.Retrieve(jsonPath, src)
 outputJSON, _ := json.Marshal(output)
 fmt.Println(string(outputJSON))
 // Output:
 // ["value"]
}
```

## Basic design

### _Streamlined Development_

- The JSONPath syntax parser is implemented using [PEG](https://github.com/pointlander/peg), which helps keep the source code simple and maintainable.
- Robust unit tests are provided to prevent bugs and ensure consistent results.

### _User-Friendly Interface_

- The library provides comprehensive error types, making it easy for users to handle errors appropriately.

### _High Compatibility_

- The library incorporates consensus behaviors from [Christoph Burgmer's json-path-comparison](https://github.com/cburgmer/json-path-comparison), ensuring high compatibility with other implementations.

## How to use

### \* Retrieve one-time or repeatedly

The `Retrieve` function extracts values from a JSON object using a JSONPath expression:

```go
output, err := jsonpath.Retrieve(jsonPath, src)
```

[:memo: Example](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2#example-Retrieve)

The `Parse` function returns a parser function that checks the JSONPath syntax in advance. You can use this parser function to repeatedly extract values with the same JSONPath:

```go
parsed, err := jsonpath.Parse(jsonPath)
output1, err1 := parsed(src1)
output2, err2 := parsed(src2)
```

[:memo: Example](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2#example-Parse)

parser function

- The parser function accepts an optional second argument: a pointer to a `[]any` buffer.
- When omitted or `nil`, a new slice is allocated on each call and returned.
- When provided (non-nil), the buffer will be reset to length 0 and filled directly, enabling reuse without extra allocations.

Example of buffer reuse:

```go
parsed, _ := jsonpath.Parse("$[*]")
buf := make([]any, 0, 4)
args := []*[]any{&buf}
out1, _ := parsed([]any{1}, args...) // writes into buf -> [1]
out2, _ := parsed([]any{2}, args...) // reuses the same buf -> now [2]
fmt.Println(out1)
fmt.Println(out2)
fmt.Println(buf)
 // Output:
 // [2]
 // [2]
 // [2]
```

[:memo: Example](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2#example-Parse-reuseBuffer)

Note: Do not share the same buffer across goroutines concurrently.

### \* Error handling

If an error occurs during API execution, a specific error type is returned. The following error types help you identify the cause:

#### Syntax errors from `Retrieve` or `Parse`

| Error type | Message format | Symptom | Ex |
| ----------------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| `ErrorInvalidSyntax` | `invalid syntax (position=%d, reason=%s, near=%s)` | The JSONPath contains invalid syntax. The _reason_ in the message provides more details. | [:memo:](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/errors#example-ErrorInvalidSyntax) |
| `ErrorInvalidArgument` | `invalid argument (argument=%s, error=%s)` | An argument in the JSONPath is invalid according to Go syntax. | [:memo:](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/errors#example-ErrorInvalidArgument) |
| `ErrorFunctionNotFound` | `function not found (path=%s)` | The specified function in the JSONPath was not found. | [:memo:](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/errors#example-ErrorFunctionNotFound) |
| `ErrorNotSupported` | `not supported (path=%s, feature=%s)` | The JSONPath uses unsupported syntax. | [:memo:](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/errors#example-ErrorNotSupported) |

#### Runtime errors from `Retrieve` or parser functions

| Error type | Message format | Symptom | Ex |
| --------------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| `ErrorMemberNotExist` | `member did not exist (path=%s)` | The specified object or array member does not exist in the JSON object. | [:memo:](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/errors#example-ErrorMemberNotExist) |
| `ErrorTypeUnmatched` | `type unmatched (path=%s, expected=%s, found=%s)` | The type of the node in the JSON object does not match what is expected by the JSONPath. | [:memo:](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/errors#example-ErrorTypeUnmatched) |
| `ErrorFunctionFailed` | `function failed (path=%s, error=%s)` | The function specified in the JSONPath failed to execute. | [:memo:](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/errors#example-ErrorFunctionFailed) |

Type checking makes it easy to determine which error occurred.

```go
import jsonpath "github.com/AsaiYusuke/jsonpath/v2"
import errors "github.com/AsaiYusuke/jsonpath/v2/errors"

_, err := jsonpath.Retrieve(jsonPath, srcJSON)
switch err.(type) {
case errors.ErrorMemberNotExist:
 fmt.Printf("retry with other srcJSON: %v", err)
 // handle or continue
case errors.ErrorInvalidArgument:
 return nil, fmt.Errorf("specified invalid argument: %v", err)
}
```

### \* Function syntax

You can use user-defined functions to format results. The function syntax is appended after the JSONPath expression.

There are two types of functions:

#### Filter function

A filter function applies a user-defined function to each value in the result, transforming them individually.

[:memo: Example](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/config#example-Config.SetFilterFunction)

#### Aggregate function

An aggregate function combines all values in the result into a single value.

[:memo: Example](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/config#example-Config.SetAggregateFunction)

### \* Accessing JSON

Instead of retrieving values directly, you can obtain accessors (_Getters_ / _Setters_) for the input JSON. These accessors allow you to update the original JSON object.

Enable this feature by calling `Config.SetAccessorMode()`.

[:memo: Example](https://pkg.go.dev/github.com/AsaiYusuke/jsonpath/v2/config#example-Accessor)

#### Accessor limitations

Setters are not available for some results, such as when using function syntax in the JSONPath.

Accessor operations follow Go's map/slice semantics. If you modify the structure of the JSON, be aware that accessors may not behave as expected. To avoid issues, obtain a new accessor each time you change the structure.

## Differences

Some behaviors in this library differ from the consensus of other implementations.
For a full comparison, see [:memo: this result](https://asaiyusuke.github.io/jsonpath/cburgmer-json-path-comparison/docs/index.html).

These behaviors may change in the future if more appropriate approaches are found.

### Character types

The following character types are allowed for identifiers in dot-child notation:

| Character type | Available | Escape required |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | --------------- |
| \* Numbers and alphabets (`0-9` `A-Z` `a-z`)| Yes | No |
| \* Hyphen and underscore (`-` `_`)| Yes | No |
| \* Non-ASCII Unicode characters (`0x80 - 0x10FFFF`) | Yes | No |
| \* Other printable symbols (`Space` `!` `"` `#` `$` `%` `&` `'` `(` `)` `*` `+` `,` `.` `/` `:` `;` `<` `=` `>` `?` `@` `[` `\` `]` `^` `` ` `` `{` `\|` `}` `~`) | Yes | Yes |
| \* ~~Control code characters~~ (`0x00 - 0x1F`, `0x7F`) | No | - |

Printable symbols (except hyphen and underscore) can be used by escaping them.

```text
JSONPath : $.abc\.def
srcJSON : {"abc.def":1}
Output : [1]
```

### Wildcard in qualifier

Wildcards in qualifiers can be specified as a union with subscripts.

```text
JSONPath : $[0,1:3,*]
srcJSON : [0,1,2,3,4,5]
Output : [0,1,2,0,1,2,3,4,5]
```

### Regular expression

Regular expression syntax follows Go's regular expression rules.
In particular, you can use "(?i)" to make the regular expression case-insensitive.

```text
JSONPath : $[?(@=~/(?i)CASE/)]
srcJSON : ["Case","Hello"]
Output : ["Case"]
```

### JSONPaths in the filter-qualifier

JSONPaths that return a value group cannot be used with a `comparator` or `regular expression`. However, you can use these syntaxes for `existence checks`.

| JSONPaths that return a value group | example |
| ----------------------------------- | ------------- |
| Recursive descent | `@..a` |
| Multiple identifier | `@['a','b']` |
| Wildcard identifier | `@.*` |
| Slice qualifier | `@[0:1]` |
| Wildcard qualifier | `@[*]` |
| Union in the qualifier | `@[0,1]` |
| Filter qualifier | `@.a[?(@.b)]` |

- comparator example (error)

```text
JSONPath : $[?(@..x == "hello world")]
srcJSON : [{"a":1},{"b":{"x":"hello world"}}]
Error : ErrorInvalidSyntax
```

- regular expression example (error)

```text
JSONPath : $[?(@..x=~/hello/)]
srcJSON : [{"a":1},{"b":{"x":"hello world"}}]
Error : ErrorInvalidSyntax
```

- existence check example

```text
JSONPath : $[?(@..x)]
srcJSON : [{"a":1},{"b":{"x":"hello world"}}]
Output : [{"b":{"x":"hello world"}}]
```

If a JSONPath filter begins with the Root, it performs a whole-match operation if any match is found.

```text
JSONPath : $[?($..x)]
srcJSON : [{"a":1},{"b":{"x":"hello world"}}]
Output : [{"a":1},{"b":{"x":"hello world"}}]
```

## Benchmarks

Benchmark results for various Go JSONPath libraries (measured by myself) are available in the following repository:

- [JSONPath Benchmark](https://github.com/AsaiYusuke/jsonpath-benchmark)

## Project progress

- Syntax
 - Identifier
 - [x] identifier in dot notation
 - [x] identifier in bracket notation
 - [x] wildcard
 - [x] multiple identifiers in brackets
 - [x] recursive retrieval
 - Qualifier
 - [x] index
 - [x] slice
 - [x] wildcard
 - Filter
 - [x] logical operations
 - [x] comparators
 - [x] JSONPath retrieval in filter
 - [ ] script
 - Function
 - [x] filter
 - [x] aggregate
 - [x] Refer to the consensus behaviors
- Architecture
 - [x] PEG syntax analysis
 - [x] Error handling
 - [x] Function support
 - [x] JSON accessors
- Go language manner
 - [x] retrieve with an object unmarshaled to interface
 - [x] retrieve with the json.Number type
- Source code
 - [x] Release version
 - Unit tests
 - [x] syntax tests
 - [x] benchmarks
 - [x] coverage >80%
 - [x] Examples
 - [x] CI automation
 - Documentation
 - [x] README
 - [ ] API documentation
 - [x] comparison results (local)
- Development status
 - [x] requirements and functional design
 - [x] Decided to follow a standard or reference implementation for JSONPath syntax
 - [x] design-based coding
 - [ ] testing
 - [ ] documentation
- Future ToDo
 - Go language affinity
 - [ ] retrieve with an object unmarshaled to struct
 - [ ] retrieve with struct tags
 - [ ] retrieve with user-defined objects

jsonpath

[![Build Status](https://travis-ci.org/bhmj/jsonslice.svg?branch=master)](https://travis-ci.org/bhmj/jsonslice)
[![Go Report Card](https://goreportcard.com/badge/github.com/bhmj/jsonslice)](https://goreportcard.com/report/github.com/bhmj/jsonslice)
[![GoDoc](https://godoc.org/github.com/bhmj/jsonslice?status.svg)](https://godoc.org/github.com/bhmj/jsonslice)

# JSON Slice

## What is it?

JSON Slice is a Go package which allows to execute fast jsonpath queries without unmarshalling the whole data. 

Sometimes you need to get a single value from incoming json using jsonpath, for example to route data accordingly or so. To do that you must unmarshall the whole data into interface{} struct and then apply some jsonpath library to it, only to get just a tiny little value. What a waste of resourses! Well, now there's `jsonslice`.

Simply call `jsonslice.Get` on your raw json data to slice out just the part you need. The `[]byte` received can then be unmarshalled into a struct or used as it is.

## Getting started

#### 1. install

```
$ go get github.com/bhmj/jsonslice
```

#### 2. use it

```golang
import "github.com/bhmj/jsonslice"
import "fmt"

func main() {
 var data = []byte(`
 { "sku": [ 
 { "id": 1, "name": "Bicycle", "price": 160, "extras": [ "flashlight", "pump" ] },
 { "id": 2, "name": "Scooter", "price": 280, "extras": [ "helmet", "gloves", "spare wheel" ] }
 ]
 } `)

 a, _ := jsonslice.Get(data, "$.sku[0].price")
 b, _ := jsonslice.Get(data, "$.sku[1].extras.count()")
 c, _ := jsonslice.Get(data, "$.sku[?(@.price > 200)].name")
 d, _ := jsonslice.Get(data, "$.sku[?(@.extras.count() < 3)].name")

 fmt.Println(string(a)) // 160
 fmt.Println(string(b)) // 3
 fmt.Println(string(c)) // ["Scooter"]
 fmt.Println(string(d)) // ["Bicycle"]
}
```
[Run in Go Playground](https://play.golang.org/p/fYv-Y12akvs)

## Package functions
 
`jsonslice.Get(data []byte, jsonpath string) ([]byte, error)` 
 - get a slice from raw json data specified by jsonpath

## Specs and references

* Originally based on [Stefan Gössner's article](http://goessner.net/articles/JsonPath/index.html#e2).
* Expression evaluator designed to comply with [ECMAScript Language: Expressions reference](https://tc39.es/ecma262/multipage/ecmascript-language-expressions.html).
* A proud member of amazing [Christoph Burgmer's json-path-comparison](https://github.com/cburgmer/json-path-comparison).

## Syntax features

1. Classic dot notation (`$.simple_key`) is limited to alphanumeric characters. For more complex cases use `$['complex key!']` or `$.'complex key!'`. 

2. A single index reference returns an element, not an array; a slice reference always returns array: 
```
$ echo '[{"name":"abc"}, {"name":"def"}]' | ./build/jsonslice '$[0].name' 
"abc"
```
```
$ echo '[{"name":"abc"}, {"name":"def"}]' | .build//jsonslice '$[0:1].name'
["abc"]
```

3. Indexing or slicing on root node is supported (assuming json is an array and not an object): 
```
$ cat example/sample1.json | ./build/jsonslice '$[0].author'
```

## Expressions

### Overview 
```
 $ -- root node (can be either object or array)
 $.node -- dot-notated child
 $.node.deeper -- dot-notated child (chain of them)
 $.'some node' -- dot-notated child (syntax extension)
 ['node'] -- bracket-notated child
 ['foo','bar'] -- bracket-notated children (aggregation)
 [5] -- array index
 [-5] -- negative index means "from the end"
 [1:9] -- array slice
 [1:9:2] -- array slice (+step)
 .* .[*] .[:] -- wildcard
 ..key -- deepscan
 .'\'' -- escape sequences supported (\", \', \n, \r, \t, \0, \\)
 .'\x0A' -- escaped hex bytes supported
 .'\u00F6' -- escaped 16-bit unicode codepoints supported
 .'\U000000F6' -- escaped 32-bit unicode codepoints supported
```
### Functions
```
 $.obj.length() -- number of elements in an array or string length, depending on the obj type
 $.obj.count() -- same as above
 $.val.size() -- value size in bytes (as is)
```
### Slices
```
 $.arr[start:end:step]
 $.arr[start:end]
```
Selects elements from `start` (inclusive) to `end` (exclusive), stepping by `step`. If `step` is omitted or zero, then 1 is implied. Out-of-bounds values are reduced to the nearest bound.

If `step` is positive:
 - empty `start` treated as the first element inclusive
 - empty `end` treated as the last element inclusive
 - `start` should be less then `end`, otherwise result will be empty

If `step` is negative:
 - empty `start` treated as last element inclusive
 - empty `end` treated as the first element inclusive
 - `start` should be greater then `end`, otherwise result will be empty

### Filters

```
 [?(<expression>)] -- filter expression. Applicable to arrays only
 @ -- the root of the current element of the array. Used only within a filter.
 @.val -- a field of the current element of the array.
```

#### Filter operators

 Operator | Description
 --- | ---
 math | `+` `-` `*` `/` `%` `**`
 `===` | Strict equality (mimics JavaScript). Examples: `true===true, 42===42`
 `==` | Abstract equality (mimics JavaScript). Examples: `true=="1", 42=="42"`. Use single or double quotes for string expressions. `[?(@.color=='red')]` or `[?(@.color=="red")]`
 `!=` | Abstract not equal to `[?(@.author != "Herman Melville")]`
 `!==` | Strict not equal to `?(@.tag !== "1")`
 `>` | Greater than `[?(@.price > 10)]`
 `>=` | Greater than or equal to
 `<` | Less than
 `<=` | Less than or equal to
 `=~` | Match a regexp `[?(@.name =~ /sword.*/i]`
 `!~` or `!=~` | Don't match a regexp `[?(@.name !~ /sword.*/i]`
 `&&` | Logical AND `[?(@.price < 10 && @isbn)]`
 `\|\|` | Logical OR `[?(@.price > 10 \|\| @.category == 'reference')]`
 `!` | Logical NOT `[?(!@.is_expensive)]`
 `\|` | Bitwise OR `[?(@.bits\|@.pieces > 0)]`
 `&` | Bitwise AND `[?(@.bits & 7 == 1)]`
 `^` | Bitwise XOR `[?(@.bits ^ 1 > 0)]`
 `~` | Bitwise NOT `[?(~@.bits == 0xF0)]`
 `<<` | Bitwise left shift `[?(@.bits << 1 == 2)]`
 `>>` | Bitwise right shift `[?(@.bits >> 1 == 0)]`

#### Comparison details
Comparison mostly complies with JavaScript specifications, see [Testing and Comparison Operations](https://tc39.es/ecma262/multipage/abstract-operations.html#sec-testing-and-comparison-operations). 
If you encounter wrong or inconsistent comparison behaviour please let me know by creating an issue in this repository.

## Examples

Assuming `sample0.json` and `sample1.json` in the example directory: 

 `cat example/sample0.json | ./build/jsonslice '$.store.book[0]'` 
 `cat example/sample0.json | ./build/jsonslice '$.store.book[0].title'` 
 `cat example/sample0.json | ./build/jsonslice '$.store.book[0:-1]'` 
 `cat example/sample1.json | ./build/jsonslice '$[1].author'` 
 `cat example/sample0.json | ./build/jsonslice '$.store.book[?(@.price > 10)]'` 
 `cat example/sample0.json | ./build/jsonslice '$.store.book[?(@.price > $.expensive)]'` 

Much more examples can be found in `jsonslice_test.go` 

## Benchmarks (Core i9-9880H)

```diff
$ go test -bench=. -benchmem -benchtime=4s
goos: darwin
goarch: amd64
pkg: github.com/bhmj/jsonslice
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
++ standard json.Unmarshal (for reference):
Benchmark_Unmarshal-16 398998 11268 ns/op 4272 B/op 130 allocs/op
++ and here's a jsonslice.Get:
Benchmark_Jsonslice_Get_Simple-16 1660604 2885 ns/op 24 B/op 1 allocs/op
++ parsing is the first part of jsonslicing, here it is separately:
Benchmark_JsonSlice_ParsePath-16 11955015 400 ns/op 0 B/op 0 allocs/op
++ aggregating non-contiguous elements may take a bit longer (extra mallocs involved):
Benchmark_Jsonslice_Get_Aggregated-16 1000000 4335 ns/op 313 B/op 10 allocs/op
++ standard json.Unmarshal of a large json (10 Mb):
Benchmark_Unmarshal_10Mb-16 100 40787414 ns/op 224 B/op 5 allocs/op
++ jsonslicing the same json, target element is at the start:
Benchmark_Jsonslice_Get_10Mb_First-16 3459492 1370 ns/op 24 B/op 1 allocs/op
++ jsonslicing the same json, target element is at the end: still beats Unmarshal
Benchmark_Jsonslice_Get_10Mb_Last-16 133 35731931 ns/op 54 B/op 1 allocs/op
PASS
ok github.com/bhmj/jsonslice 52.452s
```

## Changelog

**1.1.3** (2024-07-29) -- bugfix: space after closing `}` of a target object.

**1.1.2** (2022-01-02) -- Unicode support added. Expression parser upgrade to v0.9.1 
Bugfix: indexing of array element inside expression (`@[2]`). 
Bugfix: ecaped backslash in node key now works (`$.["\\"]`). 
See `Test_Fixes` function for bugfix details.

**1.1.1** (2021-10-20) -- Expression parser upgrade to v0.9.0

**1.1.0** (2021-11-12) -- Expression parser/evaluator has been separated to [different project](https://github.com/bhmj/xpression/) and completely rewritten. Parentheses now fully supported. Exponentiation operator added (`**`). Bitwise operators added (`|`, `&`, `^`, `<<`, `>>`). All expression calculations are JavaScript-compliant.

**1.0.6** (2021-10-31) -- JS-like behaviour on string/number/boolean values comparison. `===` operator added for strict comparison. Strings are now comparable.

**1.0.5** (2020-09-22) -- bugfix: `$..many.keys` used to trigger on `many` without recursing deeper on `keys`.

**1.0.4** (2020-05-07) -- bugfix: `$*` path generated panic.

**1.0.3** (2019-12-24) -- bugfix: `$[0].foo` `[{"foo":"\\"}]` generated "unexpected end of input".

**1.0.2** (2019-12-07) -- nested aggregation (`$[:].['a','b']`) now works as expected. TODO: add option to switch nested aggregation mode at runtime!

**1.0.1** (2019-12-01) -- "not equal" regexp operator added (`!=~` or `!~`).

**1.0.0** (2019-11-29) -- deepscan operator (`..`) added, slice with step `$[1:9:2]` is now supported, syntax extensions added. `GetArrayElements()` removed.

**0.7.6** (2019-09-11) -- bugfix: escaped backslash at the end of a string value.

**0.7.5** (2019-05-21) -- Functions `count()`, `size()`, `length()` work in filters.
> `$.store.bicycle.equipment[?(@.count() == 2)]` -> `[["light saber", "apparel"]]` 

**0.7.4** (2019-03-01) -- Mallocs reduced (see Benchmarks section).

**0.7.3** (2019-02-27) -- `GetArrayElements()` added.

**0.7.2** (2018-12-25) -- bugfix: closing square bracket inside a string value.

**0.7.1** (2018-10-16) -- bracket notation is now supported.
> `$.store.book[:]['price','title']` -> `[[8.95,"Sayings of the Century"],[12.99,"Sword of Honour"],[8.99,"Moby Dick"],[22.99,"The Lord of the Rings"]]`

**0.7.0** (2018-07-23) -- Wildcard key (`*`) added.
> `$.store.book[-1].*` -> `["fiction","J. R. R. Tolkien","The Lord of the Rings","0-395-19395-8",22.99]` 
> `$.store.*[:].price` -> `[8.95,12.99,8.99,22.99]`

**0.6.3** (2018-07-16) -- Boolean/null value error fixed.

**0.6.2** (2018-07-03) -- More tests added, error handling clarified.

**0.6.1** (2018-06-26) -- Nested array indexing is now supported.
> `$.store.bicycle.equipment[1][0]` -> `"peg leg"`

**0.6.0** (2018-06-25) -- Regular expressions added.
> `$.store.book[?(@.title =~ /(dick)|(lord)/i)].title` -> `["Moby Dick","The Lord of the Rings"]`

**0.5.1** (2018-06-15) -- Logical expressions added.
> `$.store.book[?(@.price > $.expensive && @.isbn)].title` -> `["The Lord of the Rings"]`

**0.5.0** (2018-06-14) -- Expressions (aka filters) added.
> `$.store.book[?(@.price > $.expensive)].title` -> `["Sword of Honour","The Lord of the Rings"]`

**0.4.0** (2018-05-16) -- Aggregating sub-queries added.
> `$.store.book[1:3].author` -> `["John","William"]`

**0.3.0** (2018-05-05) -- MVP.

## Roadmap

- [x] length(), count(), size() functions
- [x] filters: simple expressions
- [x] filters: complex expressions (with logical operators)
- [x] nested arrays support
- [x] wildcard operator (`*`)
- [x] bracket notation for multiple field queries (`$['a','b']`)
- [x] deepscan operator (`..`)
- [x] syntax extensions: `$.'keys with spaces'.price`
- [x] flexible syntax: `$[0]` works on both `[1,2,3]` and `{"0":"abc"}`
- [x] JavaScript-compatible expressions
- [x] Unicode support
- [ ] IN (), NOT IN ()
- [ ] cache parsed jsonpaths of used variables at filterMatch.varFunc(str)
- [ ] Optionally unmarshal the result
- [ ] Option to select aggregation mode (nested or plain)

## Contributing

1. Fork it!
2. Create your feature branch: `git checkout -b my-new-feature`
3. Commit your changes: `git commit -am 'Add some feature'`
4. Push to the branch: `git push origin my-new-feature`
5. Submit a pull request :)

## Licence

[MIT](http://opensource.org/licenses/MIT)

## Author

Michael Gurov aka BHMJ

jsonslice

Model Query Language (mql) is a query language for your database models

# MQL
[![Go
Reference](https://pkg.go.dev/badge/github.com/hashicorp/mql/mql.svg)](https://pkg.go.dev/github.com/hashicorp/mql)
[![Go Report
Card](https://goreportcard.com/badge/github.com/hashicorp/mql)](https://goreportcard.com/report/github.com/hashicorp/mql)
[![Go Coverage](https://raw.githack.com/hashicorp/mql/main/coverage/coverage.svg)](https://raw.githack.com/hashicorp/mql/main/coverage/coverage.html)

The [mql](https://pkg.go.dev/github.com/hashicorp/mql) (Model Query Language) Go package provides a language that end users can use to query your
database models, without them having to learn SQL or exposing your
application to SQL injection.

## Examples

### [github.com/go-gorm/gorm](https://github.com/go-gorm/gorm)

```Go
w, err := mql.Parse(`name="alice" or name="bob"`,User{})
if err != nil {
 return nil, err
}
err = db.Where(w.Condition, w.Args...).Find(&users).Error
```

### [database/sql](https://pkg.go.dev/database/sql)

```Go
w, err := mql.Parse(`name="alice" or name="bob"`,User{}, mql.WithPgPlaceholders())
if err != nil {
 return nil, err
}
q := fmt.Sprintf("select * from users where %s", w.Condition)
rows, err := db.Query(q, w.Args...)
```

### [github.com/hashicorp/go-dbw](https://github.com/hashicorp/go-dbw)

```Go
w, err := mql.Parse(`name="alice" or name="bob")`,User{})
if err != nil {
 return nil, err
}
err := rw.SearchWhere(ctx, &users, w.Condition, w.Args)
```

## Some bits about usage

First, you define a model you wish to query as a Go `struct` and then provide a [mql]((https://pkg.go.dev/badge/github.com/hashicorp/mql/mql.svg))
query. The package then uses the query along with a model to generate a
parameterized SQL where clause.

Fields in your model can be compared with the following operators: `=`, `!=`,
`>=`, `<=`, `<`, `>`, `%` .

Strings must be quoted. Double quotes `"`, single quotes `'` or backticks ``
` `` can be used as delimiters. Users can choose whichever supported delimiter
makes it easier to quote their string.

Comparison operators can have optional leading/trailing whitespace.

The `%` operator allows you to do partial string matching using LIKE "%value%". This
matching is case insensitive.

The `=` equality operator is case insensitive when used with string fields.

Comparisons can be combined using: `and`, `or`.

More complex queries can be created using parentheses.

See [GRAMMAR.md](./GRAMMAR.md) for a more complete documentation of [mql](https://pkg.go.dev/github.com/hashicorp/mql)'s grammar.

Example query:

`name="alice" and age > 11 and (region % 'Boston' or region="south shore")`

### Date/Time fields

If your model contains a time.Time field, then we'll append `::date` to the
column name when generating a where clause and the comparison value must be in
an `ISO-8601` format.

Note: It's possible to compare date-time fields down to the
millisecond using `::date` and a literal in `ISO-8601` format.

Currently, this is the only supported way to compare
dates, if you need something different then you'll need to provide your own
custom validator/converter via
[WithConverter(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithConverter)
when calling
[mql.Parse(...)](https://pkg.go.dev/github.com/hashicorp/mql#Parse).

We provide default validation+conversion of fields in a model when parsing
and generating a
[WhereClause](https://pkg.go.dev/github.com/hashicorp/mql#WhereClause). You can
provide optional validation+conversion functions for fields in your model via
[WithConverter(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithConverter).

Example date comparison down to the HH::MM using an ISO-8601 format:

`name="alice" and created_at>"2023-12-01 14:01"`

Note: Expressions with the same level of precedence are evaluated right to left.
Example:
`name="alice" and age > 11 and region =
"Boston"` is evaluated as: `name="alice" and (age > 11 and region =
"Boston")`



### Mapping field names

You can also provide an optional map from query column identifiers to model
field names via
[WithColumnMap(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithColumnMap)
if needed.

Example
[WithColumnMap(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithColumnMap)
usage:

``` Go
type User struct {
 FullName string
}

// map the column alice to field name FullName
columnMap := map[string]string{
 "name": "FullName",
}

w, err := mql.Parse(
 `name="alice"`,
 User{},
 mql.WithColumnMap(columnMap))

if err != nil {
 return nil, err
}
```

### Mapping via struct tags

You can use struct tags to map model fields to column names by using
[WithColumnFieldTag(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithColumnFieldTag).
This allows you to define the mapping in your struct definition rather than at query time.

Example
[WithColumnFieldTag(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithColumnFieldTag)
usage:

``` Go
type User struct {
 Name string `db:"full_name"`
}

w, err := mql.Parse(
 `Name="alice"`,
 User{},
 mql.WithColumnFieldTag("db"))

if err != nil {
 return nil, err
}

fmt.Print(w.Condition) // prints full_name=?
```

### Mapping output column names

You can also provide an optional map from model field names to output column
names via
[WithTableColumnMap(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithTableColumnMap)
if needed.

Example
[WithTableColumnMap(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithTableColumnMap)
usage:

``` Go
type User struct {
 FullName string
}

// map the field name FullName to column "u.fullname"
tableColumnMap := map[string]string{
 "fullname": "u.fullname",
}

w, err := mql.Parse(
 `FullName="alice"`,
 User{},
 mql.WithTableColumnMap(tableColumnMap))

if err != nil {
 return nil, err
}

fmt.Print(w.Condition) // prints u.fullname=?
```

### Ignoring fields

If your model (Go struct) has fields you don't want users searching then you can
optionally provide a list of columns to be ignored via [WithIgnoreFields(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithIgnoreFields)

Example
[WithIgnoreFields(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithIgnoreFields)
usage:

```Go
type User {
 Name string
 CreatedAt time.Time
 UpdatedAt time.Time
}

// you want to keep users from using queries that include the user fields
// of: created_at updated_at
w, err := mql.Parse(
 `name="alice"`,
 User{},
 mql.WithIgnoreFields("CreatedAt", "UpdatedAt"))

if err != nil {
 return nil, err
}
```

### Custom converters/validators

Sometimes the default out-of-the-box bits doesn't fit your needs. If you need to
override how expressions (column name, operator and value) is converted and
validated during the generation of a
[WhereClause](https://pkg.go.dev/github.com/hashicorp/mql#WhereClause), then
you can optionally
provide your own validator/convertor via
[WithConverter(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithConverter)

Example
[WithConverter(...)](https://pkg.go.dev/github.com/hashicorp/mql#WithConverter)
usage:

``` Go
// define a converter for mySQL dates
mySQLDateConverter := func(columnName string, comparisonOp mql.ComparisonOp, value *string) (*mql.WhereClause, error) {
 // you should add some validation of function parameters here.
 return &mql.WhereClause{
 Condition: fmt.Sprintf("%s%sSTR_TO_DATE(?)", columnName, comparisonOp),
 Args: []any{*value},
 }, nil
}

w, err := mql.Parse(
 `name="alice" and created_at > "2023-06-18"`,
 User{},
 mql.WithConverter("CreatedAt", mySqlDateConverter))

if err != nil {
 return nil, err
}

```

### Grammar

See: [GRAMMAR.md](./GRAMMAR.md)


## Security

**Please note**: We take security and our users' trust very seriously. If you
believe you have found a security issue, please *[responsibly
disclose](https://www.hashicorp.com/security#vulnerability-reporting)* by
contacting us at security@hashicorp.com.
## Contributing

Thank you for your interest in contributing! Please refer to
[CONTRIBUTING.md](https://github.com/hashicorp/mql/blob/main/CONTRIBUTING.md)
for guidance.

<img src="assets/logo.png" height="100" border="0" alt="RQL">
	 
	<a href="https://godoc.org/github.com/a8m/rql">
		<img src="https://img.shields.io/badge/api-reference-blue.svg?style=flat-square" alt="GoDoc">
	</a>
	<a href="LICENSE">
		<img src="https://img.shields.io/badge/license-MIT-blue.svg?style=flat-square" alt="LICENSE">
	</a>
	<a href="https://app.circleci.com/pipelines/github/a8m/rql?branch=master">
		<img src="https://img.shields.io/circleci/build/github/a8m/rql?style=flat-square" alt="Build Status">
	</a>


RQL is a resource query language for REST. It provides a simple and light-weight API for adding dynamic querying capabilities to web-applications that use SQL-based database. It functions as the connector between the HTTP handler and the DB engine, and manages all validations and translations for user inputs.


 <img src="assets/diagram.png" alt="rql diagram">


## Motivation
In the last several years I have found myself working on different web applications in Go, some of them were small and some of them were big with a lot of entities and relations. In all cases I never found a simple and standard API to query my resources.

What do I mean by query? Let's say our application has a table of `orders`, and we want our users to be able to search and __filter by dynamic parameters__. For example: _select all orders from today with price greater than 100_. 
In order to achieve that I used to pass these parameters in the query string like this: `created_at_gt=X&price_gt=100`. 
But sometimes it became complicated when I needed to apply a disjunction between two conditions. For example, when I wanted to _select all orders that canceled or created last week and still didn't ship_. And in SQL syntax:
```sql
SELECT * FROM ORDER WHERE canceled = 1 OR (created_at < X AND created_at > Y AND shipped = 0)
``` 
I was familiar with the MongoDB syntax and I felt that it was simple and robust enough to achieve my goals, and decided to use it as the query language for this project. I wanted it to be project agnostic in the sense of not relying on anything that related to some specific application or resource. Therefore, in order to embed rql in a new project, a user just needs to import the package and add the desired tags to his struct definition. Follow the [Getting Started](#Getting_Started) section to learn more.

## Getting Started
rql uses a subset of MongoDB query syntax. If you are familiar with the MongoDB syntax, it will be easy for you to start. Although, it's pretty simple and easy to learn. 
In order to embed rql you simply need to add the tags you want (`filter` or `sort`) to your struct definition, and rql will manage all validations for you and return an informative error for the end user if the query doesn't follow the schema.
Here's a short example of how to start using rql quickly, or you can go to [API](#API) for more expanded documentation.
```go
// An example of an HTTP handler that uses gorm, and accepts user query in either the body
// or the URL query string.
package main

var (
	db *gorm.DB
	// QueryParam is the name of the query string key.
	QueryParam = "query"
	// MustNewParser panics if the configuration is invalid.
	QueryParser = rql.MustNewParser(rql.Config{
		Model: User{},
		FieldSep: ".",
	})
)

// User is the model in gorm's terminology.
type User struct {
	ID uint `gorm:"primary_key" rql:"filter,sort"`
	Admin bool `rql:"filter"`
	Name string `rql:"filter"`
	AddressName string `rql:"filter"`
	CreatedAt time.Time `rql:"filter,sort"`
}


// GetUsers is an http.Handler that accepts a db query in either the body or the query string.
func GetUsers(w http.ResponseWriter, r *http.Request) {
	var users []User
	p, err := getDBQuery(r)
	if err != nil {
		io.WriteString(w, err.Error())
		w.WriteHeader(http.StatusBadRequest)
		return
	}
	err = db.Where(p.FilterExp, p.FilterArgs).
		Offset(p.Offset).
		Limit(p.Limit).
		Order(p.Sort).
		Find(&users).Error
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}
	if err := json.NewEncoder(w).Encode(users); err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}
	w.Header().Set("Content-Type", "application/json")
}

// getDBQuery extract the query blob from either the body or the query string
// and execute the parser.
func getDBQuery(r *http.Request) (*rql.Params, error) {
	var (
		b []byte
		err error
	)
	if v := r.URL.Query().Get(QueryParam); v != "" {
		b, err = base64.StdEncoding.DecodeString(v)
	} else {
		b, err = ioutil.ReadAll(io.LimitReader(r.Body, 1<<12))
	}
	if err != nil {
		return nil, err
	}
	return QueryParser.Parse(b)
}
```
Go to [examples/simple](examples/simple.go) to see the full working example.


## API
In order to start using rql, you need to configure your parser. Let's go over a basic example of how to do this. For more details and updated documentation, please checkout the [godoc](https://godoc.org/github.com/a8m/rql/#Config). 
There are two options to build a parser, `rql.New(rql.Config)`, and `rql.MustNew(rql.Config)`. The only difference between the two is that `rql.New` returns an error if the configuration is invalid, and `rql.MustNew` panics.
```go
// we use rql.MustPanic because we don't want to deal with error handling in top level declarations.
var Parser = rql.MustNew(rql.Config{
	// User if the resource we want to query.
	Model: User{},
	// Since we work with gorm, we want to use its column-function, and not rql default.
	// although, they are pretty the same.
	ColumnFn: gorm.ToDBName,
	// Use your own custom logger. This logger is used only in the building stage.
	Log: logrus.Printf,
	// Default limit returned by the `Parse` function if no limit provided by the user.
	DefaultLimit: 100,
	// Accept only requests that pass limit value that is greater than or equal to 200.
	LimitMaxValue: 200,
})
```
rql uses reflection in the build process to detect the type of each field, and create a set of validation rules for each one. If one of the validation rules fails or rql encounters an unknown field, it returns an informative error to the user. Don't worry about the usage of reflection, it happens only once when you build the parser.
Let's go over the validation rules:
1. `int` (8,16,32,64), `sql.NullInt6` - Round number
2. `uint` (8,16,32,64), `uintptr` - Round number and greater than or equal to 0
3. `float` (32,64), sql.NullFloat64: - Number
4. `bool`, `sql.NullBool` - Boolean
5. `string`, `sql.NullString` - String
6. `time.Time`, and other types that convertible to `time.Time` - The default layout is time.RFC3339 format (JS format), and parsable to `time.Time`.
 It's possible to override the `time.Time` layout format with custom one. You can either use one of the standard layouts in the `time` package, or use a custom one. For example:
 ```go
 type User struct {
		T1 time.Time `rql:"filter"` // time.RFC3339
		T2 time.Time `rql:"filter,layout=UnixDate"` // time.UnixDate
		T3 time.Time `rql:"filter,layout=2006-01-02 15:04"` // 2006-01-02 15:04 (custom)
 }
 ``` 

Note that all rules are applied to pointers as well. It means, if you have a field `Name *string` in your struct, we still use the string validation rule for it.

### User API
We consider developers as the users of this API (usually FE developers). Let's go over the JSON API we export for resources. 
The top-level query accepts JSON with 4 fields: `offset`, `limit`, `filter` and `sort`. All of them are optional.

#### `offset` and `limit`
These two fields are useful for paging and they are equivalent to `OFFSET` and `LIMIT` in a standard SQL syntax.
- `offset` must be greater than or equal to 0 and its default value is 0
- `limit` must be greater than 0 and less than or equal to the configured `LimitMaxValue`.
 The default value for `LimitMaxValue` is 100

#### `sort`
Sort accepts a slice of strings (`[]string`) that is translated to the SQL `ORDER BY` clause. The given slice must contain only columns that are sortable (have tag `rql:"sort"`). The default order for column is ascending order in SQL, but you can control it with an optional prefix: `+` or `-`. `+` means ascending order, and `-` means descending order. Let's see a short example:
```
For input - ["address.name", "-address.zip.code", "+age"]
Result is - address_name, address_zip_code DESC, age ASC
```

#### `select`
Select accepts a slice of strings (`[]string`) that is joined with comma (",") to the SQL `SELECT` clause.
```
For input - ["name", "age"]
Result is - "name, age"
```

#### `filter`
Filter is the one who is translated to the SQL `WHERE` clause. This object that contains `filterable` fields or the disjunction (`$or`) operator. Each field in the object represents a condition in the `WHERE` clause. It contains a specific value that matched the type of the field or an object of predicates. Let's go over them:
- Field follows the format: `field: <value>`, means the predicate that will be used is `=`. For example:
 ```
 For input:
 {
 "admin": true
 }
 
 Result is: admin = ?
 ```
 You can see that RQL uses placeholders in the generated `WHERE` statement. Follow the [examples](#examples) section
 to see how to use it properly. 
- If the field follows the format: `field: { <predicate>: <value>, ...}`, For example:
 ```
 For input:
 {
 "age": {
 "$gt": 20,
 "$lt": 30
 }
 }
 
 Result is: age > ? AND age < ?
 ```
 It means that the logical `AND` operator used between the two predicates.
 Scroll below to see the full list of the supported predicates.
- `$or` is a field that represents the logical `OR` operator, and can be in any level of the query. Its type need to be an
 array of condition objects and the result of it is the disjunction between them. For example:
 ```
 For input:
 {
 "$or": [
 { "city": "TLV" },
 { "zip": { "$gte": 49800, "$lte": 57080 } }
 ]
 }
 
 Result is: city = ? OR (zip >= ? AND zip <= ?)
 ```
To simplify that, the rule is `AND` for objects and `OR` for arrays. Let's go over the list of supported predicates and then we'll show a few examples.

##### Predicates
- `$eq` and `$neq` - can be used on all types
- `$gt`, `$lt`, `$gte` and `$lte` - can be used on numbers, strings, and timestamp
- `$like` - can be used only on type string

If a user tries to apply an unsupported predicate on a field it will get an informative error. For example:
```
For input:
{
 "age": {
 "$like": "%0"
 }
}

Result is: can not apply op "$like" on field "age"
```

## Examples
Assume this is the parser for all examples.
```go

var QueryParser = rql.MustNewParser(rql.Config{
	Model: 	User{},
	FieldSep: 	".",
	LimitMaxValue:	25,
})
	
type User struct {
	ID uint `gorm:"primary_key" rql:"filter,sort"`
	Admin bool `rql:"filter"`
	Name string `rql:"filter"`
	Address string `rql:"filter"`
	CreatedAt time.Time `rql:"filter,sort"`
}
```
#### Simple Example
```go
params, err := QueryParser.Parse([]byte(`{
 "limit": 25,
 "offset": 0,
 "filter": {
 "admin": false
 }
 "sort": ["+name"]
}`))
must(err, "parse should pass")
fmt.Println(params.Limit)	// 25
fmt.Println(params.Offset)	// 0
fmt.Println(params.Sort)	// "name ASC"
fmt.Println(params.FilterExp)	// "name = ?"
fmt.Println(params.FilterArgs)	// [true]
```

In this case you've a valid generated `rql.Param` object and you can pass its to your favorite package connector.

```go
var users []*User

// entgo.io (A type-safe entity framework)
users, err = client.User.Query().
 Where(func(s *sql.Selector) {
 s.Where(sql.ExprP(p.FilterExp, p.FilterArgs...))
 }).
 Limit(p.Limit).
 Offset(p.Offset).
 All(ctx)
must(err, "failed to query ent")

// gorm
err = db.Where(p.FilterExp, p.FilterArgs).
	Offset(p.Offset).
	Limit(p.Limit).
	Order(p.Sort).
	Find(&users).Error
must(err, "failed to query gorm")

// xorm
err = engine.Where(p.FilterExp, p.FilterArgs...).
	Limit(p.Limit, p.Offset).
	OrderBy(p.Sort).
	Find(&users)
must(err, "failed to query xorm")

// go-pg/pg
err = db.Model(&users).
	Where(p.FilterExp, p.FilterArgs).
	Offset(p.Offest).
	Limit(p.Limit).
	Order(p.Sort).
	Select()
must(err, "failed to query pg/orm")

// Have more example? feel free to add.
```

#### Medium Example
```go
params, err := QueryParser.Parse([]byte(`{
 "limit": 25,
 "filter": {
 "admin": false,
 "created_at": {
 "$gt": "2018-01-01T16:00:00.000Z",
 "$lt": "2018-04-01T16:00:00.000Z"
 }
 "$or": [
 { "address": "TLV" },
 { "address": "NYC" }
 ]
 }
 "sort": ["-created_at"]
}`))
must(err, "parse should pass")
fmt.Println(params.Limit)	// 25
fmt.Println(params.Offset)	// 0
fmt.Println(params.Sort)	// "created_at DESC"
fmt.Println(params.FilterExp)	// "admin = ? AND created_at > ? AND created_at < ? AND (address = ? OR address = ?)"
fmt.Println(params.FilterArgs)	// [true, Time(2018-01-01T16:00:00.000Z), Time(2018-04-01T16:00:00.000Z), "TLV", "NYC"]
```


## Future Plans and Contributions
If you want to help with the development of this package, here is a list of options things I want to add
- [ ] JS library for query building
- [ ] Option to ignore validation with specific tag
- [ ] Add `$not` and `$nor` operators
- [ ] Automatically (or by config) filter and sort `gorm.Model` fields
- [ ] benchcmp for PRs
- [ ] Support MongoDB. Output need to be a bison object. here's a [usage example](https://gist.github.com/congjf/8035830)
- [ ] Right now rql assume all fields are flatted in the db, even for nested fields.
 For example, if you have a struct like this:
 ```go
 type User struct {
 Address struct {
 Name string `rql:"filter"`
 }
 }
 ```
 rql assumes that the `address_name` field exists in the table. Sometimes it's not the case and `address` exists in
 a different table. Therefore, I want to add the `table=` option for fields, and support nested queries.
- [ ] Code generation version - low priority

## Performance and Reliability
The performance of RQL looks pretty good, but there is always a room for improvement. Here's the current bench result:

| __Test__ | __Time/op__ | __B/op__ | __allocs/op__ |
|---------------------|----------------|------------|----------------|
| Small | 1809 | 960 | 19 |
| Medium | 6030 | 3100 | 64 |
| Large | 14726 | 7625 | 148 |

I ran fuzzy testing using `go-fuzz` and I didn't see any crashes. You are welcome to run by yourself and find potential failures. 

## LICENSE
I am providing code in the repository to you under MIT license. Because this is my personal repository, the license you receive to my code is from me and not my employer (Facebook)

Query Parser for REST API. Filtering, validations, both `AND`, `OR` operations are supported directly in the query

# Query Parser for REST

<a href="https://awesome-go.com/"><img align="right" src="https://github.com/avelino/awesome-go/raw/main/tmpl/assets/logo.png" alt="awesome-go" title="awesome-go" /></a>

[![GoDoc](https://godoc.org/github.com/timsolov/rest-query-parser?status.png)](https://godoc.org/github.com/timsolov/rest-query-parser)
[![Coverage Status](https://coveralls.io/repos/github/timsolov/rest-query-parser/badge.svg?branch=master)](https://coveralls.io/github/timsolov/rest-query-parser?branch=master)
[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)

Query Parser is a library for easy building dynamic SQL queries to Database. It provides a simple API for web-applications which needs to do some filtering throught GET queries. It is a connector between the HTTP handler and the DB engine, and manages validations and translations for user inputs.



## Installation
 go get -u github.com/timsolov/rest-query-parser

## Idea

The idia to write this library comes to me after reading this article: 
[REST API Design: Filtering, Sorting, and Pagination](https://www.moesif.com/blog/technical/api-design/REST-API-Design-Filtering-Sorting-and-Pagination/).

And principles enumerated in article I considered very useful and practical to use in our project with amount of listings with different filtering.

## Fast start
See cmd/main.go and tests for more examples.

```go
 package main
 
 import (
 "errors"
 "fmt"
 "net/url"
 
 rqp "github.com/timsolov/rest-query-parser"
 )
 
 func main() {
 url, _ := url.Parse("http://localhost/?sort=name,-id&limit=10&id=1&i[eq]=5&s[eq]=one&email[like]=*tim*|name[like]=*tim*")
 q, _ := rqp.NewParse(url.Query(), rqp.Validations{
 "limit:required": rqp.MinMax(10, 100), // limit must present in the Query part and must be between 10 and 100 (default: Min(1))
 "sort": rqp.In("id", "name"), // sort could be or not in the query but if it is present it must be equal to "in" or "name"
 "s": rqp.In("one", "two"), // filter: s - string and equal
 "id:int": nil, // filter: id is integer without additional validation
 "i:int": func(value interface{}) error { // filter: custom func for validating
 if value.(int) > 1 && value.(int) < 10 {
 return nil
 }
 return errors.New("i: must be greater then 1 and lower then 10")
 },
 "email": nil,
 "name": nil,
 })

 fmt.Println(q.SQL("table")) // SELECT * FROM table WHERE id = ? AND i = ? AND s = ? AND (email LIKE ? OR name LIKE ?) ORDER BY name, id DESC LIMIT 10
 fmt.Println(q.Where()) // id = ? AND i = ? AND s = ? AND (email LIKE ? OR name LIKE ?)
 fmt.Println(q.Args()) // [1 5 one %tim% %tim%]

 q.AddValidation("fields", rqp.In("id", "name"))
 q.SetUrlString("http://localhost/?fields=id,name&limit=10")
 q.Parse()

 fmt.Println(q.SQL("table")) // SELECT id, name FROM table ORDER BY id LIMIT 10
 fmt.Println(q.Select()) // id, name
 fmt.Println(q.Args()) // []
 }
```

## Top level fields:
* `fields` - fields for SELECT clause separated by comma (",") Eg. `&fields=id,name`. If nothing provided will use "\*" by default. Attention! If you want to use this filter you have to define validation func for it. Use `rqp.In("id", "name")` func for limit fields for your query.
* `sort` - sorting fields list separated by comma (","). Must be validated too. Could include prefix +/- which means ASC/DESC sorting. Eg. `&sort=+id,-name` will print `ORDER BY id, name DESC`. You have to filter fields in this parameter by adding `rqp.In("id", "name")`.
* `limit` - is limit for LIMIT clause. Should be greater then 0 by default. Definition of the validation for `limit` is not required. But you may use `rqp.Max(100)` to limit top threshold.
* `offset` - is offset for OFFSET clause. Should be greater then or equal to 0 by default. Definition of the validation for `offset` is not required.

## Validation modificators:
* `:required` - parameter is required. Must present in the query string. Raise error if not.
* `:int` - parameter must be convertable to int type. Raise error if not.
* `:bool` - parameter must be convertable to bool type. Raise error if not.

## Supported types
- `string` - the default type for all provided filters if not specified another. Could be compared by `eq, ne, gt, lt, gte, lte, like, ilike, nlike, nilike, in, nin, is, not` methods (`nlike, nilike` means `NOT LIKE, NOT ILIKE` respectively, `in, nin` means `IN, NOT IN` respectively, `is, not` for comparison to NULL `IS NULL, IS NOT NULL`).
- `int` - integer type. Must be specified with tag ":int". Could be compared by `eq, ne, gt, lt, gte, lte, in, nin` methods.
- `bool` - boolean type. Must be specified with tag ":bool". Could be compared by `eq` method.

## Date usage
This is simple example to show logic which you can extend.

```go
 import (
 "fmt"
 "net/url"
 validation "github.com/go-ozzo/ozzo-validation/v4"
 )

 func main() {
 url, _ := url.Parse("http://localhost/?create_at[eq]=2020-10-02")
 q, _ := rqp.NewParse(url.Query(), rqp.Validations{
 "created_at": func(v interface{}) error {
 s, ok := v.(string)
 if !ok {
 return rqp.ErrBadFormat
 }
 return validation.Validate(s, validation.Date("2006-01-02"))
 },
 })

 q.ReplaceNames(rqp.Replacer{"created_at": "DATE(created_at)"})

 fmt.Println(q.SQL("table")) // SELECT * FROM table WHERE DATE(created_at) = ?
 }
```

Easily Convert Golang structs to GraphQL objects

[![Go Report Card](https://goreportcard.com/badge/github.com/SonicRoshan/straf)](https://goreportcard.com/report/github.com/SonicRoshan/straf) [![GoDoc](https://godoc.org/github.com/SonicRoshan/straf?status.svg)](https://godoc.org/github.com/SonicRoshan/straf) [![GoCover](https://gocover.io/_badge/github.com/SonicRoshan/straf)](https://gocover.io/github.com/SonicRoshan/straf)

# Straf
1. Convert Golang Struct To GraphQL Object On The Fly
2. Easily Create GraphQL Schemas

## Example

### Converting struct to GraphQL Object
```go
type UserExtra struct {
    Age int `description:"Age of the user"` // You can use description struct tag to add description
    Gender string `deprecationReason:"Some Reason"` // You can use deprecationReason tag to add a deprecation reason
}

type User struct {
    UserID int
    Username string `unique:"true"` // You can use unique tag to define if a field would be unique
    Extra UserExtra
    Password string `exclude:"true"` // You can use exclude tag to exclude a field
}


func main() {
    // GetGraphQLObject will convert golang struct to a graphQL object
    userType, err := straf.GetGraphQLObject(User{})

    // You can then use userType in your graphQL schema
}
```


### Using The Schema Builder
```go
type User struct {
    UserID int `isArg:"true"` // You can use isArg tag to define a field as a graphql argument
    Username string `isArg:"true"`
}

var database []User = []User{}

func main() {
    // GetGraphQLObject will convert golang struct to a graphQL object
    userType, err := straf.GetGraphQLObject(User{})

    builder := straf.NewSchemaBuilder(userType, User{})
    
    builder.AddArgumentsFromStruct(object2{}) // You can use this function to add more arguments from a struct
    builder.AddFunction("CreateUser", 
                        "Adds a user to database",
                        func(params graphql.ResolveParams) (interface{}, error)) {
                            id := params.Args["UserID"]
                            username := params.Args["Username"]
                            database = append(database, User{UserID: id, Username: Username})
                        })
    schema := builder.Schema
    // You can then use this schema
}
```

### Using Middleware In Schema Builder
```go

func middleware(function func(graphql.ResolveParams) (interface{}, error), params graphql.ResolveParams) (interface{}, error) {

    fmt.Println("This function will run as a middleware")

    return function(params)
}

func main() {
    builder := straf.NewSchemaBuilder(userType, User{}, middleware)

    builder.AddFunction("SomeFunction", 
                        "Does Something",
                        someFunction)

   // Here the middleware function would run everytime before someFunction. middleware function would act as a middleware to all functions added to schema builder.
}

```

## Author
Roshan Jignesh Mehta - sonicroshan122@gmail.com


straf

A TUI playground to experiment with your favorite programs, such as grep, sed, awk, jq and yq

# play

[![Go Report](https://img.shields.io/badge/go%20report-A%2B-brightgreen.svg)](https://goreportcard.com/report/github.com/paololazzari/play)
![Pipeline status](https://github.com/paololazzari/play/actions/workflows/go.yml/badge.svg)

play is a TUI playground for your favorite programs, such as grep, sed and awk.


 <picture>
 <source media="(prefers-color-scheme: dark)" srcset="docs/gopher.png">
 <source media="(prefers-color-scheme: light)" srcset="docs/gopher.png">
 <img alt="play" title="play" src="docs/gopher.png" width="300">
 </picture>


# Demo

![](docs/demo.gif)

It uses the excellent [tview](https://github.com/rivo/tview) library for the UI.

# Installation

```bash
$ git clone https://github.com/paololazzari/play
$ cd play
$ go build -o /usr/local/bin/
```

# Usage

```bash
./play <program>
# or
cat file | ./play <program>
```

N.B. The program must be installed on your machine.

The input is evaluated immediately as you type without any validation.
If you want to use `play` in read-only mode, thus avoding any file changes (such as those that would result if, for instance, `sed -i` was used), then you can use a docker container:

```bash
$ docker run -e "TERM=xterm-256color" --rm -it -v "$(pwd)":/play:ro plazzari/play:latest <program>
```

The `TERM` environment variable is needed to get color themes working properly.

To exit the application, use `Ctrl+C`.

To exit the application printing the input expression to stdout, use `Ctrl+S`.

## Key bindings

| Component | Key | Description |
|-----------------|---------------|-------------|
| Any | `Ctrl+C` | Exit application |
| Any | `Ctrl+S` | Exit application and print input expression to stdout |
| Command Options | `Tab` | Move focus to positional arguments |
| Command Options | `Shift+Tab` | Move focus to file picker |
| Command Options | `Enter` | Move focus to output |
| Positional Arguments | `Tab` | Move focus to file picker |
| Positional Arguments | `Shift+Tab` | Move focus to command options |
| Positional Arguments | `Enter` | Move focus to output |
| Positional Arguments | `Ctrl+O` | Open wide editor |
| Wide Editor | `Esc` | Close wide editor |
| Wide Editor | `Ctrl+O` | Close wide editor |
| Wide Editor | `Ctrl+Enter` | Enter newline |
| Wide Editor | `Enter` | Move focus to output |
| File picker | `Tab` | Move focus to command options |
| File picker | `Shift+Tab` | Move focus to positional arguments options |
| File picker | `Ctrl+O` | Open selected file/Close selected file | 
| Output | `Esc` | Move focus to previous component |

# Credits

The lovely gopher was created with [gopherkon](https://github.com/quasilyte/gopherkon).

The Best Go Libraries For Query Language (17)