REST Layer
CategoryWeb Frameworks
SubcategoryFull stack web frameworks.
Stars1,256
Framework to build REST/GraphQL API on top of databases with mostly configuration over code.
About REST Layer
This large README is shown as a plain-text preview. Read the full README at the source
# REST Layer
REST APIs made easy.
[](https://godoc.org/github.com/rs/rest-layer) [](https://raw.githubusercontent.com/rs/rest-layer/master/LICENSE) [](https://travis-ci.org/rs/rest-layer) [](https://goreportcard.com/report/github.com/rs/rest-layer)
REST Layer is an API framework heavily inspired by the excellent [Python Eve](http://python-eve.org). It helps you create a comprehensive, customizable, and secure REST (graph) API on top of pluggable [backend storages](#main-storage-handlers) with no boiler plate code so you can focus on your business logic.
Implemented as a `net/http` handler, it plays well with standard middleware like [CORS](http://github.com/rs/cors). It is also [context](https://godoc.org/context) aware. This allows deadline management to be supported down to the storage and permit an easy extensibility by passing custom data between layers of the framework.
REST Layer is an opinionated framework. Unlike many API frameworks, you don't directly control the routing and you don't have to write handlers. You just define resources and sub-resources with a [schema](#resource-configuration), the framework automatically figures out what routes need to be generated behind the scene. You don't have to take care of the HTTP headers and response, JSON encoding, etc. either. REST layer handles HTTP [conditional requests](#conditional-requests), caching, [integrity checking](#data-integrity-and-concurrency-control) for you.
A powerful and extensible [validation engine](#resource-configuration) make sure that data comes pre-validated to your [custom storage handlers](#data-storage-handler). Generic resource handlers for [MongoDB](http://github.com/rs/rest-layer-mongo), [ElasticSearch](http://github.com/rs/rest-layer-es) and other databases are also available so you have few to no code to write to get up and running.
Moreover, REST Layer let you create a graph API by linking resources between them. Thanks to its advanced [field selection](#field-selection) syntax or [GraphQL](#graphql) support, you can gather resources and their dependencies in a single request, saving you from costly network round-trips.
The REST Layer framework is composed of several sub-packages:
| Package | Coverage | Description
| --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------
| [rest](https://godoc.org/github.com/rs/rest-layer/rest) | [](https://gocover.io/github.com/rs/rest-layer/rest) | A `net/http` handler to expose a REST-ful API.
| [graphql](https://godoc.org/github.com/rs/rest-layer/graphql) | [](https://gocover.io/github.com/rs/rest-layer/graphql) | A `net/http` handler to expose your API using the GraphQL protocol.
| [schema](https://godoc.org/github.com/rs/rest-layer/schema) | [](https://gocover.io/github.com/rs/rest-layer/schema) | A validation framework for the API resources.
| [resource](https://godoc.org/github.com/rs/rest-layer/resource) | [](https://gocover.io/github.com/rs/rest-layer/resource) | Defines resources, manages the resource graph and manages the interface with resource storage handler.
## Documentation
- [Breaking Changes](#breaking-changes)
- [Features](#features)
- [Extensions](#extensions)
- [Main Storage Handlers](#main-storage-handlers)
- [Alternate Storage Handlers](#alternate-storage-handlers)
- [Usage](#usage)
- [Resource Configuration](#resource-configuration)
- [Schema](#schema)
- [Field Definition](#field-definition)
- [Binding](#binding)
- [Modes](#modes)
- [Hooks](#hooks)
- [Sub Resources](#sub-resources)
- [Dependency](#dependency)
- [HTTP Request Headers](#http-request-headers)
- [Prefer](#prefer)
- [HTTP Request Methods](#http-request-methods)
- [OPTIONS](#options)
- [HEAD](#head)
- [GET](#get)
- [POST](#post)
- [PUT](#put)
- [PATCH](#patch)
- [DELETE](#delete)
- [Querying](#querying)
- [Filtering](#filtering)
- [Sorting](#sorting)
- [Field Selection](#field-selection)
- [Field Aliasing](#field-aliasing)
- [Field Parameters](#field-parameters)
- [Embedding](#embedding)
- [Pagination](#pagination)
- [Skipping](#skipping)
- [Authentication & Authorization](#authentication-and-authorization)
- [Conditional Requests](#conditional-requests)
- [Data Integrity & Concurrency Control](#data-integrity-and-concurrency-control)
- [Data Validation](#data-validation)
- [Nullable Values](#nullable-values)
- [Extensible Data Validation](#extensible-data-validation)
- [Timeout and Request Cancellation](#timeout-and-request-cancellation)
- [Logging](#logging)
- [CORS](#cors)
- [JSONP](#jsonp)
- [Data Storage Handler](#data-storage-handler)
- [Custom Response Formatter / Sender](#custom-response-formatter--sender)
- [GraphQL](#graphql)
- [Hystrix](#hystrix)
- [JSONSchema](#jsonschema)
## Breaking Changes
Until we reach a stable v1, there will be occasional breaking changes to the rest-layer APIs. Breaking changes will however not arrive at patch releases.
### Breaking changes since v0.2.0
No breaking changes since v0.2.0.
### Breaking changes prior to v0.2.0
Below is an incomplete list of breaking changes included in v0.2.0:
- PR #151: `ValuesValidator FieldValidator` attribute in `schema.Dict` struct replaced by `Values Field`.
- PR #179: `ValuesValidator FieldValidator` attribute in `schema.Array` struct replaced by `Values Field`.
- PR #204:
- Storage drivers need to accept pointer to `Expression` implementer in `query.Predicate`.
- `filter` parameters in sub-query will be validated for type match.
- `filter` parameters will be validated for type match only, instead of type & constrains.
- PR #228: `Reference` projection fields will be validated against referenced resource schema.
- PR #230: `Connection` projection fields will be validated against connected resource schema.
- PR #241: Always call `OnUpdate` field hook on HTTP PUT for existing documents. Deleting a field with `Default` value set, will always be reset to its default value.
## Features
- [x] Automatic handling of REST resource operations
- [ ] Full test coverage
- [x] Plays well with other `net/http` middleware
- [x] Pluggable resources storage
- [x] Pluggable response sender
- [x] GraphQL query support
- [ ] GraphQL mutation support
- [ ] Swagger Documentation
- [x] JSONSchema Output (partial)
- [ ] Testing framework
- [x] Sub resources
- [ ] Cascading deletes on sub resources
- [x] Filtering
- [x] Sorting
- [x] Pagination
- [x] Aliasing
- [x] Custom business logic
- [x] Event hooks
- [x] Field hooks
- [x] Extensible data validation and transformation
- [x] Conditional requests (Last-Modified / Etag)
- [x] Data integrity and concurrency control (If-Match)
- [x] Timeout and request cancellation through [context](https://godoc.org/context)
- [x] Logging
- [x] Multi-GET
- [ ] Bulk inserts
- [x] Default and nullable values
- [ ] Per resource cache control
- [ ] Customizable authentication / authorization
- [x] Projections
- [x] Embedded resource serialization
- [x] Sub-request concurrency control
- [x] Custom ID field
- [ ] Data versioning
- [x] Per resource circuit breaker using [Hystrix](https://godoc.org/github.com/afex/hystrix-go/hystrix)
- [x] [JSON-Patch](https://tools.ietf.org/html/rfc6902) support
### Extensions
As REST Layer is a simple `net/http` handler. You can use standard middleware to extend its functionalities:
- [x] [CORS](http://github.com/rs/cors)
- [ ] Method Override
- [ ] Gzip, Deflate
- [ ] JSONP
- [x] [X-Forwarded-For](https://github.com/sebest/xff)
- [x] [Rate Limiting](https://github.com/didip/tollbooth)
- [ ] Operations Log
- [x] [Hystrix storage handler wrapper](https://github.com/rs/rest-layer-hystrix)
### Main Storage Handlers
- [x] [Memory](http://github.com/rs/rest-layer/tree/master/resource/testing/mem) (test only)
- [x] [MongoDB](http://github.com/rs/rest-layer-mongo)
### Alternate Storage Handlers
- [ElasticSearch](http://github.com/rs/rest-layer-es) (no longer actively tested)
- [SQL](https://github.com/apuigsech/rest-layer-sql) (third party)
- [Google Datastore](https://github.com/ajcrowe/rest-layer-datastore) (third party)
- [Kubernetes ConfigMap](https://github.com/Segence/rest-layer-kubernetes-configmap) (third party)
## Usage
```go
package main
import (
"log"
"net/http"
"github.com/rs/rest-layer/resource/testing/mem"
"github.com/rs/rest-layer/resource"
"github.com/rs/rest-layer/rest"
"github.com/rs/rest-layer/schema/query"
"github.com/rs/rest-layer/schema"
)
var (
// Define a user resource schema
user = schema.Schema{
Description: `The user object`,
Fields: schema.Fields{
"id": {
Required: true,
// When a field is read-only, only default values or hooks can
// set their value. The client can't change it.
ReadOnly: true,
// This is a field hook called when a new user is created.
// The schema.NewID hook is a provided hook to generate a
// unique id when no value is provided.
OnInit: schema.NewID,
// The Filterable and Sortable allows usage of filter and sort
// on this field in requests.
Filterable: true,
Sortable: true,
Validator: &schema.String{
Regexp: "^[0-9a-v]{20}$",
},
},
"created": {
Required: true,
ReadOnly: true,
Filterable: true,
Sortable: true,
OnInit: schema.Now,
Validator: &schema.Time{},
},
"updated": {
Required: true,
ReadOnly: true,
Filterable: true,
Sortable: true,
OnInit: schema.Now,
// The OnUpdate hook is called when the item is edited. Here we use
// provided Now hook which returns the current time.
OnUpdate: schema.Now,
Validator: &schema.Time{},
},
// Define a name field as required with a string validator
"name": {
Required: true,
Filterable: true,
Validator: &schema.String{
MaxLen: 150,
},
},
},
}
// Define a post resource schema
post = schema.Schema{
Description: `Represents a blog post`,
Fields: schema.Fields{
// schema.*Field are shortcuts for common fields
// (identical to users' same fields)
"id": schema.IDField,
"created": schema.CreatedField,
"updated": schema.UpdatedField,
// Define a user field which references the user owning the post.
// See bellow, the content of this field is enforced by the fact
// that posts is a sub-resource of users.
"user": {
Required: true,
Filterable: true,
Validator: &schema.Reference{
Path: "users",
},
},
"published": {
Required: true,
Filterable: true,
Default: false,
Validator: &schema.Bool{},
},
"title": {
Required: true,
Validator: &schema.String{
MaxLen: 150,
},
},
"body": {
// Dependency defines that body field can't be changed if
// the published field is not "false".
Dependency: query.MustParsePredicate(`{"published": false}`),
Validator: &schema.String{
MaxLen: 100000,
},
},
},
}
)
func main() {
// Create a REST API resource index
index := resource.NewIndex()
// Add a resource on /users[/:user_id]
users := index.Bind("users", user, mem.NewHandler(), resource.Conf{
// We allow all REST methods
// (rest.ReadWrite is a shortcut for []resource.Mode{resource.Create,
// resource.Read, resource.Update, resource.Delete, resource,List})
AllowedModes: resource.ReadWrite,
})
// Bind a sub resource on /users/:user_id/posts[/:post_id]
// and reference the user on each post using the "user" field of the posts resource.
users.Bind("posts", "user", post, mem.NewHandler(), resource.Conf{
// Posts can only be read, created and deleted, not updated
AllowedModes: []resource.Mode{resource.Read, resource.List,
resource.Create, resource.Delete},
})
// Create API HTTP handler for the resource graph
api, err := rest.NewHandler(index)
if err != nil {
log.Fatalf("Invalid API configuration: %s", err)
}
// Bind the API under /api/ path
http.Handle("/api/", http.StripPrefix("/api/", api))
// Serve it
log.Print("Serving API on http://localhost:8080")
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatal(err)
}
}
```
Just run this code (or use the provided [examples/demo](https://github.com/rs/rest-layer/blob/master/examples/demo/main.go)):
```sh
$ go run examples/demo/main.go
2015/07/27 20:54:55 Serving API on http://localhost:8080
```
Using [HTTPie](http://httpie.org/), you can now play with your API.
First create a user:
```sh
$ http POST :8080/api/users name="John Doe"
HTTP/1.1 201 Created
Content-Length: 155
Content-Location: /api/users/ar6ejgmkj5lfl98r67p0
Content-Type: application/json
Date: Mon, 27 Jul 2015 19:10:20 GMT
Etag: "1e18e148e1ff3ecdaae5ec03ac74e0e4"
Last-Modified: Mon, 27 Jul 2015 19:10:20 GMT
Vary: Origin
{
"id": "ar6ejgmkj5lfl98r67p0",
"created": "2015-07-27T21:10:20.671003126+02:00",
"updated": "2015-07-27T21:10:20.671003989+02:00",
"name": "John Doe",
}
```
As you can see, the `id`, `created` and `updated` fields have been automatically generated by our `OnInit` field hooks.
Also notice the `Etag` and `Last-Modified` headers. Those guys allow data integrity and concurrency control *down to the storage layer* through the use of the `If-Match` and `If-Unmodified-Since` headers. They can also serve for conditional requests using `If-None-Match` and `If-Modified-Since` headers.
Here is an example of conditional request:
```sh
$ http :8080/api/users/ar6ejgmkj5lfl98r67p0 \
If-Modified-Since:"Mon, 27 Jul 2015 19:10:20 GMT"
HTTP/1.1 304 Not Modified
Date: Mon, 27 Jul 2015 19:17:11 GMT
Vary: Origin
```
And here is a data integrity request following the [RFC-5789](http://tools.ietf.org/html/rfc5789) recommendations:
```sh
$ http PATCH :8080/api/users/ar6ejgmkj5lfl98r67p0 \
name="Someone Else" If-Match:invalid-etag
HTTP/1.1 412 Precondition Failed
Content-Length: 58
Content-Type: application/json
Date: Mon, 27 Jul 2015 19:33:27 GMT
Vary: Origin
{
"code": 412,
"fields": null,
"message": "Precondition Failed"
}
```
Retry with the valid etag:
```sh
$ http PATCH :8080/api/users/ar6ejgmkj5lfl98r67p0 \
name="Someone Else" If-Match:'"1e18e148e1ff3ecdaae5ec03ac74e0e4"'
HTTP/1.1 200 OK
Content-Length: 159
Content-Type: application/json
Date: Mon, 27 Jul 2015 19:36:19 GMT
Etag: "7bb7a71b0f66197aa07c4c8fc9564616"
Last-Modified: Mon, 27 Jul 2015 19:36:19 GMT
Vary: Origin
{
"created": "2015-07-27T21:33:09.168492448+02:00",
"id": "ar6ejmukj5lflde9q8bg",
"name": "Someone Else",
"updated": "2015-07-27T21:36:19.904545093+02:00"
}
```
Note that even if you don't use conditional request, the `Etag` is always used by the storage handler to manage concurrency control between requests.
Another cool thing is sub-resources. We've set our `posts` resource as a child of the `users` resource. This way we can handle ownership very easily as routes are constructed as `/users/:user_id/posts`.
Lets create a post:
```sh
$ http POST :8080/api/users/ar6ejgmkj5lfl98r67p0/posts \
title="My first post"
HTTP/1.1 200 OK
Content-Length: 212
Content-Type: application/json
Date: Mon, 27 Jul 2015 19:46:55 GMT
Etag: "307ae92df6c3dd54847bfc7d72422e07"
Last-Modified: Mon, 27 Jul 2015 19:46:55 GMT
Vary: Origin
{
"id": "ar6ejs6kj5lflgc28es0",
"created": "2015-07-27T21:46:55.355857401+02:00",
"updated": "2015-07-27T21:46:55.355857989+02:00",
"title": "My first post",
"user": "ar6ejgmkj5lfl98r67p0"
}
```
Notice how the `user` field has been set with the user id provided in the route, that's pretty cool, huh?
We defined that we can create posts but we can't modify them, lets verify that:
```sh
$ http PATCH :8080/api/users/821d…/posts/ar6ejs6kj5lflgc28es0 \
private=true
HTTP/1.1 405 Method Not Allowed
Content-Length: 53
Content-Type: application/json
Date: Mon, 27 Jul 2015 19:50:33 GMT
Vary: Origin
{
"code": 405,
"fields": null,
"message": "Invalid method"
}
```
Let's list posts for that user now:
```sh
$ http :8080/api/users/ar6ejgmkj5lfl98r67p0/posts
HTTP/1.1 200 OK
Content-Length: 257
Content-Type: application/json
Date: Mon, 27 Jul 2015 19:51:46 GMT
Vary: Origin
X-Total: 1
[
{
"id": "ar6ejs6kj5lflgc28es0",
"_etag": "307ae92df6c3dd54847bfc7d72422e07",
"created": "2015-07-27T21:46:55.355857401+02:00",
"updated": "2015-07-27T21:46:55.355857989+02:00",
"title": "My first post",
"user": "ar6ejgmkj5lfl98r67p0"
}
]
```
Notice the added `_etag` field. This is to let you get etags of multiple items without having to `GET` each one of them through individual requests.
Now, let's get user's information for each posts in a single request:
```sh
$ http :8080/api/users/ar6ejgmkj5lfl98r67p0/posts fields=='id,title,user{id,name}'
HTTP/1.1 200 OK
Content-Length: 257
Content-Type: application/json
Date: Mon, 27 Jul 2015 19:51:46 GMT
Vary: Origin
X-Total: 1
[
{
"id": "ar6ejs6kj5lflgc28es0",
"_etag": "307ae92df6c3dd54847bfc7d72422e07",
"created": "2015-07-27T21:46:55.355857401+02:00",
"updated": "2015-07-27T21:46:55.355857989+02:00",
"title": "My first post",
"user": {
"id": "ar6ejgmkj5lfl98r67p0",
"name": "John Doe"
}
}
]
```
Notice how we selected which fields we wanted in the result using the [field selection](#field-selection) query format. Thanks to sub-request support, the user name is included with each post with no additional HTTP request.
We can go even further and embed a sub-request list responses. Let's say we want a list of users with the last two posts:
```sh
$ http GET :8080/api/users fields='id,name,posts(limit:2){id,title}'
HTTP/1.1 201 Created
Content-Length: 155
Content-Location: /api/users/ar6ejgmkj5lfl98r67p0
Content-Type: application/json
Date: Mon, 27 Jul 2015 19:10:20 GMT
Etag: "1e18e148e1ff3ecdaae5ec03ac74e0e4"
Last-Modified: Mon, 27 Jul 2015 19:10:20 GMT
Vary: Origin
[
{
"id": "ar6ejgmkj5lfl98r67p0",
"name": "John Doe",
"posts": [
{
"id": "ar6ejs6kj5lflgc28es0",
"title": "My first post"
},
{
"id": "ar6ek26kj5lfljgh84qg",
"title": "My second post"
}
]
}
]
```
Sub-requests are executed concurrently whenever possible to ensure the fastest response time.
## Resource Configuration
For REST Layer to be able to expose resources, you have to first define what fields the resource contains and where to bind it in the REST API URL namespace.
### Schema
Resource field configuration is performed through the [schema](https://godoc.org/github.com/rs/rest-layer/schema) package. A schema is a struct describing a resource. A schema is composed of metadata about the resource and a description of the allowed fields through a map of field name pointing to field definition.
Sample resource schema:
```go
foo = schema.Schema{
Description: "A foo object",
Fields: schema.Fields{Frequently Asked Questions
What is REST Layer?
REST Layer is a Web Frameworks library for the Go programming language. Framework to build REST/GraphQL API on top of databases with mostly configuration over code.
How many GitHub stars does REST Layer have?
REST Layer has 1,256 GitHub stars in the directory's latest synchronization.
How do I install REST Layer?
Install REST Layer with the Go module system using `go get rs/rest-layer`. Check the repository for the current installation instructions.
What category does REST Layer belong to?
REST Layer is listed under Web Frameworks, specifically Full stack web frameworks..