teler-waf
teler-waf is a Go HTTP middleware that provide teler IDS functionality to protect against web-based attacks and improve the security of Go-based web applications. It is highly configurable and easy to integrate into existing Go applications
About teler-waf
This large README is shown as a plain-text preview. Read the full README at the source
# teler-waf
[](http://pkg.go.dev/github.com/teler-sh/teler-waf)
[](https://codecov.io/gh/teler-sh/teler-waf)
[](https://github.com/teler-sh/teler-waf/actions/workflows/tests.yaml)
[](https://securityscorecards.dev/viewer/?uri=github.com/teler-sh/teler-waf)
[](https://github.com/avelino/awesome-go)
<img src="https://user-images.githubusercontent.com/25837540/97091757-7200d880-1668-11eb-82c4-e5c4971d2bc8.png" align="right" width="250px"/>
**teler-waf** is a comprehensive security solution for Go-based web applications. It acts as an HTTP middleware, providing an easy-to-use interface for integrating IDS functionality with [teler IDS](https://github.com/teler-sh/teler-waf) into existing Go applications. By using teler-waf, you can help protect against a variety of web-based attacks, such as cross-site scripting (XSS) and SQL injection.
The package comes with a standard [`net/http.Handler`](https://pkg.go.dev/net/http#Handler), making it easy to integrate into your application's routing. When a client makes a request to a route protected by teler-waf, the request is first checked against the teler IDS to detect known malicious patterns. If no malicious patterns are detected, the request is then passed through for further processing.
In addition to providing protection against web-based attacks, teler-waf can also help improve the overall security and integrity of your application. It is highly configurable, allowing you to tailor it to fit the specific needs of your application.
**See also:**
- [teler-sh/teler](https://github.com/teler-sh/teler): Real-time HTTP intrusion detection.
- [teler-sh/teler-proxy](https://github.com/teler-sh/teler-proxy): teler Proxy enabling seamless integration with teler WAF.
- [teler-sh/teler-caddy](https://github.com/teler-sh/teler-caddy): teler Caddy integrates the powerful security features of teler WAF into the Caddy web server
## Features
teler-waf offers a range of powerful features designed to enhance the security of your Go web applications:
- **HTTP middleware** for Go web applications.
- Integration of **teler IDS** functionality.
- **Detection of known malicious patterns** using the teler IDS.
- Common web attacks, such as cross-site scripting (XSS) and SQL injection, etc.
- CVEs, covers known vulnerabilities and exploits.
- Bad IP addresses, such as those associated with known malicious actors or botnets.
- Bad HTTP referers, such as those that are not expected based on the application's URL structure or are known to be associated with malicious actors.
- Bad crawlers, covers requests from known bad crawlers or scrapers, such as those that are known to cause performance issues or attempt to extract sensitive information from the application.
- Directory bruteforce attacks, such as by trying common directory names or using dictionary attacks.
- Providing increased flexibility for creating your own **custom rules**.
- Configuration options to **whitelist specific types of requests** based on their URL or headers.
- **Easy integration** with many frameworks.
- **High configurability** to fit the specific needs of your application.
Overall, teler-waf provides a comprehensive security solution for Go-based web applications, helping to protect against web-based attacks and improve the overall security and integrity of your application.
## Install
**Dependencies**:
* **gcc** (GNU Compiler Collection) should be installed & configured to compile teler-waf.
To install teler-waf in your Go application, run the following command to download and install the teler-waf package:
```console
go get github.com/teler-sh/teler-waf
```
## Usage
> [!WARNING]
> **Deprecation notice**: Threat exclusions (`Excludes`) will be deprecated in the upcoming release (**v2**). See [#73](https://github.com/teler-sh/teler-waf/discussions/73) & [#64](https://github.com/teler-sh/teler-waf/issues/64).
Here is an example of how to use teler-waf in a Go application:
1. Import the teler-waf package in your Go code:
```go
import "github.com/teler-sh/teler-waf"
```
2. Use the `New` function to create a new instance of the `Teler` type. This function takes a variety of optional parameters that can be used to configure teler-waf to suit the specific needs of your application.
```go
waf := teler.New()
```
3. Use the `Handler` method of the `Teler` instance to create a `net/http.Handler`. This handler can then be used in your application's HTTP routing to apply teler-waf's security measures to specific routes.
```go
handler := waf.Handler(http.HandlerFunc(yourHandlerFunc))
```
4. Use the `handler` in your application's HTTP routing to apply teler-waf's security measures to specific routes.
```go
http.Handle("/path", handler)
```
That's it! You have configured teler-waf in your Go application.
**Options:**
For a list of the options available to customize teler-waf, see the [`teler.Options`](https://pkg.go.dev/github.com/teler-sh/teler-waf#Options) struct.
### Examples
Here is an example of how to customize the options and rules for teler-waf:
```go
// main.go
package main
import (
"net/http"
"github.com/teler-sh/teler-waf"
"github.com/teler-sh/teler-waf/request"
"github.com/teler-sh/teler-waf/threat"
)
var myHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// This is the handler function for the route that we want to protect
// with teler-waf's security measures.
w.Write([]byte("hello world"))
})
var rejectHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// This is the handler function for the route that we want to be rejected
// if the teler-waf's security measures are triggered.
http.Error(w, "Sorry, your request has been denied for security reasons.", http.StatusForbidden)
})
func main() {
// Create a new instance of the Teler type using the New function
// and configure it using the Options struct.
telerMiddleware := teler.New(teler.Options{
// Exclude specific threats from being checked by the teler-waf.
Excludes: []threat.Threat{
threat.BadReferrer,
threat.BadCrawler,
},
// Specify whitelisted URIs (path & query parameters), headers,
// or IP addresses that will always be allowed by the teler-waf
// with DSL expressions.
Whitelists: []string{
`request.Headers matches "(curl|Go-http-client|okhttp)/*" && threat == BadCrawler`,
`request.URI startsWith "/wp-login.php"`,
`request.IP in ["127.0.0.1", "::1", "0.0.0.0"]`,
`request.Headers contains "authorization" && request.Method == "POST"`
},
// Specify file path or glob pattern of custom rule files.
CustomsFromRule: "/path/to/custom/rules/**/*.yaml",
// Specify custom rules for the teler-waf to follow.
Customs: []teler.Rule{
{
// Give the rule a name for easy identification.
Name: "Log4j Attack",
// Specify the logical operator to use when evaluating the rule's conditions.
Condition: "or",
// Specify the conditions that must be met for the rule to trigger.
Rules: []teler.Condition{
{
// Specify the HTTP method that the rule applies to.
Method: request.GET,
// Specify the element of the request that the rule applies to
// (e.g. URI, headers, body).
Element: request.URI,
// Specify the pattern to match against the element of the request.
Pattern: `\$\{.*:\/\/.*\/?\w+?\}`,
},
},
},
{
// Give the rule a name for easy identification.
Name: `Headers Contains "curl" String`,
// Specify the conditions that must be met for the rule to trigger.
Rules: []teler.Condition{
{
// Specify the DSL expression that the rule applies to.
DSL: `request.Headers contains "curl"`,
},
},
},
},
// Specify the file path to use for logging.
LogFile: "/tmp/teler.log",
})
// Set the rejectHandler as the handler for the telerMiddleware.
telerMiddleware.SetHandler(rejectHandler)
// Create a new handler using the handler method of the Teler instance
// and pass in the myHandler function for the route we want to protect.
app := telerMiddleware.Handler(myHandler)
// Use the app handler as the handler for the route.
http.ListenAndServe("127.0.0.1:3000", app)
}
```
For more examples of how to use teler-waf or integrate it with any framework, take a look at [examples/](https://github.com/teler-sh/teler-waf/tree/master/examples) directory.
### Custom Rules
> [!TIP]
> If you want to explore configurations, delve into crafting custom rules and composing DSL expressions, you can practice and gain hands-on experience by using this [teler WAF playground](https://play.teler.sh/). Here, you can also simulate requests customized to fulfill the specific needs of your application.
To integrate custom rules into the teler-waf middleware, you have two choices: `Customs` and `CustomsFromFile`. These options offer flexibility to create your own security checks or override the default checks provided by teler-waf.
- **`Customs` option**
You can define custom rules directly using the `Customs` option, as shown in the [example](https://github.com/teler-sh/teler-waf#examples) above.
In the `Customs` option, you provide an array of `teler.Rule` structures. Each `teler.Rule` represents a custom rule with a unique name and a condition that specifies how the individual conditions within the rule are evaluated (`or` or `and`). The rule consists of one or more `teler.Condition` structures, each defining a specific condition to check. Conditions can be based on the [HTTP method](https://pkg.go.dev/github.com/teler-sh/teler-waf/request#pkg-constants), [element](https://pkg.go.dev/github.com/teler-sh/teler-waf/request#Element) (headers, body, URI, or any), and a regex pattern or a [DSL expression](https://pkg.go.dev/github.com/teler-sh/teler-waf/dsl) to match against.
- **`CustomsFromFile` option**
Alternatively, the `CustomsFromFile` option allows you to load custom rules from external files, offering even greater flexibility and manageability. These rules can be defined in YAML format, with each file containing one or more rules. Here is an example YAML structure representing a custom rule:
```yaml
- name: <name>
condition: <condition> # Valid values are: "or" or "and", in lowercase or uppercase.
rules:
- method: <method> # Valid methods are: "ALL", "CONNECT", "DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT", and "TRACE". Please refer to https://pkg.go.dev/github.com/teler-sh/teler-waf/request for further details.
element: <element> # Valid elements are: "headers", "body", "uri", and "any", in lowercase, uppercase, or title case (except for "uri").
pattern: "<pattern>" # Regular expression pattern
- dsl: "<expression>" # DSL expression
```
> [!IMPORTANT]
> Please note that the `condition`, `method`, and `element` are optional parameters. The default values assigned to them are as follows: `condition` is set to **or**, `method` is set to **ALL**, and `element` is set to **ANY**. Therefore, if desired, you can leave those parameters empty. The `pattern` parameter is mandatory, unless you specify a `dsl` expression. In such cases, when a `dsl` expression is provided, teler-waf will disregard any values assigned to `method` and `element`, even if they are defined. To see some examples, you can refer to the [`tests/rules/`](https://github.com/teler-sh/teler-waf/tree/master/tests/rules/valid) directory.
You can specify the `CustomsFromFile` option with the actual file path or glob pattern pointing to the location of your custom rule files. For example:
```go
// Create a new instance of the Teler middleware and
// specify custom rules with the CustomsFromFile option.
telerMiddleware := teler.New(teler.Options{
CustomsFromFile: "/path/to/custom/rules/**/*.yaml",
})
```
With `CustomsFromFile`, you provide the file path or glob pattern where your custom rule files are located. The pattern can include wildcards to match multiple files or a directory and its subdirectories. Each file should contain one or more custom rules defined in the proper YAML format.
By utilizing either the `Customs`, `CustomsFromFile`, or both option, you can seamlessly integrate your custom rules into the teler-waf middleware, enhancing its security capabilities to meet your specific requirements.
### DSL Expression
DSL (Domain-Specific Language) expressions provide a powerful means of defining conditions that are used to evaluate incoming requests within the context of custom rules or whitelists. With DSL expressions, you can create sophisticated and targeted conditions based on different attributes of the incoming requests. Here are some illustrative examples of DSL expression code:
#### Examples of DSL expression code:
Check if the incoming request headers contains "curl":
```sql
request.Headers contains "curl"
```
Check if the incoming request method is "GET":
```sql
request.Method == "GET"
```
Check if the incoming request method is "GET" or "POST" using regular expression, `matches` operator:
```sql
request.Method matches "^(POS|GE)T$"
```
Check if the incoming request IP address is from localhost:
```sql
request.IP in ["127.0.0.1", "::1", "0.0.0.0"]
```
Check if the any element in request contains the string "foo":
```console
one(request.ALL, # contains "foo")
```
Check if the incoming request body contains "foo":
```sql
request.Body contains "foo"
```
Check whether the current threat category being analyzed is bad crawler or directory bruteforce:
```sql
threat in [BadCrawler, DirectoryBruteforce]
```
Those examples provide a glimpse into the expressive capabilities of DSL expressions, allowing you to define intricate conditions based on various request attributes. By leveraging these expressions, you can effectively define the criteria for evaluating incoming requests and tailor your custom rules or whitelists accordingly, enabling fine-grained control over your application's behavior.
#### Available variables
When working with DSL expressions, you have access to various variables that provide valuable information about the incoming requests and the threat category being analyzed. Here is a detailed description of the available variables:
- **Threat category**
All constant identifiers of the `threat.Threat` type can be used as valid variables. These identifiers represent different threat categories that are relevant to your analysis.
- **`request`**
The `request` variable represents the incoming request and provides access to its fields and corresponding values. The following sub-variables are available within the `request` variable:
- `request.URI`: Represents the URI of the incoming request, including the path, queries, parameters, and fragments.
- `request.Headers`: Represents the headers of the incoming request, presented in multiple lines.
- `request.Body`: Represents the body of the incoming request.
- `request.Method`: Represents the method of the incoming request.
- `request.IP`: Represents the client IP address associated with the incoming request.
- `request.ALL`: Represents all the string values from the request fields mentioned above in a slice.
- **`threat`**
The `threat` variable represents the threat category being analyzed. It is of the `threat.Threat` type and allows you to evaluate and make decisions based on the specific threat category associated with the request.
By utilizing those variables within your DSL expressions, you can effectively access and manipulate the attributes of the incoming requests and assess the relevant threat categories. This enables you to create custom rule conditions<!-- and whitelists--> that tailored to your specific use case.
#### Available functions
Also, you have access to a variety of functions. These functions encompass both the [built-in functions](https://expr.medv.io/docs/Language-Definition#built-in-functions) provided by the expr package and those specifically defined within the DSL package. The functions utilize the functionalities offered by the built-in `strings` Go package. Here is a detailed list of the functions available:
- `cidr`: Get all IP addresses in range with given CIDR.
- `clone`: Create a copy of a string.
- `containsAny`: Check if a string contains any of the specified substrings.
- `equalFold`: Compare two strings in a case-insensitive manner.
- `hasPrefix`: Check if a string has a specified prefix.
- `hasSuffix`: Check if a string has a specified suffix.
- `join`: Concatenate multiple strings using a specified separator.
- `repeat`: Repeat a string a specified number of times.
- `replace`: Replace occurrences of a substring within a string.
- `replaceAll`: Replace all occurrences of a substring within a string.
- `request`: Access request-specific information within the DSL expression.
- `threat`: Access information related to the threat category being analyzed.
- `title`: Convert a string to title case.
- `toLower`: Convert a string to lowercase.
- `toTitle`: Convert a string to title case.
- `toUpper`: Convert a string to uppercase.
- `toValidUTF8`: Convert a string to a valid UTF-8 encoded string.
- `trim`: Remove leading and trailing whitespace from a string.
- `trimLeft`: Remove leading whitespace from a string.
- `trimPrefix`: Remove a specified prefix from a string.
- `trimRight`: Remove trailing whitespace from a string.
- `trimSpace`: Remove leading and trailing whitespace and collapse consecutive whitespace within a string.
- `trimSuffix`: Remove a specified suffix from a string.
For more comprehensive details on operators and built-in functions, please refer to the [Expr documentation](https://expr.medv.io/docs/Getting-Started). It provides a comprehensive guide to utilizing operators and exploring the available built-in functions in your DSL expressions.
### Streamlined Configuration Management
For effective configuration, it's essential to define a range of settings, including whitelists, custom rule definitions, logging preferences, and other parameters. The [`option`](https://pkg.go.dev/github.com/teler-sh/teler-waf/option) package streamlines this configuration workflow by enabling you to efficiently unmarshal or load configuration data from JSON and YAML formats into a format that teler-waf can readily comprehend and implement.
```go
// Load configuration from a YAML file.
opt, err := option.LoadFromYAMLFile("/path/to/teler-waf.conf.yaml")
if err != nil {
panic(err)
}
// Create a new instance of the Teler type with
// the loaded options.
telerMiddleware := teler.New(opt)
```
### Development
By default, teler-waf caches all incoming requests for 15 minutes & clear them every 20 minutes to improve the performance. However, if you're still customizing the settings to match the requirements of your application, you can disable caching during development by setting the development mode option to `true`. This will prevent incoming requests from being cached and can be helpful for debugging purposes.
```go
// Create a new instance of the Teler type using
// the New function & enable development mode option.
telerMiddleware := teler.New(teler.Options{
Development: true,
})
```
### Logs
Here is an example of what the log lines would look like if teler-waf detects a threat on a request:
```jsonFrequently Asked Questions
What is teler-waf?
teler-waf is a Security library for the Go programming language. teler-waf is a Go HTTP middleware that provide teler IDS functionality to protect against web-based attacks and improve the security of Go-based web applications. It is highly configurable and easy to integrate into existing Go applications
How do I install teler-waf?
Install teler-waf with the Go module system using `go get kitabisa/teler-waf`. Check the repository for the current installation instructions.
What category does teler-waf belong to?
teler-waf is listed under Security, specifically Security.