📚 go-streams - Awesome Go Library for Stream Processing
Go stream processing library.
🏷️ Stream Processing
📂 Libraries and tools for stream processing and reactive programming.
⭐ 1,876 stars
Detailed Description of go-streams
go-streams
A lightweight stream processing library for Go.
go-streams provides a simple and concise DSL to build data pipelines.
Wiki
In computing, a pipeline, also known as a data pipeline, is a set of data processing elements connected in series, where the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion. Some amount of buffer storage is often inserted between elements.
Overview
Building blocks:
- Source - A Source is a set of stream processing steps that has one open output.
- Flow - A Flow is a set of stream processing steps that has one open input and one open output.
- Sink - A Sink is a set of stream processing steps that has one open input. Can be used as a Subscriber.
Implemented Flows (flow package):
- Map
- FlatMap
- Filter
- Reduce
- PassThrough
- Split
- FanOut
- RoundRobin
- Merge
- Flatten
- Batch
- Throttler
- SlidingWindow
- TumblingWindow
- SessionWindow
- Keyed
Supported Connectors:
- Go channels
- File system
- Network (TCP, UDP)
- WebSocket
- Aerospike
- Apache Kafka
- Apache Pulsar
- AWS (S3)
- GCP (Storage)
- NATS
- Redis
Examples
Usage samples are available in the examples directory.
License
Licensed under the MIT License.