goio
An implementation of IO, Stream, Fiber for Golang, inspired by awesome Scala libraries cats and fs2
About goio
This large README is shown as a plain-text preview. Read the full README at the source
# goio: IO, Stream, and Fiber for Go
[](https://app.codacy.com/gh/Primetalk/goio?utm_source=github.com&utm_medium=referral&utm_content=Primetalk/goio&utm_campaign=Badge_Grade_Settings)
[](https://pkg.go.dev/github.com/primetalk/goio)
[](https://godoc.org/github.com/primetalk/goio)
[](https://goreportcard.com/report/github.com/primetalk/goio)
[](https://img.shields.io/github/v/tag/primetalk/goio)
[](https://codecov.io/gh/Primetalk/goio)
This library is an attempt to fill the gap of a decent generic streaming library in Go. The design uses Go generics to provide typed functional abstractions.
The design is inspired by awesome Scala libraries [cats-effect](https://typelevel.org/cats-effect/) and [fs2](https://fs2.io/).
## Go compatibility
The `goio` v0.5 prerelease line requires Go 1.27 or newer and introduces generic methods for fluent, type-safe composition. Version v0.4.x remains the final release line supporting Go 1.26. CI validates Go 1.27 on Linux, Windows, and macOS and runs the race detector on Linux.
Version v0.3.7 remains available for users that must retain the former Go 1.18 module floor. See [MIGRATION.md](MIGRATION.md) for upgrade steps and [RELEASE_NOTES.md](RELEASE_NOTES.md) for the compatibility notice.
Raising the Go version changes build compatibility only. It does not add cancellation, structured concurrency, or different timeout and race-selection semantics to the effect runtime.
Go 1.27 users can choose between fluent methods and the original package functions. Both forms have the same semantics, and the package functions remain supported:
```go
mapped := io.Lift(42).
Map(strconv.Itoa).
FlatMap(func(value string) io.IO[int] {
return io.Lift(len(value))
})
```
## Functions
This package provides a few general functions that are sometimes useful.
- `fun.Const[A any, B any](b B)func(A)B`
- `fun.ConstUnit[B any](b B) func(Unit)B`
- `fun.ConstNoArg[B any](b B) func() B` - ConstNoArg creates a function that will return the specified value.
- `fun.Delay[A any](a A) func() A` - Delay returns a function that will return the value.
- `fun.Identity[A any](a A) A` - Identity function returns the given value unchanged.
- `fun.Swap[A any, B any, C any](f func(a A)func(b B)C) func(b B)func(a A)C`
- `fun.Curry[A any, B any, C any](f func(a A, b B)C) func(a A)func(b B)C`
- `fun.Compose[A any, B any, C any](f func(A) B, g func (B) C) func (A) C` - Compose executes the given functions in sequence.
- `fun.Memoize[A comparable, B any](f func(a A) B) func(A) B` - Memoize returns a function that will remember the original function in a map. It's thread safe, however, not super performant.
There are also basic data structures - Unit, Pair and Either.
- `fun.Unit` - type that has only one instance
- `fun.Unit1` - the instance of the Unit type
- `fun.Pair[A any, B any]` - type that represents both A and B.
- `fun.PairV1[A any, B any](p Pair[A, B]) A` - PairV1 returns the first element of the pair.
- `fun.PairV2[A any, B any](p Pair[A, B]) B` - PairV2 returns the second element of the pair.
- `fun.PairBoth[A any, B any](p Pair[A, B]) (A, B)` - PairBoth returns both parts of the pair.
- `fun.PairSwap[A any, B any](p Pair[A, B]) Pair[B, A]` - PairSwap returns a pair with swapped parts.
For debug purposes it's useful to convert arbitrary data to strings.
- `fun.ToString[A any](a A) string` - converts the value to string using `Sprintf` `%v`.
For compatibility with `interface {}`:
- `fun.CastAsInterface[A any](a A) interface {}` - CastAsInterface casts a value of an arbitrary type as interface {}.
- `fun.Cast[A any](i Any) (a A, err error)` - Cast converts interface {} to ordinary type A. It'a simple operation i.(A) represented as a function. In case the conversion is not possible, returns an error.
- `fun.UnsafeCast[A any](i interface {}) A` - UnsafeCast converts interface {} to ordinary type A. It'a simple operation i.(A) represented as a function. In case the conversion is not possible throws a panic.
Is there a way to obtain a value of an arbitrary type?
- `fun.Nothing[A any]() A` - This function can be used anywhere where type `A` is needed. It'll always fail if invoked at runtime.
### Predicates
Predicate is a function with a boolean result type.
```go
type Predicate[A any] func(A) bool
```
- `fun.IsEqualTo[A comparable](a A) Predicate[A]` - IsEqualTo compares two arguments for equality.
- `fun.Not[A any](p Predicate[A]) Predicate[A]` - Not negates the given predicate.
## Option
A convenient data structure `Option[A]` that provides safe mechanisms to work with a potentially empty value.
- `option.None[A any]() Option[A]` - None constructs an option without value.
- `option.Some[A any](a A) Option[A]` - Some constructs an option with value.
- `option.Map[A any, B any](oa Option[A], f func(A) B) Option[B]` - Map applies a function to the value inside option if any.
- `option.Match[A any, B any](oa Option[A], f func(A) B, g func() B) (b B)` - Match transforms all possible values of OptionA using two provided functions.
- `option.Filter[A any](oa Option[A], predicate func(A) bool) Option[A]` - Filter leaves the value inside option only if predicate is true.
- `option.FlatMap[A any, B any](oa Option[A], f func(A) Option[B]) Option[B]` - FlatMap converts an internal value if it is present using the provided function.
- `option.Flatten[A any](ooa Option[Option[A]]) Option[A]` - Flatten simplifies option of option to just Option[A].
- `option.Get[A any](oa Option[A]) A` - Get is an unsafe function that unwraps the value from the option.
- `option.ForEach[A any](oa Option[A], f func(A))` - ForEach runs the given function on the value if it's available.
- `option.IsDefined[A any](oa Option[A]) bool` - IsDefined checks whether the option contains a value.
- `option.IsEmpty[A any](oa Option[A]) bool` - IsEmpty checks whether the option is empty.
## Either
Data structure that models sum type. It can contain either A or B.
- `either.Either[A any, B any]` - type that represents either A or B.
For `Either` there are a few helper functions:
- `either.Left[A any, B any](a A) Either[A, B]`
- `either.Right[A any, B any](b B) Either[A, B]`
- `either.IsLeft[A any, B any](eab Either[A, B]) bool`
- `either.IsRight[A any, B any](eab Either[A, B]) bool`
- `either.Fold[A any, B any, C any](eab Either[A, B], left func(A)C, right func(B)C) C` - Fold pattern matches Either with two given pattern match handlers.
- `either.GetLeft[A any, B any](eab Either[A, B]) option.Option[A]` - GetLeft returns left if it's defined.
- `either.GetRight[A any, B any](eab Either[A, B]) option.Option[B]` - GetRight returns left if it's defined.
## IO
IO encapsulates a calculation and provides a mechanism to compose a few calculations (flat map or bind).
### Error handling
An arbitrary calculation may either return good result that one might expect, or fail with an error or even panic. In Go a recommended pattern is to represent failure explicitly in the function signature by returning both result and an error. There is a convention that says that when an error is not `nil`, the result should not be used.
While the requirement to explicitly deal with errors helps implementing robust systems a lot, it is often very verbose and it advocates the bad practice of explicit control flow via `return`:
```go
a, err = foo()
if err != nil {
return
}
```
Here a single semantically important action (`foo`) requires 4 lines of code, a branch and a return statement.
This style is also not very friendly to function composition. If you need to pass the result of `foo` further to `bar`, you'll have to first bow to error handling ceremony.
The composition of two consequtive calculations is fundamental to programming. There is even a mathematical model that studies the properties of composition of calculations.
From error handling perspective `IO[A]` provides the following features:
- encapsulates a calculation that may return `A` or might fail;
- execution through `io.UnsafeRunSync`, `io.RunSync`, or `io.ObtainResult` recovers panics that cross that run boundary and returns them as errors;
- provides convenient mechanisms for composing consequtive calculations (`io.Map`, `io.FlatMap`).
`IO[A]` is an exported Go function type, so directly calling an `IO` value is an ordinary function call and does not establish a recovering run boundary. Code that starts its own goroutines is likewise responsible for recovering or publishing panics from those goroutines. Library-started fibers execute their work through `io.UnsafeRunSync`, so a work panic is observable as an error from `Fiber.Join`.
### Interaction with outer world vs simple (pure) functions/calculations.
From compiler's perspective things that happen in the program can be either ordinary pure computations or modification of some state outside of the function. Pure computation is special, because it has the following benefits:
- one can execute the same computation and receive exactly the same results;
- except obtaining the result of the computation nothing is changed elsewhere;
- it's much easier to reason about what is happening in the program made of pure computations (because nothing is happening apart from the computation itself).
The ability to understand and reason about programs is crucial to the ability of creation of somewhat complex programs.
Unfortunately all these nice and desired properties break when there are so called "side effects" - change of state, outer world interaction, ... - all things that make the computation to produce a different effect (and probably return different function results) even being called with the same arguments.
`IO[A]` provides a mechanism to arrange these side-effectful computations in such a way that it's easier to predict what is happening in the program. The main feature is delaying actual effect execution until an explicit run boundary. `io.Eval`, `io.Delay`, `io.Pure`, `io.LiftFunc`, `io.Map`, `io.FlatMap`, `io.ForEach`, and `io.Async` registration defer their user functions until the returned IO is executed. It's often possible to construct a whole large computation and only then execute it.
### Construction
To construct an IO one may use the following functions:
- `io.Lift[A any](a A) IO[A]` - lifts a plain value to IO
- `io.LiftFunc[A any, B any](f func(A) B) func(A) IO[B]` - LiftFunc converts `f` into a function whose application constructs a lazy IO. The original function is invoked only when that IO is executed. A panic from `f` is returned as an error when execution uses a recovering run boundary such as `io.UnsafeRunSync`.
- `io.Fail[A any](err error) IO[A]` - lifts an error to IO
- `io.FromConstantGoResult[A any](gr GoResult[A]) IO[A]` - FromConstantGoResult converts an existing GoResult value into an IO. Important! This is not for normal delayed IO execution. It cannot provide any guarantee for the moment when this go result was evaluated in the first place. This is just a combination of Lift and Fail.
- `io.Eval[A any](func () (A, error)) IO[A]` - lifts and delays an arbitrary computation. Panics crossing a recovering run boundary are represented as errors.
- `io.FromPureEffect(f func())IO[fun.Unit]` - FromPureEffect constructs IO from the simplest function signature.
- `io.Delay[A any](f func()IO[A]) IO[A]` - represents a function as a plain IO
- `io.Fold[A any, B any](io IO[A], f func(a A)IO[B], recover func (error)IO[B]) IO[B]` - handles both happy and sad paths.
- `io.Recover[A any](io IO[A], recover func(err error)IO[A])IO[A]` - handle only sad path and recover some errors to happy path.
- `io.OnError[A any](io IO[A], onError func(err error) IO[fun.Unit]) IO[A]` - OnError executes a side effect when there is an error.
- `io.Retry[A any, S any](ioa IO[A], strategy func(s S, err error) IO[option.Option[S]], zero S) IO[A]` - Retry performs the same operation a few times based on the retry strategy.
- `io.RetryS[A any, S any](ioa IO[A], strategy func(s S, err error) IO[option.Option[S]], zero S) IO[fun.Pair[A, S]]` - RetryS performs the same operation a few times based on the retry strategy. Also returns the last state of the error-handling strategy.
- `io.RetryStrategyMaxCount(substring string) func(s int, err error) IO[option.Option[int]]` - RetryStrategyMaxCount is a strategy that retries n times immediately.
### Manipulation
The following functions could be used to manipulate computations:
- `io.FlatMap[A any, B any](ioa IO[A], f func(A)IO[B]) IO[B]`
- `io.AndThen[A any, B any](ioa IO[A], iob IO[B]) IO[B]` - AndThen runs the first IO, ignores it's result and then runs the second one.
- `io.Map[A any, B any](ioA IO[A], f func(a A) B) IO[B]`
- `io.MapErr[A any, B any](ioA IO[A], f func(a A) (B, error)) IO[B]`
- `io.MapConst[A any, B any](ioA IO[A], b B) IO[B]` - MapConst ignores the result and replaces it with the given constant.
- `io.Sequence[A any](ioas []IO[A]) (res IO[[]A])`
- `io.SequenceUnit(ious []IO[Unit]) (res IOUnit)`
- `io.Unptr[A any](ptra *A) IO[A]` - retrieves the value at pointer. Fails if nil
- `io.Wrapf[A any](io IO[A], format string, args...interface{}) IO[A]` - wraps an error with additional context information
- `io.Finally[A any](io IO[A], finalizer IO[fun.Unit]) IO[A]` - Finally runs the finalizer regardless of the success of the IO. In case finalizer fails as well, the second error is printed to log.
- `io.Ignore[A any](ioa IO[A]) IOUnit` - Ignore throws away the result of IO.
- `io.MapSlice[A any, B any](ioas IO[[]A], f func(a A) B) IO[[]B]` - MapSlice converts each element of the slice inside IO[[]A] using the provided function that cannot fail.
To and from `GoResult` - allows to handle both good value and an error:
- `io.FoldToGoResult[A any](io IO[A]) IO[GoResult[A]]` - FoldToGoResult converts either value or error to go result. It should never fail.
- `io.UnfoldGoResult[A any](iogr IO[GoResult[A]]) IO[A]` - UnfoldGoResult represents GoResult back to ordinary IO.
Sometimes there is a need to perform some sideeffectful operation on a value. This can be achieved with `Consumer[A]`.
```go
// Consumer can receive an instance of A and perform some operation on it.
type Consumer[A any] func(A) IOUnit
```
- `io.CoMap[A any, B any](ca Consumer[A], f func(b B) A) Consumer[B]` - CoMap changes the input argument of the consumer.
### Execution
To run a constructed computation synchronously, use `UnsafeRunSync` or `RunSync`:
- `io.UnsafeRunSync[A any](ioa IO[A])` - executes the IO and returns its value and error; panics crossing this boundary are recovered as errors.
- `io.RunSync[A any](io IO[A]) GoResult[A]` - executes through `UnsafeRunSync` and returns the outcome as `GoResult`.
- `io.ObtainResult[A any](c Continuation[A]) (A, error)` - lower-level interpreter boundary for explicit continuation chains; it also recovers panics crossing the boundary.
`io.ForEach[A any](io IO[A], cb func(a A)) IO[fun.Unit]` is not an execution boundary. It constructs another lazy IO that invokes the callback only after the source IO succeeds and the returned IO is executed.
### Auxiliary functions
- `io.Memoize[A comparable, B any](f func(a A) IO[B]) func(A) IO[B]` - Memoize returns a function that will remember the original function in a map. It's thread safe, however, not super performant.
### Implementation details
IO might be implemented in various ways. Here we implement IO using continuations. A simple step in the constructed IO program might either complete (returning a result or an error), or return a continuation—another execution of the same kind. In order to obtain a result, the continuation is executed by the run boundary.
`ObtainResult` evaluates explicit continuation chains iteratively. The current composition combinators are not a Cats-style single bind interpreter: left-associated `FlatMap`, `Map`, and `Sequence` programs can enter nested `ObtainResult` calls. This project supports and tests these composition forms at a bounded depth of 10000, including failure propagation. This is a practical compatibility guarantee, not a claim of unlimited stack safety; callers should avoid assuming arbitrarily deep composition is safe.
- `type Continuation[A any] func() ResultOrContinuation[A]` - Continuation represents some multistep computation. Here `ResultOrContinuation[A]` is either a final result (value or error) or another continuation.
- `io.ObtainResult[A any](c Continuation[A]) (res A, err error)` - ObtainResult executes an explicit continuation chain until a final result is obtained. The mutable `io.MaxContinuationDepth` variable limits continuation-function invocations; its current default is 1,000,000 and each execution snapshots it once. A final result on the last allowed invocation succeeds. Zero or negative values execute no continuation functions and return a limit error. Nil initial or intermediate continuations return an error.
`io.MaxContinuationDepth` remains an exported mutable variable for compatibility. Configure it before starting concurrent IO execution and do not mutate it concurrently: external writes are unsynchronized and can race with the snapshot read at a run boundary. The limit is a safety ceiling, not fairness, yielding, or cancellation.
In the current composition architecture, `MapErr`, `FlatMap`, and `Fold` may start nested `ObtainResult` interpreter executions. Each such interpreter execution has one stable limit snapshot, but a concurrent external write may affect a later nested execution. Eliminating nested interpreters requires the larger instruction-tree runtime redesign.
## Resources
Resource is a thing that could only be used inside brackets - acquire/release.
```go
type Resource[A any]
```
The only allowed way to use the resource is through `Use`:
- `resource.Use[A any, B any](res Resource[A], f func(A) io.IO[B]) io.IO[B]` - Use is a only way to access the resource instance. It guarantees that the resource instance will be closed after use regardless of the failure/success result.
`ClosableIO` is a simple resource that implements Close method:
```go
type ClosableIO interface {
Close() io.IOUnit
}
```
- `resource.FromClosableIO[A ClosableIO](ioa io.IO[A]) Resource[A]` - FromClosableIO constructs a new resource from some value that itself supports method Close.
- `resource.BoundedExecutionContextResource(size int, queueLimit int) Resource[io.ExecutionContext]` - BoundedExecutionContextResource returns a resource that is a bounded execution context.
- `resource.Fail[A any](err error) Resource[A]` - Fail creates a resource that will fail during acquisition.
## Transaction-like resources
- `transaction.Bracket[A any, T any](acquire io.IO[T], commit func(T) io.IOUnit, rollback func(T) io.IOUnit) func(tr func(t T) io.IO[A]) io.IO[A]` - Bracket executes user computation with transactional guarantee. If user computation is successful - commit is executed. Otherwise - rollback.
## Parallel computing
Running work is observed using the `Fiber[A]` interface. A fiber publishes one terminal observation: either the underlying work result or observation shutdown, whichever wins first.
```go
type Fiber[A any] interface {
// Join waits for results of the fiber.
// When work completes before observation is closed, Join returns its result.
// When Close wins first, current and future joins fail with ErrorFiberClosed.
Join() IO[A]Frequently Asked Questions
What is goio?
goio is a Stream Processing library for the Go programming language. An implementation of IO, Stream, Fiber for Golang, inspired by awesome Scala libraries cats and fs2
How do I install goio?
Install goio with the Go module system using `go get primetalk/goio`. Check the repository for the current installation instructions.
What category does goio belong to?
goio is listed under Stream Processing, specifically Stream Processing.