goai

About goai

This large README is shown as a plain-text preview. Read the full README at the source

<p align="center">
  <img src="goai.png" alt="GoAI" width="400">
</p>

<h1 align="center">GoAI</h1>

<p align="center"><em>AI SDK, the Go way.</em></p>
<p align="center">Go SDK for building AI applications. One SDK, 25+ providers, MCP support.</p>

<p align="center">
  <a href="bench/RESULTS.md"><img src="https://img.shields.io/badge/streaming-1.1x_faster-brightgreen" alt="Streaming"></a>
  <a href="bench/RESULTS.md"><img src="https://img.shields.io/badge/cold_start-24x_faster-brightgreen" alt="Cold Start"></a>
  <a href="bench/RESULTS.md"><img src="https://img.shields.io/badge/memory-3.1x_less-brightgreen" alt="Memory"></a>
</p>

<p align="center"><strong>1.1x faster streaming</strong>, <strong>24x faster cold start</strong>, <strong>3.1x less memory</strong> vs Vercel AI SDK (<a href="bench/RESULTS.md">benchmarks</a>)</p>

<p align="center">
  <a href="https://goai.sh">Website</a> &middot;
  <a href="https://goai.sh/getting-started/installation">Docs</a> &middot;
  <a href="https://goai.sh/architecture">Architecture</a> &middot;
  <a href="https://goai.sh/providers/">Providers</a> &middot;
  <a href="https://goai.sh/examples">Examples</a> &middot;
  <a href="https://github.com/zendev-sh/zenflow">Orchestration (zenflow)</a>
</p>

---

Inspired by the [Vercel AI SDK](https://sdk.vercel.ai). The same clean abstractions, idiomatically adapted for Go with generics, interfaces, and functional options.

## Selected Releases

> **v0.9.0** - Provider-neutral file upload and remote file references. `FileUploader` interface + `RemoteFileRef` for OpenAI (Files API), Anthropic (Files API), and Google Gemini (Files API). Compat providers map file parts to native OpenAI shapes. [Changelog →](https://github.com/zendev-sh/goai/releases)
>
> **v0.8.5** - New provider: [Requesty](https://goai.sh/providers/requesty) (OpenAI-compatible LLM gateway). Plus Anthropic native structured output, per-step `Reasoning`, and Gemini schema sanitization fixes. [Changelog →](https://github.com/zendev-sh/goai/releases)
>
> **v0.8.4** - `WithRetryObserver` callback for observing retry attempts, split tool-call stream metadata fix, and Anthropic `redacted_thinking` capture in streaming. [Changelog →](https://github.com/zendev-sh/goai/releases)
>
> **v0.8.3** - Native Ollama provider (`/api/chat` + `/api/embed`, think mode, no third-party SDK) and duplicate tool name detection. [Changelog →](https://github.com/zendev-sh/goai/releases)
>
> **v0.8.0** - MCP OAuth 2.1 + PKCE for remote servers, `NewTool[In]` typed constructor, and lifecycle hook panics surfaced as `*PanicError` via `WithOnPanic` (breaking). [Changelog →](https://github.com/zendev-sh/goai/releases)

## Features

- **8 core functions**: `GenerateText`, `StreamText`, `GenerateObject[T]`, `StreamObject[T]`, `Embed`, `EmbedMany`, `GenerateImage`, `GenerateVideo`
- **25+ providers**: OpenAI, Anthropic, Google, Bedrock, Azure, Vertex, Mistral, xAI, Groq, Cohere, DeepSeek, MiniMax, Fireworks, Together, DeepInfra, OpenRouter, Requesty, Perplexity, Cerebras, Ollama, vLLM, RunPod, Cloudflare Workers AI, FPT Smart Cloud, NVIDIA NIM, Heabsy, Opper, llama.cpp, + generic OpenAI-compatible
- **Auto tool loop**: Define tools with `Execute` handlers, set `MaxSteps` for `GenerateText` and `StreamText`
- **Structured output**: `GenerateObject[T]` auto-generates JSON Schema from Go types via reflection
- **Streaming**: Real-time text and partial object streaming via channels
- **Dynamic auth**: `TokenSource` interface for OAuth, rotating keys, cloud IAM, with `CachedTokenSource` for TTL-based caching
- **Prompt caching**: Automatic cache control for supported providers (Anthropic, Bedrock, MiniMax)
- **Citations/sources**: Grounding and inline citations from xAI, Perplexity, Google, OpenAI
- **Web search**: Built-in web search tools for OpenAI, Anthropic, Google, Groq, xAI. Model decides when to search
- **Code execution**: Server-side Python sandboxes via OpenAI, Anthropic, Google. No local setup
- **Computer use**: Anthropic computer, bash, text editor tools for autonomous desktop interaction
- **26 provider-defined tools**: Web fetch, file search, image generation, X search, and more - [full list](#provider-defined-tools)
- **MCP client**: Connect to any MCP server (stdio, HTTP, SSE), auto-convert tools for use with GoAI
- **Observability**: Built-in Langfuse and OpenTelemetry (OTel) integrations for tracing generations, tools, and multi-step loops
- **Multi-agent orchestration**: For declarative YAML workflows on top of GoAI, see [zenflow](https://github.com/zendev-sh/zenflow)
- **10 lifecycle hooks**: Observability (`OnRequest`, `OnResponse`, `OnToolCallStart`, `OnToolCall`, `OnStepFinish`, `OnFinish`, `OnPanic`) and interceptor (`OnBeforeToolExecute`, `OnAfterToolExecute`, `OnBeforeStep`) hooks for permission gates, secret scanning, output transformation, and loop control
- **Retry/backoff**: Automatic retry with exponential backoff on retryable HTTP errors (429/5xx)
- **Minimal dependencies**: Core depends on `golang.org/x/oauth2` + one indirect (`cloud.google.com/go/compute/metadata`). Optional `observability/otel` and `observability/langfuse` submodules each use their own separate `go.mod`.

## Performance vs Vercel AI SDK

| Metric               | GoAI   | Vercel AI SDK | Improvement |
| -------------------- | ------ | ------------- | ----------- |
| Streaming throughput | 1.46ms | 1.62ms        | 1.1x faster |
| Cold start           | 569us  | 13.9ms        | 24x faster  |
| Memory (1 stream)    | 220KB  | 676KB         | 3.1x less   |
| GenerateText         | 56us   | 79us          | 1.4x faster |

> Mock HTTP server, identical SSE fixtures, Apple M2. [Full report](bench/RESULTS.md)

## Install

```bash
go get github.com/zendev-sh/goai@latest
```

Requires Go 1.25+.

## Quick Start

Most hosted providers auto-resolve API keys from environment variables. Local/custom providers may require explicit options:

```go
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/zendev-sh/goai"
	"github.com/zendev-sh/goai/provider/openai"
)

func main() {
	// Reads OPENAI_API_KEY from environment automatically.
	model := openai.Chat("gpt-4o")

	result, err := goai.GenerateText(context.Background(), model,
		goai.WithPrompt("What is the capital of France?"),
	)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(result.Text)
}
```

## Streaming

```go
ctx := context.Background()

stream, err := goai.StreamText(ctx, model,
	goai.WithSystem("You are a helpful assistant."),
	goai.WithPrompt("Write a haiku about Go."),
)
if err != nil {
	log.Fatal(err)
}

for text := range stream.TextStream() {
	fmt.Print(text)
}

result := stream.Result()
if err := stream.Err(); err != nil {
	log.Fatal(err)
}
fmt.Printf("\nTokens: %d in, %d out\n",
	result.TotalUsage.InputTokens, result.TotalUsage.OutputTokens)
```

OpenAI Responses streams enforce a five-minute provider-event idle timeout.
Configure this per model with `openai.WithResponsesStreamIdleTimeout`; pass `0`
to explicitly disable the watchdog. Premature EOF and a bare `[DONE]` sentinel
are rejected by default. Non-standard endpoints that require `[DONE]` can opt
in explicitly with `openai.WithResponsesStreamDoneCompatibility(true)`; this
option changes only terminal-sentinel handling and does not relax event schema
validation. Every JSON data event must identify its type through either the JSON
`type` property or the SSE `event:` field, and malformed recognized events
terminate the stream. Endpoints should use SSE comments, not untyped data
frames, for keepalives. Idle and protocol interruptions are exposed as typed
`openai.StreamIdleTimeoutError` and `openai.StreamProtocolError` values.

Streaming with tools:

```go
import "github.com/zendev-sh/goai/provider"

stream, err := goai.StreamText(ctx, model,
	goai.WithPrompt("What's the weather in Tokyo?"),
	goai.WithTools(weatherTool),
	goai.WithMaxSteps(5),
)
for chunk := range stream.Stream() {
	switch chunk.Type {
	case provider.ChunkText:
		fmt.Print(chunk.Text)
	case provider.ChunkStepFinish:
		fmt.Println("\n[step complete]")
	}
}
```

## Structured Output

Auto-generates JSON Schema from Go types. Works with OpenAI, Anthropic, and Google.

```go
type Recipe struct {
	Name        string   `json:"name" jsonschema:"description=Recipe name"`
	Ingredients []string `json:"ingredients"`
	Steps       []string `json:"steps"`
	Difficulty  string   `json:"difficulty" jsonschema:"enum=easy|medium|hard"`
}

result, err := goai.GenerateObject[Recipe](ctx, model,
	goai.WithPrompt("Give me a recipe for chocolate chip cookies"),
)
if err != nil {
	log.Fatal(err)
}
fmt.Printf("Recipe: %s (%s)\n", result.Object.Name, result.Object.Difficulty)
```

Streaming partial objects:

```go
stream, err := goai.StreamObject[Recipe](ctx, model,
	goai.WithPrompt("Give me a recipe for pancakes"),
)
if err != nil {
	log.Fatal(err)
}
for partial := range stream.PartialObjectStream() {
	fmt.Printf("\r%s (%d ingredients so far)", partial.Name, len(partial.Ingredients))
}
final, err := stream.Result()
```

## Tools

Define a tool from a typed input struct with `goai.NewTool` - the JSON Schema is
generated from the struct and arguments are unmarshaled for you. Set `MaxSteps`
to enable the auto tool loop.

```go
weatherTool := goai.NewTool("get_weather", "Get the current weather for a city.",
	func(ctx context.Context, args struct {
		City string `json:"city" jsonschema:"description=City name"`
	}) (string, error) {
		return fmt.Sprintf("22°C and sunny in %s", args.City), nil
	})

result, err := goai.GenerateText(ctx, model,
	goai.WithPrompt("What's the weather in Tokyo?"),
	goai.WithTools(weatherTool),
	goai.WithMaxSteps(3),
)
if err != nil {
	log.Fatal(err)
}
fmt.Println(result.Text) // "It's 22°C and sunny in Tokyo."
```

## MCP (Model Context Protocol)

Connect to any MCP server and use its tools with GoAI. Supports stdio, Streamable HTTP, and legacy SSE transports.

```go
import "github.com/zendev-sh/goai/mcp"

// Connect to any MCP server
transport := mcp.NewStdioTransport("npx", []string{"-y", "@modelcontextprotocol/server-filesystem", "."})
client := mcp.NewClient("my-app", "1.0", mcp.WithTransport(transport))
_ = client.Connect(ctx)
defer client.Close()

// Use MCP tools with GoAI
tools, _ := client.ListTools(ctx, nil)
goaiTools := mcp.ConvertTools(client, tools.Tools)

result, _ := goai.GenerateText(ctx, model,
    goai.WithTools(goaiTools...),
    goai.WithPrompt("List files in the current directory"),
    goai.WithMaxSteps(5),
)
```

For OAuth-protected remote servers, GoAI handles authorization-server discovery, dynamic client registration, PKCE, token refresh, and retry-on-401 natively — no external wrapper needed:

```go
ts, _ := mcp.NewOAuthTokenSource(ctx, mcp.OAuthConfig{
    ServerURL:   serverURL,
    RedirectURI: "http://127.0.0.1:8765/callback",
    Authorize:   openBrowserAndAwaitRedirect, // returns the OAuth redirect URL
})
transport := mcp.NewHTTPTransport(serverURL, mcp.WithHTTPClient(mcp.NewOAuthHTTPClient(ts, nil)))
client := mcp.NewClient("my-app", "1.0", mcp.WithTransport(transport))
```

See [examples/mcp-tools](examples/mcp-tools/), [examples/mcp-oauth](examples/mcp-oauth/), and the [MCP documentation](https://goai.sh/concepts/mcp) for more.

## Companion Projects

**[zenflow](https://github.com/zendev-sh/zenflow)** - multi-agent orchestration engine for Go. Declarative YAML workflows, typed inter-agent mailboxes, single static binary. Built on GoAI for the LLM layer; reach for it when one `MaxSteps` loop is no longer enough.

## Citations / Sources

Providers that support grounding (Google, xAI, Perplexity) or inline citations (OpenAI) return sources:

```go
result, err := goai.GenerateText(ctx, model,
	goai.WithPrompt("What were the major news events today?"),
)
if err != nil {
	log.Fatal(err)
}

if len(result.Sources) > 0 {
	for _, s := range result.Sources {
		fmt.Printf("[%s] %s - %s\n", s.Type, s.Title, s.URL)
	}
}

// Sources are also available per-step in multi-step tool loops.
for _, step := range result.Steps {
	for _, s := range step.Sources {
		fmt.Printf("  Step source: %s\n", s.URL)
	}
}
```

## Computer Use

See [Provider-Defined Tools > Computer Use](#computer-use-1) and [examples/computer-use](examples/computer-use/) for Anthropic computer, bash, and text editor tools. Works with both Anthropic direct API and Bedrock.

## Embeddings

```go
ctx := context.Background()
model := openai.Embedding("text-embedding-3-small")

// Single
result, err := goai.Embed(ctx, model, "Hello world")
if err != nil {
	log.Fatal(err)
}
fmt.Printf("Dimensions: %d\n", len(result.Embedding))

// Batch (auto-chunked, parallel)
many, err := goai.EmbedMany(ctx, model, []string{"foo", "bar", "baz"},
	goai.WithMaxParallelCalls(4),
)
if err != nil {
	log.Fatal(err)
}
```

## Image Generation

```go
ctx := context.Background()
model := openai.Image("gpt-image-1")

result, err := goai.GenerateImage(ctx, model,
	goai.WithImagePrompt("A sunset over mountains, oil painting style"),
	goai.WithImageSize("1024x1024"),
)
if err != nil {
	log.Fatal(err)
}
os.WriteFile("sunset.png", result.Images[0].Data, 0644)
```

Also supported: Google Imagen (`google.Image("imagen-4.0-generate-001")`) and Vertex AI (`vertex.Image(...)`).

## Video Generation

Google Veo video generation uses a long-running operation. GoAI starts the operation, polls until completion, and downloads the resulting video bytes.

```go
ctx := context.Background()
model := google.Video("veo-3.1-generate-preview")

result, err := goai.GenerateVideo(ctx, model,
	goai.WithVideoPrompt("A paper plane taking flight at sunrise"),
	goai.WithVideoAspectRatio("16:9"),
	goai.WithVideoResolution("720p"),
	goai.WithVideoDuration(8*time.Second),
	goai.WithVideoAudio(true),
)
if err != nil {
	log.Fatal(err)
}
os.WriteFile("plane.mp4", result.Video.Data, 0644)
```

Image-to-video is supported with `WithVideoImage`. First/last frames and image references are available through `WithVideoFrameImages` and `WithVideoInputReferences`.

## Observability

Built-in [Langfuse](https://langfuse.com) and [OpenTelemetry](https://opentelemetry.io) integrations. Ten lifecycle hooks cover the full generation pipeline -- observability providers use them to trace LLM calls, tool executions, panics, and multi-step agent loops:

```go
import "github.com/zendev-sh/goai/observability/langfuse"

// Credentials from env: LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST
result, err := goai.GenerateText(ctx, model,
    goai.WithPrompt("Hello"),
    goai.WithTools(weatherTool),
    goai.WithMaxSteps(5),
    langfuse.WithTracing(langfuse.TraceName("my-agent")),
)
```

Interceptor hooks let you control tool execution without modifying core code:

```go
// Permission gate: block dangerous tools
goai.WithOnBeforeToolExecute(func(info goai.BeforeToolExecuteInfo) goai.BeforeToolExecuteResult {
    if info.ToolName == "delete_file" {
        return goai.BeforeToolExecuteResult{Skip: true, Result: "Permission denied."}
    }
    return goai.BeforeToolExecuteResult{}
}),

// Detect max-steps exhaustion
goai.WithOnFinish(func(info goai.FinishInfo) {
    if info.StepsExhausted {
        log.Printf("Loop exhausted after %d steps", info.TotalSteps)
    }
}),
```

See [examples/hooks](examples/hooks/), [examples/langfuse](examples/langfuse/), [examples/otel](examples/otel/), and the [observability docs](https://goai.sh/concepts/observability) for details.

## Providers

Many providers auto-resolve credentials from environment variables. Others (for example `ollama`, `vllm`, `compat`) use explicit options:

```go
// Auto-resolved: reads OPENAI_API_KEY from env
model := openai.Chat("gpt-4o")

// Explicit key (overrides env)
model := openai.Chat("gpt-4o", openai.WithAPIKey("sk-..."))

// Cloud IAM auth (Vertex, Bedrock)
model := vertex.Chat("gemini-2.5-pro",
	vertex.WithProject("my-project"),
	vertex.WithLocation("us-central1"),
)

// AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION from env)
model := bedrock.Chat("anthropic.claude-sonnet-4-6-v1:0")

// Local (Ollama, vLLM)
model := ollama.Chat("llama3", ollama.WithBaseURL("http://localhost:11434"))

result, err := goai.GenerateText(ctx, model, goai.WithPrompt("Hello"))
```

### Provider Table

| Provider   | Chat                                                         | Embed                                                      | Image         | Auth                                                                                               | E2E  | Import                |
| ---------- | ------------------------------------------------------------ | ---------------------------------------------------------- | ------------- | -------------------------------------------------------------------------------------------------- | ---- | --------------------- |
| OpenAI     | `gpt-4o`, `o3`, `codex-*`                                    | `text-embedding-3-*`                                       | `gpt-image-1` | `OPENAI_API_KEY`, `OPENAI_BASE_URL`, TokenSource                                                   | Full | `provider/openai`     |
| Anthropic  | `claude-*`                                                   | -                                                          | -             | `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, TokenSource                                             | Full | `provider/anthropic`  |
| Google     | `gemini-*`                                                   | `text-embedding-004`                                       | `imagen-*`    | `GOOGLE_GENERATIVE_AI_API_KEY` / `GEMINI_API_KEY`, TokenSource                                     | Full | `provider/google`     |
| Bedrock    | `anthropic.*`, `meta.*`                                      | `titan-embed-*`, `cohere.embed-*`, `nova-2-*`, `marengo-*` | -             | AWS keys, `AWS_BEARER_TOKEN_BEDROCK`, `AWS_BEDROCK_BASE_URL`                                       | Full | `provider/bedrock`    |
| Vertex     | `gemini-*`                                                   | `text-embedding-004`                                       | `imagen-*`    | TokenSource, ADC, or `GOOGLE_API_KEY` / `GEMINI_API_KEY` / `GOOGLE_GENERATIVE_AI_API_KEY` fallback | Unit | `provider/vertex`     |
| Azure      | `gpt-4o`, `claude-*`                                         | -                                                          | via Azure     | `AZURE_OPENAI_API_KEY`, TokenSource                                                                | Full | `provider/azure`      |
| OpenRouter | various                                                      | -                                                          | -             | `OPENROUTER_API_KEY`, TokenSource                                                                  | Unit | `provider/openrouter` |
| Mistral    | `mistral-large`, `magistral-*`                               | -                                                          | -             | `MISTRAL_API_KEY`, TokenSource                                                                     | Full | `provider/mistral`    |
| Groq       | `mixtral-*`, `llama-*`                                       | -                                                          | -             | `GROQ_API_KEY`, TokenSource                                                                        | Full | `provider/groq`       |
| xAI        | `grok-*`                                                     | -                                                          | -             | `XAI_API_KEY`, TokenSource                                                                         | Unit | `provider/xai`        |
| Cohere     | `command-r-*`                                                | `embed-*`                                                  | -             | `COHERE_API_KEY`, TokenSource                                                                      | Unit | `provider/cohere`     |

Frequently Asked Questions