crewai-go

About crewai-go

This large README is shown as a plain-text preview. Read the full README at the source

# crewai-go

> **Languages:** **English** (current) · [Português](README.pt-BR.md)

[![Go Reference](https://pkg.go.dev/badge/github.com/rhgs/crewai-go.svg)](https://pkg.go.dev/github.com/rhgs/crewai-go)
[![CI](https://github.com/rhgs/crewai-go/actions/workflows/ci.yml/badge.svg)](https://github.com/rhgs/crewai-go/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/rhgs/crewai-go/graph/badge.svg)](https://codecov.io/gh/rhgs/crewai-go)
[![Release](https://img.shields.io/github/v/release/rhgs/crewai-go?label=release)](https://github.com/rhgs/crewai-go/releases)
[![Go Version](https://img.shields.io/github/go-mod/go-version/rhgs/crewai-go)](https://github.com/rhgs/crewai-go/blob/main/go.mod)
[![Last Commit](https://img.shields.io/github/last-commit/rhgs/crewai-go)](https://github.com/rhgs/crewai-go/commits)
[![Zero Dependencies](https://img.shields.io/badge/dependencies-zero-success)](https://pkg.go.dev/github.com/rhgs/crewai-go)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/rhgs/crewai-go/blob/main/LICENSE)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)

<p align="center">
  <img src="gopher.jpg" alt="crewai-go cover" width="600">
</p>

**Orchestration of autonomous, collaborative AI agents in Go.**

`crewai-go` is an idiomatic Go port of the [CrewAI](https://github.com/crewAIInc/crewAI) framework. It lets you assemble teams (_crews_) of agents with distinct roles that collaborate — sequentially or hierarchically — to complete complex tasks using large language models (LLMs).

> Built with **zero external dependencies** — only the Go standard library. Easy to install, audit, and integrate.

---

## Table of Contents

- [Why crewai-go](#why-crewai-go)
- [Concepts](#concepts)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [LLM Providers](#llm-providers)
- [Tools](#tools)
- [Native tool calling](#native-tool-calling)
- [Web search](#web-search)
- [Logging](#logging)
- [Processes: Sequential, Hierarchical, and Staged](#processes-sequential-hierarchical-and-staged)
- [Agentic loop](#agentic-loop)
- [Structured output](#structured-output)
- [Guardrails](#guardrails)
- [Facts & provenance](#facts--provenance)
- [MCP](#mcp-model-context-protocol)
- [Flows](#flows)
- [Progress & warnings](#progress--warnings)
- [Memory](#memory)
- [Examples](#examples)
- [Documentation](#documentation)
- [Tests](#tests)
- [Comparison with CrewAI (Python)](#comparison-with-crewai-python)
- [Compatibility](#compatibility)
- [v1 scope (by design)](#v1-scope-by-design)
- [Contributing](#contributing)
- [License](#license)

---

## Why crewai-go

- 🧩 **Simple, composable API** — `Agent`, `Task`, `Crew`, `Tool`, `LLM`, plus typed `Flow[S]`.
- ⚡ **No dependencies** — stdlib only; small, fast builds.
- 🔌 **Any LLM** — OpenAI (and compatible: Ollama, Groq, Azure…), Anthropic (Claude), or your own implementation of the `LLM` interface.
- 🛠️ **Tools via ReAct** — agents reason and call tools in plain text.
- 📋 **Structured output** — tasks can require JSON validated against a JSON Schema, with a bounded repair loop.
- 🛡️ **Guardrails** — code-enforced post-output validation that blocks publication of outputs violating business invariants.
- 📌 **Facts & provenance** — first-class Fact type populated only by deterministic connector tools, never by the LLM, with full provenance metadata.
- 🔧 **Native tool calling** — use provider-native function calling (OpenAI, Anthropic, Ollama) instead of text-based ReAct, with automatic fallback and full trace observability.
- 🔎 **Web search** — agent-driven search via the `WebSearcher` interface (Ollama, OpenAI, Anthropic, xAI) or model-driven search via `WebSearchTool` with 7 providers (Wikipedia, LangSearch, Serpstack, DuckDuckGo, Google, Brave). SSRF-protected.
- 📝 **Structured logging via `log/slog`** — inject a custom `*slog.Logger` on `Crew` and `Agent`, with backward-compatible `Verbose` fallback.
- ⚡ **Async waves (v0.6)** — independent `Task.Async` tasks overlap under
  Sequential/Hierarchical; `Task.Context` is the DAG; **declaration-order fold
  after each wave barrier** (never completion order). `NewCrew` defaults
  `AsyncMaxWorkers = 8`.
- 🧱 **Semantic-race safe parallelism** — Staged/Async groups are independent
  mid-flight; barrier + slot-by-index merge; Memory AutoSave commits at the
  barrier in declaration order (D-M7). `-race` is necessary, not sufficient —
  see [docs/concurrency.md](docs/concurrency.md).
- 🧠 **Memory** between tasks and chainable **context** — pluggable
  `MemoryStore`, durable `FileStore` (JSONL), optional embeddings + cosine recall.
  Prefer `WithContext` to merge parallel siblings; Memory inject sees the
  **committed** snapshot only.
- 👔 **Hierarchical process** with a manager that delegates dynamically.
- 🪜 **Staged process** — stages in sequence, tasks within a stage in parallel;
  same barrier/fold contract as Async waves.
- 🔁 **Agentic loop** — optional Plan-Execute-Evaluate-Refine cycle with self-evaluation and iterative refinement.
- 🔌 **MCP** — connect to Model Context Protocol servers and expose their tools as `crewai.Tool` (schema-preserving).
- 📡 **Progress & warnings** — real-time `WithProgress` callbacks and per-task non-fatal warnings.
- 🌊 **Streaming** — optional `StreamingLLM` + `Crew.WithStream` for final-answer token deltas (ReAct/native no-tools); Task/Agent demux on Async waves.
- 🔀 **Flows** — typed `Flow[S]` event-driven runner (`Start` / `Listen` / `Router`) that composes crews as steps without replacing Sequential/Hierarchical/Staged.
- 📊 **Lifecycle events** — `WithEvents` / `CrewEvent` metadata telemetry (llm_call, react turns, repairs) alongside Progress.
- ✅ **Testable** — mock LLM included; ~90% core coverage.

## Concepts

| Concept     | What it is                                                              |
|--------------|-------------------------------------------------------------------------|
| **Agent**    | A worker with a role, a goal, a backstory, an LLM, and tools.           |
| **Task**     | A unit of work with a description, expected output, and an assignee.   |
| **Crew**     | The team: groups agents and tasks and orchestrates them.                |
| **Process**  | Execution strategy: `Sequential` (`DAG` alias), `Hierarchical`, or `Staged`. |
| **Tool**     | A capability an agent can invoke (calculation, search, API…).          |
| **LLM**      | Abstraction over the language model. Several providers ready to use.   |
| **Memory**   | Short-term bag + pluggable `MemoryStore` (FileStore, embeddings).      |
| **StructuredOutput** | Configures a task to require JSON validated against a JSON Schema. |
| **Guardrail** | Post-output validation hook that blocks publication of invalid outputs. |
| **Fact** | Data from a deterministic connector tool with provenance (source, hash). |
| **FactSource** | Optional interface for tools that produce Facts. |
| **ToolMode** | Tool execution strategy: `"react"` (default) or `"native"`. |
| **ToolCallingLLM** | Optional LLM interface for native function calling. |
| **StreamingLLM** | Optional LLM interface for token/delta streaming (`CallStream`). |
| **StreamChunk** | One streaming unit: `Delta`, `Task`, `Agent`, `Done`, `Err`. |
| **CrewEvent** | Metadata lifecycle record for `WithEvents` telemetry. |
| **Flow[S]** | Event-driven runner over typed state (`Start` / `Listen` / `Router`). |
| **ToolTrace** | Records each native tool invocation (name, args, output, duration). |

## Installation

Requires **Go 1.24+**.

```bash
go get github.com/rhgs/crewai-go@latest
```

In your code:

```go
import "github.com/rhgs/crewai-go"
```

## Quick Start

```go
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/rhgs/crewai-go"
	"github.com/rhgs/crewai-go/llm/openai"
)

func main() {
	// 1. Pick an LLM (uses OPENAI_API_KEY from the environment).
	llm := openai.New("gpt-4o-mini")

	// 2. Create an agent.
	poet := crewai.NewAgent(
		"Poet",
		"Write short, memorable poems",
		"You are an award-winning poet, master of brevity.",
		llm,
	)

	// 3. Define a task.
	task := crewai.NewTask(
		"Write a haiku about the Go programming language.",
		"A haiku (3 lines) in English.",
		poet,
	)

	// 4. Assemble the crew and run it.
	crew := crewai.NewCrew([]*crewai.Agent{poet}, []*crewai.Task{task})
	out, err := crew.Kickoff(context.Background(), nil)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(out.Final)
}
```

```bash
export OPENAI_API_KEY=sk-...
go run .
```

## LLM Providers

Any type implementing the interface below works:

```go
type LLM interface {
	Call(ctx context.Context, messages []Message) (string, error)
	Model() string
}
```

Ready to use:

```go
import (
	"github.com/rhgs/crewai-go/llm/openai"
	"github.com/rhgs/crewai-go/llm/anthropic"
	"github.com/rhgs/crewai-go/llm/ollama"
	"github.com/rhgs/crewai-go/llm/xai"
	"github.com/rhgs/crewai-go/llm/mock" // for tests
)

// OpenAI (and compatible: Groq, Azure, Together…)
llm := openai.New("gpt-4o-mini")

// Anthropic (Claude)
llm := anthropic.New("claude-sonnet-5")

// Ollama local (no key)
llm := ollama.New("llama3.2")

// Ollama Cloud (uses OLLAMA_API_KEY)
llm := ollama.NewCloud("gpt-oss:120b")

// xAI (Grok) via API key
llm := xai.New("grok-4")

// xAI (Grok) via subscription OAuth (SuperGrok / X Premium) — no per-token key
df := xai.NewDeviceFlow(clientID)
ts, _ := xai.LoadTokenSource("~/.crewai-xai-token.json", df)
llm := xai.NewWithOAuth("grok-4", ts)
```

| Provider | Package | Authentication |
|----------|--------|----------------|
| OpenAI (and compatible) | `llm/openai` | `OPENAI_API_KEY` / `WithTokenSource` |
| Anthropic (Claude) | `llm/anthropic` | `ANTHROPIC_API_KEY` |
| Ollama local | `llm/ollama` (`New`) | none |
| Ollama Cloud | `llm/ollama` (`NewCloud`) | `OLLAMA_API_KEY` |
| xAI (Grok) — API key | `llm/xai` (`New`) | `XAI_API_KEY` |
| xAI (Grok) — subscription | `llm/xai` (`NewWithOAuth`) | OAuth Device Flow |
| Mock (tests) | `llm/mock` | none |

See [`docs/llms.md`](docs/llms.md) for all options.

## Tools

Create a tool from any Go function:

```go
search := crewai.NewTool(
	"web_search",
	"Searches the web for a term. Input: the search term.",
	func(ctx context.Context, term string) (string, error) {
		// ... your logic ...
		return result, nil
	},
)

agent.WithTools(search)
```

Built-in tools in the `tools` package:

```go
import "github.com/rhgs/crewai-go/tools"

agent.WithTools(
	tools.Calculator(),        // evaluates arithmetic expressions
	tools.CurrentTime(""),      // current date/time
	tools.WordCount(),         // counts words/characters
	tools.NewHTTPFetch(tools.WithHTTPAllowlist("api.example.com")),
	tools.NewFileRead("/var/data"),
)
```

The agent uses tools via the **ReAct** protocol (`Thought → Action → Action Input → Observation → Final Answer`). Details in [`docs/tools.md`](docs/tools.md).

## Native tool calling

For providers that support native function calling (OpenAI, Anthropic, Ollama), set `Agent.ToolMode` to use the provider's built-in tool calling instead of text-based ReAct:

```go
llm := ollama.New("llama3.2")

agent := crewai.NewAgent("Researcher", "Find answers", "You are a researcher.", llm)
agent.ToolMode = crewai.ToolModeNative
agent.WithTools(
    tools.Calculator(),
    tools.CurrentTime(""),
)

task := crewai.NewTask("What is 15% of 200?", "A short answer.", agent)
crew := crewai.NewCrew([]*crewai.Agent{agent}, []*crewai.Task{task})
out, _ := crew.Kickoff(context.Background(), nil)

// ToolTraces record each native tool call for observability.
for _, trace := range out.TasksOutput[0].ToolTraces {
    fmt.Printf("%s(%s) -> %s\n", trace.Tool, string(trace.Args), trace.Output)
}
```

When `ToolMode` is `"native"` but the LLM does not implement `ToolCallingLLM`, the executor returns `ErrNativeToolsUnsupported`. The default (`""` or `"react"`) uses the existing ReAct loop — no changes to existing code. Details in [`docs/tools.md`](docs/tools.md).

## Web search

Web search is available in two patterns — agent-driven (Go code controls queries) and model-driven (the LLM decides when to search via the ReAct loop).

### Agent-driven: `WebSearcher` interface

For LLM providers that have a native web search API (Ollama Cloud, OpenAI, Anthropic, xAI), call `SearchWeb` directly from Go code:

```go
import "github.com/rhgs/crewai-go"

// OpenAI requires a search-capable model (e.g. gpt-4o-search-preview).
llm := openai.New("gpt-4o-search-preview")

hits, err := crewai.SearchWeb(ctx, llm, "Go programming language", 5)
if err != nil {
    // Returns ErrWebSearchUnsupported if the LLM doesn't implement WebSearcher.
    log.Fatal(err)
}
for _, hit := range hits {
    fmt.Printf("%s -- %s\n%s\n\n", hit.Title, hit.URL, hit.Content)
}
```

| Provider | How it works |
|----------|-------------|
| **Ollama Cloud** | `POST /api/web_search` — pure search endpoint, no model invocation, no tokens consumed |
| **OpenAI** | `web_search_options` in Chat Completions (NOT `tools`); requires search models (`gpt-4o-search-preview`, `gpt-5-search-api`); results as nested `url_citation` annotations |
| **Anthropic** | `web_search_20250305` server tool; results as `web_search_tool_result` blocks with `encrypted_content` |
| **xAI (Grok)** | Delegates to the OpenAI-compatible client |

### Model-driven: `WebSearchTool`

For the ReAct loop, use `WebSearchTool` so the LLM decides when to search. It implements both `Tool` and `FactSource`, so results are collected as `Fact`s with provenance.

```go
import "github.com/rhgs/crewai-go/tools"

// Wikipedia (default, free, no API key) — searches Wikipedia articles only.
search := tools.NewWebSearch(nil)

// LangSearch (100% free, semantic summaries) — general web search.
// Get a free key at https://langsearch.com
search := tools.NewWebSearch(tools.NewLangSearch("LANGSEARCH_API_KEY"))

// Serpstack (1000 free searches/month) — Google SERP data.
search := tools.NewWebSearch(tools.NewSerpstack("SERPSTACK_API_KEY"))

// Google Custom Search (requires API key + CSE ID).
search := tools.NewWebSearch(tools.NewGoogleSearch("GOOGLE_API_KEY", "GOOGLE_CSE_ID"))

// Brave Search (requires API key).
search := tools.NewWebSearch(tools.NewBraveSearch("BRAVE_API_KEY"))

// DuckDuckGo (no key, but may be blocked by captcha).
search := tools.NewWebSearch(tools.NewDuckDuckGoSearch())

agent.WithTools(search)
```

All search results are **SSRF-protected**: URLs pointing to localhost, private/CGNAT/multicast IPs, `0.0.0.0`, link-local addresses (`169.254.x`), metadata aliases, userinfo, and unspecified addresses are filtered out. Domain names are resolved via DNS to prevent DNS rebinding attacks (fail-closed).

Details in [`docs/llms.md`](docs/llms.md) (WebSearcher) and [`docs/tools.md`](docs/tools.md) (WebSearchTool).

## Logging

crewai-go uses [`log/slog`](https://pkg.go.dev/log/slog) (structured logging, Go 1.21+) from the standard library. Every log call is structured (key-value pairs), not format strings.

Inject a custom `*slog.Logger` via `WithLogger` on both `Crew` and `Agent`:

```go
log := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
    Level: slog.LevelDebug,
}))

crew := crewai.NewCrew(agents, tasks).WithLogger(log)
agent := crewai.NewAgent("a", "g", "b", llm).WithLogger(log)
```

If no logger is injected, `Kickoff` creates a text-format logger on stderr. The default level depends on `Crew.Verbose`:

| `Verbose` | Default level | Effect                                                  |
|-----------|---------------|---------------------------------------------------------|
| `false`   | `LevelError`  | Only warnings and errors (matches legacy nop behavior). |
| `true`    | `LevelDebug`  | Everything: debug, info, warn, error.                   |

When `WithLogger` is used, the injected logger is used as-is — the caller controls the level and handler.

To mask likely secrets in log attributes and messages (best-effort, opt-in):

```go
log := slog.New(crewai.RedactHandler(
    slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{Level: slog.LevelInfo}),
))
crew := crewai.NewCrew(agents, tasks).WithLogger(log)
```

See [`examples/logging/`](examples/logging/) for a full wiring sample. Prefer `LevelInfo` (or higher) in production; `LevelDebug` can include full LLM output and tool args.

Example log line (JSON handler):

```json
{"time":"...","level":"INFO","msg":"agent thought","agent":"Poet","output":"Final Answer: ..."}
```

Subpackages (`llm/*`, `tools/*`) do not log internally — they return errors that the executor logs at the appropriate level.

### Logging safety

> Debug-level logs include full LLM output (`agent thought` → the entire model response) and tool inputs/arguments. If your log destination is shared (e.g. remote log aggregator), the output may contain PII or proprietary model responses. Provider errors (logged at `WARN` on delegation failures, for example) may include API keys in their message.
>
> Mitigations:
>
> - Pick your destination accordingly (sink to local files, not a shared stream, when handling user data).
> - Use a level filter (e.g. `LevelError` only) to keep secrets out of logs by default.
> - Wrap your handler with a redactor. See [`examples/logging/`](examples/logging/) for a drop-in redaction handler that masks likely-secrets (API keys, bearer tokens, long alphanumeric tokens).

### Thread-safety

`WithLogger` is **not concurrent-safe**. Set the logger before calling `Kickoff`/`Execute` and do not mutate it concurrently. Multiple sequential calls to `WithLogger` are idempotent — the last call wins.

```go
log := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
    Level: slog.LevelDebug,
}))

crew := crewai.NewCrew(agents, tasks).WithLogger(log)
agent := crewai.NewAgent("a", "g", "b", llm).WithLogger(log)
```

If no logger is injected, `Kickoff` creates a text-format logger on stderr. The default level depends on `Crew.Verbose`:

| `Verbose` | Default level | Effect                                                  |
|-----------|---------------|---------------------------------------------------------|
| `false`   | `LevelError`  | Only warnings and errors (matches legacy nop behavior). |
| `true`    | `LevelDebug`  | Everything: debug, info, warn, error.                   |

When `WithLogger` is used, the injected logger is used as-is — the caller controls the level and handler. `Agent.Execute` (standalone, without a crew) falls back to `slog.Default()` unless `Agent.WithLogger` is set.

Example log line (JSON handler):

```json
{"time":"...","level":"INFO","msg":"agent thought","agent":"Poet","output":"Final Answer: ..."}
```

Subpackages (`llm/*`, `tools/*`) do not log internally — they return errors that the executor logs at the appropriate level.

## Processes: Sequential, Hierarchical, and Staged

**Sequential** — tasks in order, each output becomes context for the next:

```go
crew := crewai.NewCrew(agents, tasks)
crew.Process = crewai.Sequential
```

**Hierarchical** — a manager delegates each task to the most suitable agent:

```go
crew.Process = crewai.Hierarchical
crew.ManagerLLM = llm // or crew.ManagerAgent = myManager
```

**Staged** — stages run in sequence, but the tasks within a single stage run
concurrently. The output of each stage is available as context to the tasks of
the following stages. A stage marked `Optional` does not abort the crew when
one of its tasks fails; otherwise the first failure aborts `Kickoff`.

```go
crew := crewai.NewCrew(agents, nil)
crew.Process = crewai.Staged
crew.Stages = []crewai.Stage{
    {Name: "collect", Tasks: []*crewai.Task{researchA, researchB}},
    {Name: "synthesize", Tasks: []*crewai.Task{write}},
}
```

Chain context explicitly with `WithContext`:

```go
analysis := crewai.NewTask("Analyze the data", "insights", analyst).
	WithContext(collection) // receives the output of the 'collection' task
```

Frequently Asked Questions