rabtap
RabbitMQ swiss army knife cli app
About rabtap
This large README is shown as a plain-text preview. Read the full README at the source
# rabtap - RabbitMQ wire tap
[](https://coveralls.io/github/jandelgado/rabtap?branch=master)
[](https://goreportcard.com/report/github.com/jandelgado/rabtap)
Swiss army knife for RabbitMQ. Tap/Pub/Sub messages, create/delete/bind queues
and exchanges, inspect broker.
## Contents
<!-- vim-markdown-toc GFM -->
- [Features](#features)
- [Screenshots](#screenshots)
- [Show broker topology](#show-broker-topology)
- [Visualize broker topology with graphviz](#visualize-broker-topology-with-graphviz)
- [Tap messages](#tap-messages)
- [Installation](#installation)
- [Docker image](#docker-image)
- [Pre-compiled binaries](#pre-compiled-binaries)
- [Arch Linux](#arch-linux)
- [Installation from source](#installation-from-source)
- [Usage](#usage)
- [Command overview](#command-overview)
- [Broker URI specification](#broker-uri-specification)
- [Authentication](#authentication)
- [Environment variables](#environment-variables)
- [Default RabbitMQ broker](#default-rabbitmq-broker)
- [Default RabbitMQ management API endpoint](#default-rabbitmq-management-api-endpoint)
- [Default RabbitMQ TLS config](#default-rabbitmq-tls-config)
- [Colored output](#colored-output)
- [Command reference and examples](#command-reference-and-examples)
- [Broker info](#broker-info)
- [Wire-tapping messages](#wire-tapping-messages)
- [Tap all messages published or delivered (RabbitMQ FireHose)](#tap-all-messages-published-or-delivered-rabbitmq-firehose)
- [Replaying messages from the FireHose exchange](#replaying-messages-from-the-firehose-exchange)
- [Connect to multiple brokers](#connect-to-multiple-brokers)
- [Message recorder](#message-recorder)
- [Subscribe messages](#subscribe-messages)
- [Publish messages](#publish-messages)
- [Poor mans shovel](#poor-mans-shovel)
- [Close connection](#close-connection)
- [Exchange commands](#exchange-commands)
- [Queue commands](#queue-commands)
- [Format specification for tap and sub command](#format-specification-for-tap-and-sub-command)
- [JSON message format](#json-message-format)
- [Filtering output](#filtering-output)
- [Filtering expressions](#filtering-expressions)
- [Evaluation context](#evaluation-context)
- [Examples](#examples)
- [Type reference](#type-reference)
- [Exchange type](#exchange-type)
- [Queue type](#queue-type)
- [Binding type](#binding-type)
- [Connection type](#connection-type)
- [Channel type](#channel-type)
- [Message type](#message-type)
- [Build from source](#build-from-source)
- [Download and build using go install](#download-and-build-using-go-install)
- [Build using Makefile and tests](#build-using-makefile-and-tests)
- [Experimental WASM/wasip1 port](#experimental-wasmwasip1-port)
- [Test data generator](#test-data-generator)
- [Contributing](#contributing)
- [Author](#author)
- [Copyright and license](#copyright-and-license)
<!-- vim-markdown-toc -->
## Features
- tap to messages being sent to exchanges using RabbitMQ
exchange-to-exchange bindings without affecting actual message delivery (aka _tapping_)
- display broker related information using the
[RabbitMQ REST management API](https://rawcdn.githack.com/rabbitmq/rabbitmq-management/rabbitmq_v3_6_14/priv/www/api/index.html)
- save messages and meta data for later analysis and replay
- publish messages to exchanges
- consume messages from queues and streams (subscribe)
- filter output using flexible expressions
- supports TLS
- no runtime dependencies (statically linked golang single file binary)
- simple to use command line tool
- runs on Linux, Windows, Mac and wherever you can compile go
## Screenshots
### Show broker topology
Output of `rabtap info` command:
Output of `rabtap info --stats` command, showing additional statistics:
### Visualize broker topology with graphviz
Using the `--format=dot` option, the `info` command can generate output in the
[dot format](https://graphviz.org/doc/info/lang.html), which can be visualized
using graphviz, e.g. `rabtap info --show-default --format dot | dot -T svg >
mybroker.svg`. The resulting SVG file can be visualized with a web browser.
### Tap messages
Output of rabtap in `tap` mode, showing message meta data and the message body:
## Installation
### Docker image
A docker image is provided so rabtap can be used as a docker container, e.g.
```console
$ docker run --rm -ti ghcr.io/jandelgado/rabtap:latest
```
### Pre-compiled binaries
Pre-compiled binaries can be downloaded for multiple platforms from the
[releases page](https://github.com/jandelgado/rabtap/releases).
### Arch Linux
Rabtap can be installed from the Arch Linux User Repository (AUR):
```console
$ yay -S rabtap
```
### Installation from source
See the [build from source section](#build-from-source) if you prefer to
compile from source.
## Usage
```text
rabtap - RabbitMQ wire tap. github.com/jandelgado/rabtap
Usage:
rabtap info [--api=APIURI] [--consumers] [--stats] [--filter=EXPR] [--omit-empty]
[--show-default] [--mode=MODE] [--format=FORMAT] [TLSOPTIONS] [COMMON OPTIONS]
rabtap tap EXCHANGES [--uri=URI] [--saveto=DIR] [--format=FORMAT|--json] [--limit=NUM]
[--idle-timeout=DURATION] [--filter=EXPR] [--silent] [TLSOPTIONS] [COMMON OPTIONS]
rabtap (tap --uri=URI EXCHANGES)... [--saveto=DIR] [--format=FORMAT|--json] [--limit=NUM]
[--idle-timeout=DURATION] [--filter=EXPR] [--silent] [TLSOPTIONS] [COMMON OPTIONS]
rabtap sub QUEUE [--uri URI] [--saveto=DIR] [--format=FORMAT|--json] [--limit=NUM]
[--offset=OFFSET] [--args=KV]... [(--reject [--requeue])] [--silent]
[--filter=EXPR] [--idle-timeout=DURATION] [TLSOPTIONS] [COMMON OPTIONS]
rabtap pub [--uri=URI] [SOURCE] [--exchange=EXCHANGE] [--format=FORMAT|--json]
[--routingkey=KEY | (--header=KV)...] [ (--property=KV)... ] [--confirms]
[--mandatory] [--delay=DURATION | --speed=FACTOR] [TLSOPTIONS] [COMMON OPTIONS]
rabtap exchange create EXCHANGE [--uri=URI] [--type=TYPE] [--args=KV]...
[--autodelete] [--transient] [TLSOPTIONS] [COMMON OPTIONS]
rabtap exchange bind EXCHANGE to DESTEXCHANGE [--uri=URI]
(--bindingkey=KEY | (--header=KV)... (--all|--any)) [TLSOPTIONS] [COMMON OPTIONS]
rabtap exchange rm EXCHANGE [--uri=URI] [TLSOPTIONS] [COMMON OPTIONS]
rabtap queue create QUEUE [--uri=URI] [--queue-type=TYPE] [--args=KV]...
[--autodelete] [--transient] [--lazy] [TLSOPTIONS] [COMMON OPTIONS]
rabtap queue bind QUEUE to EXCHANGE [--uri=URI]
(--bindingkey=KEY | (--header=KV)... (--all|--any)) [TLSOPTIONS] [COMMON OPTIONS]
rabtap queue unbind QUEUE from EXCHANGE [--uri=URI]
(--bindingkey=KEY | (--header=KV)... (--all|--any)) [TLSOPTIONS] [COMMON OPTIONS]
rabtap queue rm QUEUE [--uri=URI] [TLSOPTIONS] [COMMON OPTIONS]
rabtap queue purge QUEUE [--uri=URI] [TLSOPTIONS] [COMMON OPTIONS]
rabtap conn close CONNECTION [--api=APIURI] [--reason=REASON] [TLSOPTIONS] [COMMON OPTIONS]
rabtap --version
rabtap (-h | --help | help) [properties]
Arguments and options:
EXCHANGES comma-separated list of exchanges and optional binding keys,
e.g. 'amq.topic:#' or 'exchange1:key1,exchange2:key2'
EXCHANGE name of an exchange, e.g. 'amq.direct'
DESTEXCHANGE name of a a destination exchange in an exchange-to-exchange binding
SOURCE file or directory to publish in pub mode. If omitted, stdin will be read
QUEUE name of a queue
CONNECTION name of a connection
DIR directory to read messages from
DURATION a numerical duration with a unit suffix like "ms", "s", "m", "h"
-a, --autodelete create auto delete exchange/queue
--all set x-match=all option in header based routing
--any set x-match=any option in header based routing
--api=APIURI connect to given API server. If APIURL is omitted, the environment
variable RABTAP_APIURI will be used
--args=KV A key value pair in the form of "key=value" passed as additional
arguments. e.g. '--args=x-queue-type=quorum'
-b, --bindingkey=KEY binding key to use in bind queue command
--by-connection output of info command starts with connections
--confirms enable publisher confirms and wait for confirmations
--consumers include consumers and connections in output of info command
--delay=DURATION Time to wait between sending messages during publish. If not set,
then messages will be delayed as recorded.
--exchange=EXCHANGE optional exchange to publish to. If omitted, exchange will be taken
from message being published (see JSON message format)
--filter=EXPR Predicate for sub, tap, info command to filter the output [default: true]
--format=FORMAT for tap, pub, sub command: format to write/read messages to console
and optionally to file (when --saveto DIR is given).
Valid options are: 'raw', 'json', 'json-nopp'. Default: 'raw'
for info command: controls generated output format. Valid options
are: 'text', 'dot'. Default: 'text'
-h, --help prints this help
--header=KV A key value pair in the form of "key=value" used as a routing- or
binding-key. Can occur multiple times
--idle-timeout=DURATION end reading messages when no new message was received for the
given duration
-j, --json deprecated. Use "--format=json" instead
--lazy create a lazy queue
--limit=NUM Stop afer NUM messages were received. When set to 0, will run until
terminated [default: 0]
--mandatory enable mandatory publishing (messages must be delivered to queue)
--mode=MODE mode for info command. One of 'byConnection', 'byExchange' [default: byExchange]
--omit-empty don't show echanges without bindings in info command
--offset=OFFSET Offset when reading from a stream. Can be 'first', 'last', 'next',
a DURATION like '10m', a RFC3339-Timestamp or an integer index value.
Basically it is an alias for '--args=x-stream-offset=OFFSET'
--property=KV A key value pair in the form of "key=value" to specify message properties
like e.g. the content-type.
--queue-type=TYPE type of queue [default: classic]
--reason=REASON reason why the connection was closed [default: closed by rabtap]
--reject Reject messages. Default behaviour is to acknowledge messages
--requeue Instruct broker to requeue rejected message
-r, --routingkey=KEY routing key to use in publish mode. If omitted, routing key
will be taken from message being published (see JSON message format)
--saveto=DIR also save messages and metadata to DIR
--show-default include default exchange in output info command
-s, --silent suppress message output to stdout
--speed=FACTOR Speed factor to use during publish [default: 1.0]
--stats include statistics in output of info command
-t, --type=TYPE type of exchange [default: fanout]
--transient create a transient exchange/queue (default is durable)
--uri=URI connect to given AQMP broker. If omitted, the environment variable
RABTAP_AMQPURI will be used
--version show version information and exit
Common options:
-c, --color force colored output
-n, --no-color don't colorize output (see also environment variable NO_COLOR)
-v, --verbose enable verbose mode
TLS options:
--tls-cert-file=CERTFILE A Cert file to use for client authentication
--tls-key-file=KEYFILE A Key file to use for client authentication
--tls-ca-file=CAFILE A CA Cert file to use with TLS
-k, --insecure allow insecure TLS connections (disables certificate check)
Examples:
rabtap tap --uri amqp://guest:guest@localhost/ amq.fanout:
rabtap tap --uri amqp://guest:guest@localhost/ amq.topic:#,amq.fanout:
rabtap pub --uri amqp://guest:guest@localhost/ --exchange amq.topic message.json --format=json
rabtap info --api http://guest:guest@localhost:15672/api
# use RABTAP_AMQPURI environment variable to specify broker instead of --uri
export RABTAP_AMQPURI=amqp://guest:guest@localhost:5672/
rabtap queue create JDQ
rabtap queue bind JDQ to amq.topic --bindingkey=key
echo "Hello"| gzip | rabtap pub --exchange amq.topic --routingkey "key" --property ContentType=gzip
rabtap sub JDQ
# print only messages that have ".Name == 'JAN'" in their JSON payload
rabtap sub JDQ --filter="let b=fromJSON(r.toStr(r.body(r.msg))); b.Name == 'JAN'"
rabtap queue rm JDQ
# use RABTAP_APIURI environment variable to specify mgmt api uri instead of --api
export RABTAP_APIURI=http://guest:guest@localhost:15672/api
rabtap info
rabtap info --filter "r.binding.Source == 'amq.topic'" --omit-empty
rabtap conn close "172.17.0.1:40874 -> 172.17.0.2:5672"
# use RABTAP_TLS_CERTFILE | RABTAP_TLS_KEYFILE | RABTAP_TLS_CAFILE environments variables
# instead of specifying --tls-cert-file=CERTFILE --tls-key-file=KEYFILE --tls-ca-file=CAFILE
```
### Command overview
Rabtap understands the following commands:
- `tap` - taps to an exchange and receives messages sent to the exchange,
without affecting actual message delivery (using an exchange-to-exchange
binding).
- `sub` - subscribes to a queue and consumes from the queue
- `pub` - publish messages to an exchange, optionally with the timing as recorded
- `info` - show broker related info (exchanges, queues, bindings, stats).
- `queue` - create,bind,unbind,remove or purge queues
- `exchange` - create or remove exchanges
- `conn` - close connections
See the below for detailed information.
### Broker URI specification
The specification of the RabbitMQ broker URI follows the [AMQP URI
specification](https://www.rabbitmq.com/uri-spec.html) as implemented by the
[go RabbitMQ client library](https://github.com/streadway/amqp).
Examples:
- `amqp://guest:guest@localhost:5672/`
- `amqps://guest:[email protected]:5671/`
- `amqps://guest:[email protected]:5671/vhost`
Note that according to [RFC3986](https://tools.ietf.org/html/rfc3986) it might be
necessary to escape certain characters like e.g. `?` (%3F) or `#` (%23) as otherwise
parsing of the URI may fail with an error.
#### Authentication
Authentication is either by the username and password provided in the broker
URI as desribed above (RabbitMQ `PLAIN` method), or by mTLS providing a client
certificate and key using the `--tls-key-file`, `--tls-cert-file` options (RabbitMQ
`EXTERNAL` method). If both mTLS and a username and password is provided, then
rabtap will use mTLS and `PLAIN` authentication with the given username and
password.
### Environment variables
Use environment variables to specify standard values for broker and api endpoint.
#### Default RabbitMQ broker
In cases where the URI argument is optional, e.g. `rabtap tap [-uri
URI] exchange ...`, the URI of the RabbitMQ broker can be set with the
environment variable `RABTAP_AMQPURI`. Example:
```console
$ export RABTAP_AMQPURI=amqp://guest:guest@localhost:5672/
$ rabtap tap amq.fanout:
...
```
#### Default RabbitMQ management API endpoint
The default RabbitMQ management API URI can be set using the `RABTAP_APIURI`
environment variable. Example:
```console
$ export RABTAP_APIURI=http://guest:guest@localhost:15672/api
$ rabtap info
...
```
The timeout for API requests against this endpoint can be configured using
the `RABTAP_HTTP_TIMEOUT` environment variable using a
[time.Duration](https://pkg.go.dev/time#ParseDuration) value, e.g.
`export RABTAP_HTTP_TIMEOUT=30s`. The default timeout is `10s`. Increase
the timeout when dealing with brokers with many queues and exchanges, since
otherwise `rabtap info` may run into timeouts.
#### Default RabbitMQ TLS config
The default TLS certificates path can be set using the
`RABTAP_TLS_CERTFILE` and `RABTAP_TLS_KEYFILE` and `RABTAP_TLS_CAFILE`
environments variables. All certificate and key files are expected in PEM
format. Example:
```console
$ export RABTAP_TLS_CERTFILE=/path/to/certs/user.crt
$ export RABTAP_TLS_KEYFILE=/path/to/certs/user.key
$ export RABTAP_TLS_CAFILE =/path/to/certs/ca.crt
$ echo "Hello" | rabtap pub --exchange amq.topic --routingkey "key"
...
```
#### Colored output
Output is colored, when writing to a terminal. This behaviour can be changed:
- set environment variable `NO_COLOR` to disable color output (or set
`--no-color` option)
- set `--color` option to force colored output
### Command reference and examples
The following examples assume a RabbitMQ broker running on `localhost:5672` and
the management API available on port `localhost:15672`. Easiest way to start
such an instance is by running `docker run -ti --rm -p 5672:5672 -p 15672:15672
rabbitmq:3-management` or similar command to start a RabbitMQ container.
#### Broker info
The `info` command uses the REST API of RabbitMQ to gather and display
topolgy related information from the broker.
The `--mode MODE` option controls how the output is structured. Valid options
for `MODE` are `byExchange` (default) or `byConnection`.
The `--format=FORMAT` option controls the format of generated output. Valid
options are `text` for console text format (default) or `dot` to output the
tree structure in dot format for visualization with graphviz.
The features of an exchange are displayed in square brackets with `D`
(durable), `AD` (auto delete) and `I` (internal). The features of a queue are
displayed in square brackets with `D` (durable), `AD` (auto delete) and `EX`
(exclusive).
If the `--stats` option is enabled, basic statistics are included in the
output.
The `--filter` option allows to filter output. See
[filtering](#filtering-output) section for details. Use the
`--by-connection` to sort output by connection (implies `--consumers`)
Examples (assume that `RABTAP_APIURI` environment variable is set):
- `rabtap info --consumers` - shows virtual hosts exchanges, queues and
consumers of given broker in a tree view (see [screenshot](#screenshots)).
- `rabtap info --mode=byConnection` - shows virtual hosts, connections,
consumers and queues of given broker in an tree view.
- `rabtap info --format=dot | dot -T svg > broker.svg` - renders broker info
into `dot` format and uses graphviz to render a SVG file for final
visualization.
#### Wire-tapping messages
The `tap` command allows to tap to exchanges and transparently receive the messages
sent to the exchanges. The general form of the tap command is either
```text
rabtap tap EXCHANGES [--uri=URI] [--saveto=DIR] [--format=FORMAT] [--limit=NUM]
[--idle-timeout=DURATION] [--filter=EXPR] [-jkncsv]
[(--tls-cert-file=CERTFILE --tls-key-file=KEYFILE)] [--tls-ca-file=CAFILE]
```
or, to connect to multiple brokers simultanously,
```text
rabtap (tap --uri=URI EXCHANGES)... [--saveto=DIR] [--format=FORMAT] [--limit=NUM]
[--idle-timeout=DURATION] [--filter=EXPR] [-jkncsv]Frequently Asked Questions
What is rabtap?
rabtap is a Messaging library for the Go programming language. RabbitMQ swiss army knife cli app
How do I install rabtap?
Install rabtap with the Go module system using `go get jandelgado/rabtap`. Check the repository for the current installation instructions.
What category does rabtap belong to?
rabtap is listed under Messaging, specifically Messaging.