Library for adding support for interacting and monitoring Celery workers, tasks and events in Go.

# Celeriac

Golang client library for adding support for interacting and monitoring Celery workers and tasks.

It provides functionality to place tasks on the task queue, as well as monitor both task and worker events.

## Dependencies

This library depends upon the following packages:

- github.com/rabbitmq/amqp091-go
- github.com/sirupsen/logrus
- github.com/nu7hatch/gouuid
- github.com/mailru/easyjson

## Install `easyjson`

```bash
$ go get -u github.com/mailru/easyjson/...
```

## Usage

Installation: `go get github.com/svcavallar/celeriac.v1`

This imports a new namespace called `celeriac`

```go
package main

import (
	"log"
	"os"

	"github.com/svcavallar/celeriac.v1"
)

func main() {
	taskBrokerURI := "amqp://user:pass@localhost:5672/vhost"

	// Connect to RabbitMQ task queue
	TaskQueueMgr, err := celeriac.NewTaskQueueMgr(taskBrokerURI)
	if err != nil {
		log.Printf("Failed to connect to task queue: %v", err)
		os.Exit(-1)
	}

	log.Printf("Service connected to task queue - (URL: %s)", taskBrokerURI)

	// Go routine to monitor the Celery events emitted on the celeriac events channel
	go func() {
        for {
            select {
            default:
                ev := <-TaskQueueMgr.Monitor.EventsChannel

                if ev != nil {

                    if x, ok := ev.(*celeriac.WorkerEvent); ok {
                        log.Printf("Celery Event Channel: Worker event - %s [Hostname]: %s", x.Type, x.Hostname)
                    } else if x, ok := ev.(*celeriac.TaskEvent); ok {
                        log.Printf("Celery Event Channel: Task event - %s [ID]: %s", x.Type, x.UUID)
                    } else if x, ok := ev.(*celeriac.Event); ok {
                        log.Printf("Celery Event Channel: General event - %s [Hostname]: %s - [Data]: %v", x.Type, x.Hostname, x.Data)
                    } else {
                        log.Printf("Celery Event Channel: Unhandled event: %v", ev)
                    }
                }
            }
        }
	}()
}
```

## Dispatching Tasks

### By Name
This will create and dispatch a task incorporating the supplied data. The task will automatically be allocated and identified by a UUID returned in the task object. The UUID is represented in the form of "6ba7b810-9dad-11d1-80b4-00c04fd430c8".

	// Dispatch a new task
	taskName := "root.test.task"
	taskData := map[string]interface{}{
		"foo": "bar"
	}
	routingKey := "root.test"

	task, err := TaskQueueMgr.DispatchTask(taskName, taskData, routingKey)
	if err != nil {
		log.Errorf("Failed to dispatch task to queue: %v", err)
	}


### By ID & Name
This will create and dispatch a task incorporating the supplied data, and identified by the user-supplied task identifier.

	// Dispatch a new task
	taskID := "my_task_id_123456789"
	taskName := "root.test.task"
	taskData := map[string]interface{}{
		"foo": "bar"
	}
	routingKey := "root.test"

	task, err := TaskQueueMgr.DispatchTaskWithID(taskID, taskName, taskData, routingKey)
	if err != nil {
		log.Errorf("Failed to dispatch task to queue: %v", err)
	}

## Modifying `task_event.go`

If you modify the properties of any the structs in `task_event.go` you will need to re-generate the `easyjson` version of this file. This is easily achieved by issuing the following command:

```bash
$ easyjson -all task_eventtest.go
```
## Processing Redis Backend Result Automatically

If you are using a Redis backend for storing results you can easily process new/updated entries by subscribing to Redis keyspace events. 
This will save polling for results, and is made convenient to integrate by using my golang helper package `go-redis-event-sink`, available at the repo [https://github.com/svcavallar/go-redis-event-sink](https://github.com/svcavallar/go-redis-event-sink)

An example on how to use this is provided within the repository. Essentially, just provide it with the Celery task naming mask patten to watch: `celery-task-meta-*`

celeriac

Library for enabling asynchronous dependency health checks in your service.

[![LICENSE](https://img.shields.io/badge/license-MIT-orange.svg)](LICENSE)
[![Build Status](https://travis-ci.org/InVisionApp/go-health.svg?branch=master)](https://travis-ci.org/InVisionApp/go-health)
[![Maintainability](https://api.codeclimate.com/v1/badges/973b603c7f6ad3a59d0a/maintainability)](https://codeclimate.com/github/InVisionApp/go-health/maintainability)
[![Test Coverage](https://api.codeclimate.com/v1/badges/973b603c7f6ad3a59d0a/test_coverage)](https://codeclimate.com/github/InVisionApp/go-health/test_coverage)
[![Go Report Card](https://goreportcard.com/badge/github.com/InVisionApp/go-health)](https://goreportcard.com/report/github.com/InVisionApp/go-health)
[![Godocs](https://img.shields.io/badge/golang-documentation-blue.svg)](https://godoc.org/github.com/InVisionApp/go-health)

<img align="right" src="images/go-health.svg" width="200">

# go-health
A library that enables *async* dependency health checking for services running on an orchestrated container platform such as kubernetes or mesos.

## Why is this important?
Container orchestration platforms require that the underlying service(s) expose a "health check" which is used by the platform to determine whether the container is in a good or bad state.

While this can be achieved by simply exposing a `/status` endpoint that performs synchronous checks against its dependencies (followed by returning a `200` or `non-200` status code), it is not optimal for a number of reasons:

* **It does not scale**
    + The more dependencies you add, the longer your health check will take to complete (and potentially cause your service to be killed off by the orchestration platform).
    + Depending on the complexity of a given dependency, your check may be fairly involved where it is _okay_ for it to take `30s+` to complete.
* **It adds unnecessary load on yours deps or at worst, becomes a DoS target**
    + **Non-malicious scenario**
        + Thundering herd problem -- in the event of a deployment (or restart, etc.), all of your service containers are likely to have their `/status` endpoints checked by the orchestration platform as soon as they come up. Depending on the complexity of the checks, running that many simultaneous checks against your dependencies could cause at worst the dependencies to experience problems and at minimum add unnecessary load.
        + Security scanners -- if your organization runs periodic security scans, they may hit your `/status` endpoint and trigger unnecessary dep checks.
    + **Malicious scenario**
        + Loading up any basic HTTP benchmarking tool and pointing it at your `/status` endpoint could choke your dependencies (and potentially your service).

With that said, not everyone _needs_ asynchronous checks. If your service has one dependency (and that is unlikely to change), it is trivial to write a basic, synchronous check and it will probably suffice.

However, if you anticipate that your service will have several dependencies, with varying degrees of complexity for determining their health state - you should probably think about introducing asynchronous health checks.

## How does this library help?
Writing an async health checking framework for your service is not a trivial task, especially if Go is not your primary language.

This library:

* Allows you to define how to check your dependencies.
* Allows you to define warning and fatal thresholds.
* Will run your dependency checks on a given interval, in the background. **[1]**
* Exposes a way for you to gather the check results in a *fast* and *thread-safe* manner to help determine the final status of your `/status` endpoint. **[2]**
* Comes bundled w/ [pre-built checkers](/checkers) for well-known dependencies such as `Redis`, `Mongo`, `HTTP` and more.
* Makes it simple to implement and provide your own checkers (by adhering to the checker interface).
* Allows you to trigger listener functions when your health checks fail or recover using the [`IStatusListener` interface](#oncomplete-hook-vs-istatuslistener).
* Allows you to run custom logic when a specific health check completes by using the [`OnComplete` hook](#oncomplete-hook-vs-istatuslistener).

**[1]** Make sure to run your checks on a "sane" interval - ie. if you are checking your
Redis dependency once every five minutes, your service is essentially running _blind_
for about 4.59/5 minutes. Unless you have a really good reason, check your dependencies
every X _seconds_, rather than X _minutes_.

**[2]** `go-health` continuously writes dependency health state data and allows
you to query that data via `.State()`. Alternatively, you can use one of the
pre-built HTTP handlers for your `/healthcheck` endpoint (and thus not have to
manually inspect the state data).

## Example

For _full_ examples, look through the [examples dir](examples/)

1. Create an instance of `health` and configure a checker (or two)

```golang
import (
	health "github.com/InVisionApp/go-health/v2"
	"github.com/InVisionApp/go-health/v2/checkers"
	"github.com/InVisionApp/go-health/v2/handlers"
)

// Create a new health instance
h := health.New()

// Create a checker
myURL, _ := url.Parse("https://google.com")
myCheck, _ := checkers.NewHTTP(&checkers.HTTPConfig{
    URL: myURL,
})
```

2. Register your check with your `health` instance

```golang
h.AddChecks([]*health.Config{
    {
        Name:     "my-check",
        Checker:  myCheck,
        Interval: time.Duration(2) * time.Second,
        Fatal:    true,
    },
})
```

3. Start the health check

```golang
h.Start()
```

From here on, you can either configure an endpoint such as `/healthcheck` to use a built-in handler such as `handlers.NewJSONHandlerFunc()` or get the current health state of all your deps by traversing the data returned by `h.State()`.

## Sample /healthcheck output
Assuming you have configured `go-health` with two `HTTP` checkers, your `/healthcheck`
output would look something like this:

```json
{
    "details": {
        "bad-check": {
            "name": "bad-check",
            "status": "failed",
            "error": "Ran into error while performing 'GET' request: Get google.com: unsupported protocol scheme \"\"",
            "check_time": "2017-12-30T16:20:13.732240871-08:00"
        },
        "good-check": {
            "name": "good-check",
            "status": "ok",
            "check_time": "2017-12-30T16:20:13.80109931-08:00"
        }
    },
    "status": "ok"
}
```

## Additional Documentation
* [Examples](/examples)
  * [Status Listeners](/examples/status-listener)
  * [OnComplete Hook](/examples/on-complete-hook)
* [Checkers](/checkers)

## OnComplete Hook VS IStatusListener
At first glance it may seem that these two features provide the same functionality. However, they are meant for two different use cases:

The `IStatusListener` is useful when you want to run a custom function in the event that the overall status of your health checks change. I.E. if `go-health` is currently checking the health for two different dependencies A and B, you may want to trip a circuit breaker for A and/or B. You could also put your service in a state where it will notify callers that it is not currently operating correctly. The opposite can be done when your service recovers.

The `OnComplete` hook is called whenever a health check for an individual dependency is complete. This means that the function you register with the hook gets called every single time `go-health` completes the check. It's completely possible to register different functions with each configured health check or not to hook into the completion of certain health checks entirely. For instance, this can be useful if you want to perform cleanup after a complex health check or if you want to send metrics to your APM software when a health check completes. It is important to keep in mind that this hook effectively gets called on roughly the same interval you define for the health check.

## Contributing
All PR's are welcome, as long as they are well tested. Follow the typical fork->branch->pr flow.


go-health

Lightweight, high performance messaging system for microservices, IoT, and cloud native systems.

<p align="center">
  <img src="logos/nats-horizontal-color.png" width="300" alt="NATS Logo">
</p>

[NATS](https://nats.io) is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Foundation ([CNCF](https://cncf.io)). NATS has over [40 client language implementations](https://nats.io/download/), and its server can run on-premise, in the cloud, at the edge, and even on a Raspberry Pi. NATS can secure and simplify design and operation of modern distributed systems.

[![License][License-Image]][License-Url] [![Build][Build-Status-Image]][Build-Status-Url] [![Release][Release-Image]][Release-Url] [![Slack][Slack-Image]][Slack-Url] [![Coverage][Coverage-Image]][Coverage-Url] [![Docker Downloads][Docker-Image]][Docker-Url] [![CII Best Practices][CIIBestPractices-Image]][CIIBestPractices-Url] [![Artifact Hub][ArtifactHub-Image]][ArtifactHub-Url]

## Documentation

- [Official Website](https://nats.io)
- [Official Documentation](https://docs.nats.io)
- [FAQ](https://docs.nats.io/reference/faq)
- Watch [a video overview](https://rethink.synadia.com/episodes/1/) of NATS.
- Watch [this video from SCALE 13x](https://www.youtube.com/watch?v=sm63oAVPqAM) to learn more about its origin story and design philosophy.

## Contact

- [Twitter](https://twitter.com/nats_io): Follow us on Twitter!
- [Google Groups](https://groups.google.com/forum/#!forum/natsio): Where you can ask questions
- [Slack](https://natsio.slack.com): Click [here](https://slack.nats.io) to join. You can ask question to our maintainers and to the rich and active community.

## Contributing

If you are interested in contributing to NATS, read about our...

- [Contributing guide](./CONTRIBUTING.md)
- [Report issues or propose Pull Requests](https://github.com/nats-io)

[License-Url]: https://www.apache.org/licenses/LICENSE-2.0
[License-Image]: https://img.shields.io/badge/License-Apache2-blue.svg
[Docker-Image]: https://img.shields.io/docker/pulls/_/nats.svg
[Docker-Url]: https://hub.docker.com/_/nats
[Slack-Image]: https://img.shields.io/badge/chat-on%20slack-green
[Slack-Url]: https://slack.nats.io
[Fossa-Url]: https://app.fossa.io/projects/git%2Bgithub.com%2Fnats-io%2Fnats-server?ref=badge_shield
[Fossa-Image]: https://app.fossa.io/api/projects/git%2Bgithub.com%2Fnats-io%2Fnats-server.svg?type=shield
[Build-Status-Url]: https://travis-ci.com/github/nats-io/nats-server
[Build-Status-Image]: https://travis-ci.com/nats-io/nats-server.svg?branch=main
[Release-Url]: https://github.com/nats-io/nats-server/releases/latest
[Release-image]: https://img.shields.io/github/v/release/nats-io/nats-server
[Coverage-Url]: https://coveralls.io/r/nats-io/nats-server?branch=main
[Coverage-image]: https://coveralls.io/repos/github/nats-io/nats-server/badge.svg?branch=main
[ReportCard-Url]: https://goreportcard.com/report/nats-io/nats-server
[ReportCard-Image]: https://goreportcard.com/badge/github.com/nats-io/nats-server
[CIIBestPractices-Url]: https://bestpractices.coreinfrastructure.org/projects/1895
[CIIBestPractices-Image]: https://bestpractices.coreinfrastructure.org/projects/1895/badge
[ArtifactHub-Url]: https://artifacthub.io/packages/helm/nats/nats
[ArtifactHub-Image]: https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/nats
[github-release]: https://github.com/nats-io/nats-server/releases/

## Roadmap

The NATS product roadmap can be found [here](https://nats.io/about/#roadmap).

## Adopters

Who uses NATS? See our [list of users](https://nats.io/#who-uses-nats) on [https://nats.io](https://nats.io).

## Security

### Security Audit

A third party security audit was performed by Cure53, you can see the full report [here](https://github.com/nats-io/nats-general/blob/main/reports/Cure53_NATS_Audit.pdf).

### Reporting Security Vulnerabilities

If you've found a vulnerability or a potential vulnerability in the NATS server, please let us know at
[nats-security](mailto:security@nats.io).

## License

Unless otherwise noted, the NATS source files are distributed
under the Apache Version 2.0 license found in the LICENSE file.


NATS

Scalable, fault-tolerant application-layer sharding for Go applications.

ringpop-go [![Build Status](https://github.com/uber/ringpop-go/workflows/Test/badge.svg)](https://github.com/uber/ringpop-go/actions?query=workflow%3ATest) [![Coverage Status](https://coveralls.io/repos/uber/ringpop-go/badge.svg?branch=master&service=github)](https://coveralls.io/github/uber/ringpop-go?branch=master)
==========

**(This project is no longer under active development.)**

Ringpop is a library that brings cooperation and coordination to distributed
applications. It maintains a consistent hash ring on top of a membership
protocol and provides request forwarding as a routing convenience. It can be
used to shard your application in a way that's scalable and fault tolerant.

Getting started
---------------

To install ringpop-go:

```
go get github.com/uber/ringpop-go
```

Developing
----------

First make certain that `thrift` (OSX: `brew install thrift`) and `glide` are
in your path (above). Then,

```
make setup
```

to install remaining golang dependencies and install the pre-commit hook.

Finally, run the tests by doing:

```
make test
```

Documentation
--------------

Interested in where to go from here? Read the docs at
[ringpop.readthedocs.org](https://ringpop.readthedocs.org)


The Best Go Libraries For Distributed Systems - Packages that help with building Distributed Systems. (4)

celeriac

go-health

NATS

ringpop-go