pagoda

About pagoda

This large README is shown as a plain-text preview. Read the full README at the source

## Pagoda: Rapid, easy full-stack web development starter kit in Go

[![Go Report Card](https://goreportcard.com/badge/github.com/mikestefanello/pagoda)](https://goreportcard.com/report/github.com/mikestefanello/pagoda)
[![Test](https://github.com/mikestefanello/pagoda/actions/workflows/test.yml/badge.svg)](https://github.com/mikestefanello/pagoda/actions/workflows/test.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Go Reference](https://pkg.go.dev/badge/github.com/mikestefanello/pagoda.svg)](https://pkg.go.dev/github.com/mikestefanello/pagoda)
[![GoT](https://img.shields.io/badge/Made%20with-Go-1f425f.svg)](https://go.dev)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)

<p align="center"><img alt="Logo" src="https://user-images.githubusercontent.com/552328/147838644-0efac538-a97e-4a46-86a0-41e3abdf9f20.png" height="200px"/></p>

## Table of Contents
* [Introduction](#introduction)
    * [Overview](#overview)
    * [Foundation](#foundation)
      * [Backend](#backend)
      * [Frontend](#frontend)
      * [Storage](#storage)
    * [Screenshots](#screenshots)
* [Getting started](#getting-started)
  * [Dependencies](#dependencies)
  * [Getting the code](#getting-the-code)
  * [Installing tools](#installing-tools)
  * [Create an admin account](#create-an-admin-account)
  * [Start the application](#start-the-application)
  * [Live reloading](#live-reloading)
* [Service container](#service-container)
  * [Dependency injection](#dependency-injection)
  * [Test dependencies](#test-dependencies)
* [Configuration](#configuration)
    * [Environment overrides](#environment-overrides)
    * [Environments](#environments)
* [Database](#database)
    * [Auto-migrations](#auto-migrations)
    * [Separate test database](#separate-test-database)
* [ORM](#orm)
  * [Entity types](#entity-types)
  * [New entity type](#new-entity-type)
* [Sessions](#sessions)
  * [Encryption](#encryption)
* [Authentication](#authentication)
  * [Login / Logout](#login--logout)
  * [Forgot password](#forgot-password)
  * [Registration](#registration)
  * [Admins](#admins)
  * [Authenticated user](#authenticated-user)
    * [Middleware](#middleware)
  * [Email verification](#email-verification)
* [Admin panel](#admin-panel)
  * [Code generation](#code-generation)
  * [Access](#access)
  * [Considerations](#considerations)
  * [Roadmap](#roadmap)
* [Routes](#routes)
  * [Custom middleware](#custom-middleware)
  * [Handlers](#handlers)
  * [Errors](#errors)
  * [Redirects](#redirects)
  * [Testing](#testing)
    * [HTTP server](#http-server)
    * [Request / Request helpers](#request--response-helpers)
    * [Goquery](#goquery)
* [User interface](#user-interface)
  * [Why Gomponents?](#why-gomponents)
  * [HTMX support](#htmx-support)
    * [Header management](#header-management)
    * [Conditional and partial rendering](#conditional-and-partial-rendering)
    * [CSRF token](#csrf-token)
  * [CSS](#css)
  * [Request](#request)
    * [Title and metatags](#title-and-metatags)
    * [URL generation](#url-generation)
  * [Components](#components)
  * [Layouts](#layouts)
  * [Pages](#pages)
    * [Rendering](#rendering)
  * [Forms](#forms)
    * [Submission processing](#submission-processing)
    * [Inline validation](#inline-validation)
    * [CSRF](#csrf)
  * [Models](#models)
  * [Icons](#icons)
  * [Node caching](#node-caching)
  * [Flash messaging](#flash-messaging)
* [Pager](#pager)
* [Cache](#cache)
  * [Set data](#set-data)
  * [Get data](#get-data)
  * [Flush data](#flush-data)
  * [Flush tags](#flush-tags)
* [Tasks](#tasks)
  * [Queues](#queues)
  * [Dispatcher](#dispatcher)
  * [Monitoring tasks and queues](#monitoring-tasks-and-queues)
* [Cron](#cron)
* [Files](#files)
  * [Uploads](#uploads)
  * [Static files](#static-files)
    * [Cache-buster](#cache-buster)
  * [Public files](#public-files)
  * [Cache control headers](#cache-control-headers)
* [Email](#email)
* [HTTPS](#https)
* [Logging](#logging)
* [Credits](#credits)

## Introduction

### Overview

_Pagoda_ is not a framework but rather a base starter-kit for rapid, easy full-stack web development in Go, aiming to provide much of the functionality you would expect from a complete web framework as well as establishing patterns, procedures and structure for your web application.

Built on a solid [foundation](#foundation) of well-established frameworks and modules, _Pagoda_ aims to be a starting point for any web application with the benefit over a mega-framework in that you have full control over all of the code, the ability to easily swap any frameworks or modules in or out, no strict patterns or interfaces to follow, and no fear of lock-in.

While separate JavaScript frontends have surged in popularity, many prefer the reliability, simplicity and speed of a full-stack approach with server-side rendered HTML. Even the popular JS frameworks all have SSR options. This project aims to highlight that _Go_ alone can be powerful and easy to work with as a full-stack solution, and interesting [frontend](#frontend) libraries can provide the same modern functionality and behavior without having to write any JS or CSS at all. In fact, you can even avoid writing HTML as well.

### Foundation

While many great projects were used to build this, all of which are listed in the [credits](#credits) section, the following provide the foundation of the back and frontend. It's important to note that you are **<ins>not required to use any of these</ins>**. Swapping any of them out will be relatively easy.

#### Backend

- [Echo](https://echo.labstack.com/): High performance, extensible, minimalist Go web framework.
- [Ent](https://entgo.io/): Simple, yet powerful ORM for modeling and querying data.
- [Gomponents](https://github.com/maragudk/gomponents): HTML components written in pure Go. They render to HTML 5, and make it easy for you to build reusable components.

#### Frontend

Go server-side rendered HTML combined with the projects below enable you to create slick, modern UIs without writing any JavaScript or CSS.

- [HTMX](https://htmx.org/): Access AJAX, CSS Transitions, WebSockets and Server Sent Events directly in HTML, using attributes, so you can build modern user interfaces with the simplicity and power of hypertext.
- [Alpine.js](https://alpinejs.dev/): Rugged, minimal tool for composing behavior directly in your markup. Think of it like jQuery for the modern web. Plop in a script tag and get going.
- [DaisyUI](https://daisyui.com/): The [Tailwind CSS](https://tailwindcss.com/) plugin you will love! It provides useful component class names to help you write less code and build faster. No JavaScript dependencies.

#### Storage

- [SQLite](https://sqlite.org/): A small, fast, self-contained, high-reliability, full-featured, SQL database engine and the most used database engine in the world.

Originally, Postgres and Redis were chosen as defaults but since the aim of this project is rapid, simple development, it was changed to SQLite which now provides the primary data storage as well as persistent, background [task queues](#tasks). For [caching](#cache), a simple in-memory solution is provided. If you need to use something like Postgres or Redis, swapping those in can be done quickly and easily. For reference, [this branch](https://github.com/mikestefanello/pagoda/tree/postgres-redis) contains the code that included those (but is no longer maintained).

### Screenshots

#### Inline form validation

<img src="https://raw.githubusercontent.com/mikestefanello/readmeimages/main/pagoda/inline-validation.png" alt="Inline validation"/>

#### Switch layout templates, user registration

<img src="https://raw.githubusercontent.com/mikestefanello/readmeimages/main/pagoda/register.png" alt="Registration"/>

#### Alpine.js modal, HTMX AJAX request

<img src="https://raw.githubusercontent.com/mikestefanello/readmeimages/main/pagoda/modal.png" alt="Alpine and HTMX"/>

#### User entity list (admin panel)

<img src="https://raw.githubusercontent.com/mikestefanello/readmeimages/main/pagoda/admin-user_list.png" alt="User entity list"/>

#### User entity edit (admin panel)

<img src="https://raw.githubusercontent.com/mikestefanello/readmeimages/main/pagoda/admin-user_edit.png" alt="User entity edit"/>

#### Monitor task queues (provided by Backlite via the admin panel)

<img src="https://raw.githubusercontent.com/mikestefanello/readmeimages/main/backlite/failed.png" alt="Manage task queues"/>

## Getting started

### Dependencies

Ensure that [Go](https://go.dev/) is installed on your system.

### Getting the code

Start by checking out the repository. Since this repository is a _template_ and not a Go _library_, you **do not** use `go get`.

```
git clone [email protected]:mikestefanello/pagoda.git
cd pagoda
```

### Installing tools

Several optional tools are available to make development easier for you. This includes [Ent](#orm) code-generator, for generating ORM code, [Air](https://github.com/air-verse/air) CLI, to provide [live reloading](#live-reloading), and [Tailwind CSS](https://tailwindcss.com/docs/installation/tailwind-cli) CLI, to generate CSS. 

If you don't want to use Tailwind and/or [Daisy UI](https://daisyui.com/), or don't want to use Tailwind's standalone CLI, but rather `npm`, for example, modify the `tailwind-install` and `css` [make targets](https://github.com/mikestefanello/pagoda/blob/main/Makefile) based on your preferences. If the script cannot automatically determine the proper Tailwind package to install, modify the `TAILWIND_PACKAGE` variable to match your operating system.

To easily install all tools, run `make install` from the root of the repo. There are also separate _make targets_ for each tool (run `make help` to list all targets).

### Create an admin account

To access the [admin panel](#admin-panel), you must log in with an admin user and to create your first admin user account, you must use the command-line. Execute `make admin [email protected]` from the root of the codebase, and an admin account will be generated using that email address. The console will print the randomly generated password for the account.

Once you have one admin account, you can use that account to manage other users and admins from within the UI.

### Start the application

From within the root of the codebase, simply run `make run`.

By default, you should be able to access the application in your browser at `localhost:8000`. Your data will be stored within the `dbs` directory. If you ever want to quickly delete all data, just remove this directory.

These settings, and many others, can be changed via the [configuration](#configuration).

### Live reloading

Rather than using `make run`, if you prefer live reloading so your app automatically rebuilds and runs whenever you save code changes, start by installing [Air](https://github.com/air-verse/air), if you haven't already, by running `make air-install` (or `make install` to install all tools), then use `make watch` to start the application with automatic live reloading.

When code changes are detected, `make css` will run to re-compile Tailwind styles automatically before restarting the app. If you choose not to use Tailwind, either modify `make css` to run whatever commands you require, or remove this from the `make build` target.

## Service container

The container is located at `pkg/services/container.go` and is meant to house all of your application's services and/or dependencies. It is easily extensible and can be created and initialized in a single call. The services currently included in the container are:

- Authentication
- Cache
- Configuration
- Database
- Files
- Mail
- ORM
- Tasks
- Validator
- Web

A new container can be created and initialized via `services.NewContainer()`. It can be later shutdown via `Shutdown()`, which will attempt to gracefully shutdown all services.

### Dependency injection

The container exists to facilitate easy dependency-injection both for services within the container and areas of your application that require any of these dependencies. For example, the container is automatically passed to the `Init()` method of your route [handlers](#handlers) so that the handlers have full, easy access to all services.

### Test dependencies

It is common that your tests will require access to dependencies, like the database, or any of the other services available within the container. Keeping all services in a container makes it especially easy to initialize everything within your tests. You can see an example pattern for doing this [here](#environments).

## Configuration

The `config` package provides a flexible, extensible way to store all configuration for the application. Configuration is added to the `Container` as a _Service_, making it accessible across most of the application.

Be sure to review and adjust all the default configuration values provided in `config/config.yaml`.

### Environment overrides

Leveraging the functionality of [viper](https://github.com/spf13/viper) to manage configuration, all configuration values can be overridden by environment variables. The name of the variable is determined by the set prefix and the name of the configuration field in `config/config.yaml`.

In `config/config.go`, the prefix is set as `pagoda` via `viper.SetEnvPrefix("pagoda")`. Nested fields require an underscore between levels. For example:

```yaml
http:
  port: 1234
```

can be overridden by setting an environment variable with the name `PAGODA_HTTP_PORT`.

### Environments

The configuration value for the current _environment_ (`Config.App.Environment`) is an important one as it can influence some behavior significantly (will be explained in later sections).

A helper function (`config.SwitchEnvironment`) is available to make switching the environment easy, but this must be executed prior to loading the configuration. The common use-case for this is to switch the environment to `Test` before tests are executed:

```go
func TestMain(m *testing.M) {
    // Set the environment to test
    config.SwitchEnvironment(config.EnvTest)

    // Start a new container
    c = services.NewContainer()

    // Run tests
    exitVal := m.Run()

    // Shutdown the container
    if err := c.Shutdown(); err != nil {
        panic(err)
    }

    os.Exit(exitVal)
}
```

## Database

The database currently used is [SQLite](https://sqlite.org/) but you are free to use whatever you prefer. If you plan to continue using [Ent](https://entgo.io/), the incredible ORM, you can check their supported databases [here](https://entgo.io/docs/dialects). The database driver is provided by [go-sqlite3](https://github.com/mattn/go-sqlite3). A reference to the database is included in the `Container` if direct access is required.

Database configuration can be found and managed within the `config` package.

### Auto-migrations

[Ent](https://entgo.io/) provides automatic migrations which are executed on the database whenever the `Container` is created, which means they will run when the application starts.

### Separate test database

Since many tests can require a database, this application supports a separate database specifically for tests. Within the `config`, the test database can be specified at `Config.Database.TestConnection`, which is the database connection string that will be used. By default, this will be an in-memory SQLite database.

When a `Container` is created, if the [environment](#environments) is set to `config.EnvTest`, the database client will connect to the test database instead and run migrations so your tests start with a clean, ready-to-go database.

When this project was using Postgres, it would automatically drop and recreate the test database. Since the current default is in-memory, that is no longer needed. If you decide to use a test database not in-memory, you can alter the `Container` initialization code to do this for you.

## ORM

As previously mentioned, [Ent](https://entgo.io/) is the supplied ORM. It can be swapped out, but I highly recommend it. I don't think there is anything comparable for Go, at the current time. If you decide to remove Ent, you will lose the dynamic [admin panel](#admin-panel) which allows you to administer all entity types from within the UI. If you're not familiar with Ent, take a look through their top-notch [documentation](https://entgo.io/docs/getting-started).

An Ent client is included in the `Container` to provide easy access to the ORM throughout the application.

Ent relies on code-generation for the entities you create to provide robust, type-safe data operations. Everything within the `ent` directory in this repository is generated code for the two entity types listed below except the [schema declaration](#https://github.com/mikestefanello/pagoda/tree/main/ent/schema) and [custom extension](https://github.com/mikestefanello/pagoda/tree/main/ent/admin) to generate code for the [admin panel](#admin-panel).

### Entity types

The two included entity types are:
- User
- PasswordToken

### New entity type

While you should refer to their [documentation](https://entgo.io/docs/getting-started) for detailed usage, it's helpful to understand how to create an entity type and generate code. To make this easier, the `Makefile` contains some helpers.

1. Ensure all Ent code is downloaded by executing `make ent-install`.
2. Create the new entity type by executing `make ent-new name=User` where `User` is the name of the entity type. This will generate a file like you can see in `ent/schema/user.go` though the `Fields()` and `Edges()` will be left empty.
3. Populate the `Fields()` and optionally the `Edges()` (which are the relationships to other entity types).
4. When done, generate all code by executing `make ent-gen`.

The generated code is extremely flexible and impressive. An example to highlight this is one used within this application:

```go
entity, err := c.ORM.PasswordToken.
    Query().
    Where(passwordtoken.ID(tokenID)).
    Where(passwordtoken.HasUserWith(user.ID(userID))).
    Where(passwordtoken.CreatedAtGTE(expiration)).
    Only(ctx.Request().Context())
```

This executes a database query to return the _password token_ entity with a given ID that belong to a user with a given ID and has a _created at_ timestamp field that is greater than or equal to a given time.

## Sessions

Sessions are provided and handled via [Gorilla sessions](https://github.com/gorilla/sessions) and configured as middleware in the router located at `pkg/handlers/router.go`. Session data is currently stored in cookies but there are many [options](https://github.com/gorilla/sessions#store-implementations) available if you wish to use something else.

Here's a simple example of loading data from a session and saving new values:

```go
func SomeFunction(ctx echo.Context) error {
    sess, err := session.Get(ctx, "some-session-key")
    if err != nil {
        return err
    }
    sess.Values["hello"] = "world"
    sess.Values["isSomething"] = true
    return sess.Save(ctx.Request(), ctx.Response())
}
```

### Encryption

Session data is encrypted for security purposes. The encryption key is stored in [configuration](#configuration) at `Config.App.EncryptionKey`. While the default is fine for local development, it is **imperative** that you change this value for any live environment otherwise session data can be compromised.

## Authentication

Included are standard authentication features you expect in any web application. Authentication functionality is bundled as a _Service_ within `services/AuthClient` and added to the `Container`. If you wish to handle authentication in a different manner, you could swap this client out or modify it as needed.

Authentication currently requires [sessions](#sessions) and the session middleware.

### Login / Logout

Frequently Asked Questions