afero

Abstract File Storage (mem, scp, zip, tar, cloud: s3, gs) for Go

# afs - abstract file storage

[![GoReportCard](https://goreportcard.com/badge/github.com/viant/afs)](https://goreportcard.com/report/github.com/viant/afs)
[![GoDoc](https://godoc.org/github.com/viant/afs?status.svg)](https://godoc.org/github.com/viant/afs)
![goversion-image](https://img.shields.io/badge/Go-1.11+-00ADD8.svg)
<a href='https://github.com/jpoles1/gopherbadger' target='_blank'>![gopherbadger-tag-do-not-edit](https://img.shields.io/badge/Go%20Coverage-78%25-brightgreen.svg?longCache=true&style=flat)</a>


Please refer to [`CHANGELOG.md`](CHANGELOG.md) if you encounter breaking changes.

- [Motivation](#motivation)
- [Introduction](#introduction)
- [Usage](#usage)
- [Matchers](#matchers)
- [Content modifiers](#content-modifiers)
- [Streaming data](#streaming-data)
- [Options](#options)
- [Storage Implementations](#storage-implementations)
- [Testing mode](#testing-mode)
- [Storage Manager](#storage-managers)
- [GoCover](#gocover)
- [License](#license)
- [Credits and Acknowledgements](#credits-and-acknowledgements)

## Motivation

When dealing with various storage systems, like cloud storage, SCP, container or local file system, using shared API for typical storage operation provides an excellent simplification.
What's more, the ability to simulate storage-related errors like Auth or EOF allows you to test an app error handling.

## Introduction

This library uses a storage manager abstraction to provide an implementation for a specific storage system with following 

* **CRUD Operation:**

```go
List(ctx context.Context, URL string, options ...Option) ([]Object, error)

Walk(ctx context.Context, URL string, handler OnVisit, options ...Option) error

Open(ctx context.Context, object Object, options ...Option) (io.ReadCloser, error)

OpenURL(ctx context.Context, URL string, options ...Option) (io.ReadCloser, error)


Upload(ctx context.Context, URL string, mode os.FileMode, reader io.Reader, options ...Option) error

Create(ctx context.Context, URL string, mode os.FileMode, isDir bool, options ...Option) error

Delete(ctx context.Context, URL string, options ...Option) error
``` 

* **Batch uploader:**

```go
type Upload func(ctx context.Context, parent string, info os.FileInfo, reader io.Reader) error
 
Uploader(ctx context.Context, URL string, options ...Option) (Upload, io.Closer, error)
```

* **Utilities:**

```go

Copy(ctx context.Context, sourceURL, destURL string, options ...Option) error

Move(ctx context.Context, sourceURL, destURL string, options ...Option) error

NewWriter(ctx context.Context, URL string, mode os.FileMode, options ...storage.Option) (io.WriteCloser, error)

DownloadWithURL(ctx context.Context, URL string, options ...Option) ([]byte, error)

Download(ctx context.Context, object Object, options ...Option) ([]byte, error)

```


URL scheme is used to identify storage system, or alternatively relative/absolute path can be used for local file storage.
By default, all operations using the same baseURL share the same corresponding storage manager instance.
For example, instead supplying SCP auth details for all operations, auth option can be used only once.

```go

func main() {

 ctx := context.Background()
 {
 //auth with first call 
 fs := afs.New()
 defer fs.Close()
 keyAuth, err := scp.LocalhostKeyAuth("")
 if err != nil {
 log.Fatal(err)
 }
 reader1, err := fs.OpenURL(ctx, "scp://host1:22/myfolder/asset.txt", keyAuth)
 if err != nil {
 log.Fatal(err)
 }
 ...
 reader2, err := fs.OpenURL(ctx, "scp://host1:22/myfolder/asset.txt", keyAuth)
 }
 
 {
 //auth per baseURL 
 fs := afs.New()
 err = fs.Init(ctx, "scp://host1:22/", keyAuth)
 if err != nil {
 log.Fatal(err)
 }
 defer fs.Destroy("scp://host1:22/")
 reader, err := fs.OpenURL(ctx, "scp://host1:22/myfolder/asset.txt")
 }
}

```

## Usage

##### Downloading location content

```go
func main() {
	
 fs := afs.New()
 ctx := context.Background()
 objects, err := fs.List(ctx, "/tmp/folder")
 if err != nil {
 log.Fatal(err)
 }
 for _, object := range objects {
 fmt.Printf("%v %v\n", object.Name(), object.URL())
 if object.IsDir() {
 continue
 }
 reader, err := fs.Open(ctx, object)
 if err != nil {
 log.Fatal(err)
 }
 data, err := ioutil.ReadAll(reader)
 if err != nil {
 log.Fatal(err)
 }
 fmt.Printf("%s\n", data)
 }
}
```

##### Uploading Content

```go
func main() {
	
 fs := afs.New()
 ctx := context.Background()
 keyAuth, err := scp.LocalhostKeyAuth("")
 if err != nil {
 log.Fatal(err)
 }
 err = fs.Init(ctx, "scp://127.0.0.1:22/", keyAuth)
 if err != nil {
 log.Fatal(err)
 }	
 err = fs.Upload(ctx, "scp://127.0.0.1:22/folder/asset.txt", 0644, strings.NewReader("test me"))
 if err != nil {
 log.Fatal(err)
 }
 ok, err := fs.Exists(ctx, "scp://127.0.0.1:22/folder/asset.txt")
 if err != nil {
 log.Fatal(err)
 }
 fmt.Printf("has file: %v\n", ok)
 _ = fs.Delete(ctx, "scp://127.0.0.1:22/folder/asset.txt")
}
```
##### Uploading Content With Writer

```go
func main() {
	
 fs := afs.New()
 ctx := context.Background()
 keyAuth, err := scp.LocalhostKeyAuth("")
 if err != nil {
 log.Fatal(err)
 }
 err = fs.Init(ctx, "scp://127.0.0.1:22/", keyAuth)
 if err != nil {
 log.Fatal(err)
 }	
 writer = fs.NewWriter(ctx, "scp://127.0.0.1:22/folder/asset.txt", 0644)
 _, err := writer.Write([]byte("test me")))
 if err != nil {
 log.Fatal(err)
 }
 err = writer.Close()
 if err != nil {
 log.Fatal(err)
 }
 ok, err := fs.Exists(ctx, "scp://127.0.0.1:22/folder/asset.txt")
 if err != nil {
 log.Fatal(err)
 }
 fmt.Printf("has file: %v\n", ok)
 _ = fs.Delete(ctx, "scp://127.0.0.1:22/folder/asset.txt")
}
```



##### Data Copy

```go
func main() {

 fs := afs.New()
 ctx := context.Background()
 keyAuth, err := scp.LocalhostKeyAuth("")
 if err != nil {
 log.Fatal(err)
 }
 err = fs.Copy(ctx, "s3://mybucket/myfolder", "scp://127.0.0.1/tmp", option.NewSource(), option.NewDest(keyAuth))
 if err != nil {
 log.Fatal(err)
 }
}
```

##### Archiving content

```go

func main() {
	
 secretPath := path.Join(os.Getenv("HOME"), ".secret", "gcp-e2e.json")
 auth, err := gs.NewJwtConfig(option.NewLocation(secretPath))
 if err != nil {
 return
 }
 sourceURL := "mylocalPath/"
 destURL := "gs:mybucket/test.zip/zip://localhost/dir1"
 fs := afs.New()
 ctx := context.Background()
 err = fs.Copy(ctx, sourceURL, destURL, option.NewDest(auth))
 if err != nil {
 log.Fatal(err)
 }

}	
```


##### Archive Walker

Walker can be created for tar or zip archive.

```go
func main() {
	
 ctx := context.Background()
	fs := afs.New()
	walker := tar.NewWalker(s3afs.New())
	err := fs.Copy(ctx, "/tmp/test.tar", "s3:///dest/folder/test", walker)
	if err != nil {
		log.Fatal(err)
	}
```


##### Archive Uploader

Uploader can be created for tar or zip archive.

```go
func main() {
	
 ctx := context.Background()
	fs := afs.New()
	uploader := zip.NewBatchUploader(gsafs.New())
	err := fs.Copy(ctx, "gs:///tmp/test/data", "/tmp/data.zip", uploader)
	if err != nil {
		log.Fatal(err)
	}
}
```


##### Data Move

```go
func main() {
	
 fs := afs.New()
	ctx := context.Background()
	keyAuth, err := scp.LocalhostKeyAuth("")
	if err != nil {
		log.Fatal(err)
	}
	err = fs.Move(ctx, "/tmp/transient/app", "scp://127.0.0.1/tmp", option.NewSource(), option.NewDest(keyAuth))
	if err != nil {
		log.Fatal(err)
	}
}	
```

##### Batch Upload

```go
func main() {
	
 fs := afs.New()
	ctx := context.Background()
	upload, closer, err := fs.Uploader(ctx, "/tmp/clone")
	if err != nil {
		log.Fatal(err)
	}
	defer closer.Close()
	assets := []*asset.Resource{
		asset.NewFile("asset1.txt", []byte("test 1"), 0644),
		asset.NewFile("asset2.txt", []byte("test 2"), 0644),
		asset.NewDir("folder1", file.DefaultDirOsMode),
		asset.NewFile("folder1/asset1.txt", []byte("test 3"), 0644),
		asset.NewFile("folder1/asset2.txt", []byte("test 4"), 0644),
	}
	for _, asset := range assets {
		relative := ""
		var reader io.Reader
		if strings.Contains(asset.Name, "/") {
			relative, _ = path.Split(asset.Name)
		}
		if ! asset.Dir {
			reader = bytes.NewReader(asset.Data)
		}
		err = upload(ctx, relative, asset.Info(), reader)
		if err != nil {
			log.Fatal(err)
		}
	}
}
```
## Matchers

To filter source content you can use [Matcher](option/matcher.go) option. 
The following have been implemented.


**[Basic Matcher](matcher/basic.go)**

```go
func main() {
	
 matcher, err := NewBasic("/data", ".avro", nil)
 fs := afs.New()
 ctx := context.Background()
 err := fs.Copy(ctx, "/tmp/data", "s3://mybucket/data/", matcher.Match)
 if err != nil {
 log.Fatal(err)
 }
}
```

Exclusion

```go
func main() {
	
 matcher := matcher.Basic{Exclusion:".+/data/perf/\\d+/.+"}
 fs := afs.New()
 ctx := context.Background()
 err := fs.Copy(ctx, "/tmp/data", "s3://mybucket/data/", matcher.Match)
 if err != nil {
 log.Fatal(err)
 }
}
```

**[Filepath matcher](matcher/filepath.go)**

OS style filepath match, with the following terms:
- '*' matches any sequence of non-Separator characters
- '?' matches any single non-Separator character
- '[' [ '^' ] { character-range } ']'

```go

func main() {
	
 matcher := matcher.Filepath("*.avro")
 fs := afs.New()
 ctx := context.Background()
 err := fs.Copy(ctx, "/tmp/data", "gs://mybucket/data/", matcher)
 if err != nil {
 log.Fatal(err)
 }
}	
		

```

**[Ignore Matcher](matcher/ignore.go)**

Ignore matcher represents matcher that matches file that are not in the ignore rules.
The syntax of ignore borrows heavily from that of .gitignore; see https://git-scm.com/docs/gitignore or man gitignore for a full reference.


```go
func mian(){
	
	ignoreMatcher, err := matcher.NewIgnore([]string{"*.txt", ".ignore"})
 	//or matcher.NewIgnore(option.NewLocation(".cloudignore"))
	if err != nil {
		log.Fatal(err)
	}
	fs := afs.New()
	ctx := context.Background()
	objects, err := fs.List(ctx, "/tmp/folder", ignoreMatcher.Match)
	if err != nil {
		log.Fatal(err)
	}
	for _, object := range objects {
		fmt.Printf("%v %v\n", object.Name(), object.URL())
		if object.IsDir() {
			continue
		}
	}
}	
```



**[Modification Time Matcher](matcher/modification.go)**

Modification Time Matcher represents matcher that matches file that were modified either before or after specified time.

```go
func mian(){
	
	before, err := toolbox.TimeAt("2 days ago in UTC")
 if err != nil {
		log.Fatal(err)
	}	
	modTimeMatcher, err := matcher.NewModification(before, nil)
	if err != nil {
		log.Fatal(err)
	}
	fs := afs.New()
	ctx := context.Background()
	objects, err := fs.List(ctx, "/tmp/folder", modTimeMatcher.Match)
	if err != nil {
		log.Fatal(err)
	}
	for _, object := range objects {
		fmt.Printf("%v %v\n", object.Name(), object.URL())
		if object.IsDir() {
			continue
		}
	}
}	
```


## Content modifiers

To modify resource content on the fly you can use [Modifier](option/modifier.go) option.

```go
func main() {
	fs := afs.New()
	ctx := context.Background()
	sourceURL := "file:/tmp/app.war/zip://localhost/WEB-INF/classes/config.properties"
	destURL := "file:/tmp/app.war/zip://localhost/"
	err := fs.Copy(ctx, sourceURL, destURL, modifier.Replace(map[string]string{
		"${changeMe}": os.Getenv("USER"),
	}))
	if err != nil {
		log.Fatal(err)
	}
}
```

```go
package main

import (
	"context"
	"log"
	"github.com/viant/afs"
	"io"
	"fmt"
	"io/ioutil"
	"os"
	"strings"
)

func modifyContent(info os.FileInfo, reader io.ReadCloser) (closer io.ReadCloser, e error) {
 if strings.HasSuffix(info.Name() ,".info") {
 data, err := ioutil.ReadAll(reader)
 if err != nil {
 return nil, err
 }
 _ = reader.Close()
 expanded := strings.Replace(string(data), "$os.User", os.Getenv("USER"), 1)
 reader = ioutil.NopCloser(strings.NewReader(expanded))
 }
 return reader, nil
} 

func main() {

 fs := afs.New()
 reader ,err := fs.OpenURL(context.Background(), "s3://mybucket/meta.info", modifyContent)
 if err != nil {
 log.Fatal(err)	
 }
 
 defer reader.Close()
 content, err := ioutil.ReadAll(reader)
 if err != nil {
 log.Fatal(err)	
 }
 fmt.Printf("content: %s\n", content)
	
}
```


### Streaming data

Streaming data allows data reading and uploading in chunks with small memory footprint.


```go

 jwtConfig, err := gs.NewJwtConfig()
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	fs := afs.New()
	sourceURL := "gs://myBucket/path/myasset.gz"
	reader, err := fs.OpenURL(ctx, sourceURL, jwtConfig, option.NewStream(64*1024*1024, 0))
	if err != nil {
		log.Fatal(err)
	}
 
	_ = os.Setenv("AWS_SDK_LOAD_CONFIG", "true")
	destURL := "s3://myBucket/path/myasset.gz"
	err = fs.Upload(ctx, destURL, 0644, reader, &option.Checksum{Skip:true})
	if err != nil {
		log.Fatal(err)
	}

 // or
 writer = fs.NewWriter(ctx, destURL, 0644, &option.Checksum{Skip:true})
 _, err = io.Copy(writer, reader)
 if err != nil {
 log.Fatal(err)
 }
 err = writer.Close()
	if err != nil {
		log.Fatal(err)
	}
```



## Options


* **[Page](option/page.go)**

To control number and position of listed resources you can yse page option.

* **[Timeout](option/timeout.go)**

Provider specific timeout.

* **[BasicAuth](option/cred.go)** 

Provides user/password auth.

* **Source & Dest Options**

Groups options by source or destination options. This options work with Copy or Move operations.

```go

func main() {
	
 fs := afs.New()
 secretPath := path.Join(os.Getenv("HOME"), ".secret","gcp.json")
 jwtConfig, err := gs.NewJwtConfig(option.NewLocation(secretPath))
 if err != nil {
 	log.Fatal(err)
 }
 sourceOptions := option.NewSource(jwtConfig)
 authConfig, err := s3.NewAuthConfig(option.NewLocation("aws.json"))
 if err != nil {
 log.Fatal(err)
 }
 destOptions := option.NewDest(authConfig)
	err = fs.Copy(ctx, "gs://mybucket/data", "s3://mybucket/data", sourceOptions, destOptions)
}

```


* **[option.Checksum](option/checksum.go)** skip computing checksum if Skip is set, this option allows streaming upload in chunks
* **[option.Stream](option/stream.go)**: download reader reads data with specified stream PartSize 





Check out [storage manager](#storage-managers) for additional options. 

## Storage Implementations

- [File](file/README.md)
- [In Memory](mem/README.md)
- [SSH - SCP](scp/README.md)
- [HTTP](http/README.md)
- [Tar](tar/README.md)
- [Zip](zip/README.md)
- [GCP - GS](https://github.com/viant/afsc/tree/master/gs)
- [AWS - S3](https://github.com/viant/afsc/tree/master/s3)

## Testing fs

To unit test all storage operation all in memory you can use faker fs.

In addition you can use error options to test exception handling.

- **DownloadError**
```go

func mian() {
	fs := afs.NewFaker()
	ctx := context.Background()
	err := fs.Upload(ctx, "gs://myBucket/folder/asset.txt", 0, strings.NewReader("some data"), option.NewUploadError(io.EOF))
	if err != nil {
		log.Fatalf("expect upload error: %v", err)
	}
}
```

- **ReaderError**
```go

func mian() {
 fs := afs.NewFaker()
	ctx := context.Background()
	err := fs.Upload(ctx, "gs://myBucket/folder/asset.txt", 0, strings.NewReader("some data"), option.NewDownloadError(io.EOF))
	if err != nil {
		log.Fatal(err)
	}
	_, err = fs.OpenURL(ctx, "gs://myBucket/folder/asset.txt")
	if err != nil {
		log.Fatalf("expect download error: %v", err)
	}
}
```

- **UploadError**
```go

func mian() {
 fs := afs.NewFaker()
 ctx := context.Background()
 err := fs.Upload(ctx, "gs://myBucket/folder/asset.txt", 0, strings.NewReader("some data"), option.NewUploadError(io.EOF))
 if err != nil {
 log.Fatalf("expect upload error: %v", err)
 }
}
```


##### Code generation for static or in memory go file


Generate with mem storage

```
package main

import (
 "log"
 "github.com/viant/afs/parrot
)

func mian() {
 ctx := context.Background()
 err := parrot.GenerateWithMem(ctx, "pathToBinaryAsset", "gen.go", false)
 if err != nil {
 log.Fatal(err)
 }
}

```

Generate static data files

```
package main

import (
 "log"
 "github.com/viant/afs/parrot
)

func mian() {
 ctx := context.Background()
 err := parrot.Generate(ctx, "pathToBinaryAsset", "data/", false)
 if err != nil {
 log.Fatal(err)
 }
}

```


## Test setup utilities

Package [asset](asset) defines basic utilities to quickly manage asset related unit tests.

```go

func Test_XXX(t *testing.T) {

 var useCases = []struct {
		description string
		location string
		options []storage.Option
		assets []*asset.Resource
	}{

	}

	ctx := context.Background()
	for _, useCase := range useCases {
		fs := afs.New()
		mgr, err := afs.Manager(useCase.location, useCase.options...)
		if err != nil {
			log.Fatal(err)
		}
		err = asset.Create(mgr, useCase.location, useCase.assets)
		if err != nil {
			log.Fatal(err)
		}
		
		//... actual app logic

		actuals, err := asset.Load(mgr, useCase.location)
		if err != nil {
			log.Fatal(err)
		}
 for _, expect := range useCase.assets {
 actual, ok := actuals[expect.Name]
 if !assert.True(t, ok, useCase.description+": "+expect.Name+fmt.Sprintf(" - actuals: %v", actuals)) {
 continue
 }
 assert.EqualValues(t, expect.Name, actual.Name, useCase.description+" "+expect.Name)
 assert.EqualValues(t, expect.Mode, actual.Mode, useCase.description+" "+expect.Name)
 assert.EqualValues(t, expect.Dir, actual.Dir, useCase.description+" "+expect.Name)
 assert.EqualValues(t, expect.Data, actual.Data, useCase.description+" "+expect.Name)
 }

		_ = asset.Cleanup(mgr, useCase.location)

	}
}

```




## GoCover

[![GoCover](https://gocover.io/github.com/viant/afs)](https://gocover.io/github.com/viant/afs)


<a name="License"></a>
## License

The source code is made available under the terms of the Apache License, Version 2, as stated in the file `LICENSE`.

Individual files may be made available under their own specific license,
all compatible with Apache License, Version 2. Please see individual files for details.

<a name="Credits-and-Acknowledgements"></a>

## Credits and Acknowledgements

**Library Author:** Adrian Witas

A file transfer system, support to manage files with http api, rpc call and ftp client.

### Bigfile ———— a file transfer system that supports http, rpc and ftp protocol [简体中文](https://learnku.com/docs/bigfile) ∙ [English](https://bigfile.site)
----


 <a href="https://travis-ci.org/bigfile/bigfile"><img src="https://travis-ci.org/bigfile/bigfile.svg?branch=master"/></a>
 <a href="https://codecov.io/gh/bigfile/bigfile"><img src="https://codecov.io/gh/bigfile/bigfile/branch/master/graph/badge.svg"/></a>
 <a href="https://godoc.org/github.com/bigfile/bigfile"><img src="https://godoc.org/github.com/bigfile/bigfile?status.svg"/></a>
 <a href="https://github.com/bigfile/bigfile"><img src="https://img.shields.io/badge/release-v1.0.11-blue"/></a>
 <a href="https://app.fossa.io/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile?ref=badge_shield"><img src="https://app.fossa.io/api/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile.svg?type=shield"/></a>
 <a href="https://goreportcard.com/report/github.com/bigfile/bigfile"><img src="https://goreportcard.com/badge/github.com/bigfile/bigfile"/></a>
 <a href="https://golangci.com/r/github.com/bigfile/bigfile"><img src="https://golangci.com/badges/github.com/bigfile/bigfile.svg"/></a>
 <a href="https://github.com/avelino/awesome-go"><img src="https://awesome.re/mentioned-badge.svg"/></a>



 <img src="https://bigfile.site/bigfile.png" />


**Bigfile** is a file transfer system, supports http, ftp and rpc protocol. Designed to provide a file management service and give developers more help. At the bottom, bigfile splits the file into small pieces of **1MB**, the same slice will only be stored once.

In fact, we built a file organization system based on the database. Here you can find familiar files and folders.

Since the rpc and http protocols are supported, those languages supported by [grpc](https://grpc.io/) and other languages can be quickly accessed.

More detailed documents can be found here

### Features

* Support HTTP(s) protocol

 * Support rate limit by ip
 * Support cors
 * Support to avoid replay attack
 * Support to validate parameter signature
 * Support Http Single Range Protocol

* Support FTP(s) protocol

* Support RPC protocol

* Support to deploy by [docker](https://hub.docker.com/r/bigfile/bigfile)

* Provide document with English and Chinese


### Install Bigfile

There are kinds of ways to install Bigfile, you can find more detailed documentation here [English](https://bigfile.site/start/) [简体中文](https://learnku.com/docs/bigfile)

### Start Bigfile

1. generate certificates

> bigfile rpc:make-cert

2. start server

> bigfile multi:server
>

This will print some information as follows:

 [2019/09/19 15:38:32.817] 56628 DEBUG bigfile http service listening on: https://0.0.0.0:10985
 [2019/09/19 15:38:32.818] 56628 DEBUG Go FTP Server listening on 2121
 [2019/09/19 15:38:32.819] 56628 DEBUG bigfile rpc service listening on: tcp://[::]:10986


### HTTP Example (Token Create)

```go
package main

import (
	"fmt"
	"io/ioutil"
	libHttp "net/http"
	"strings"
	"time"

	"github.com/bigfile/bigfile/databases/models"
	"github.com/bigfile/bigfile/http"
)

func main() {
	appUid := "42c4fcc1a620c9e97188f50b6f2ab199"
	appSecret := "f8f2ae1fe4f70b788254dcc991a35558"
	body := http.GetParamsSignBody(map[string]interface{}{
		"appUid": appUid,
		"nonce": models.RandomWithMD5(128),
		"path": "/images/png",
		"expiredAt": time.Now().AddDate(0, 0, 2).Unix(),
		"secret": models.RandomWithMD5(44),
		"availableTimes": -1,
		"readOnly": false,
	}, appSecret)
	request, err := libHttp.NewRequest(
		"POST", "https://127.0.0.1:10985/api/bigfile/token/create", strings.NewReader(body))
	if err != nil {
		fmt.Println(err)
		return
	}
	request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
	resp, err := libHttp.DefaultClient.Do(request)
	if err != nil {
		fmt.Println(err)
		return
	}
	if bodyBytes, err := ioutil.ReadAll(resp.Body); err != nil {
		fmt.Println(err)
		return
	} else {
		fmt.Println(string(bodyBytes))
	}
}
```

### RPC Example (Token Create)

```go
package main

import (
	"crypto/tls"
	"crypto/x509"
	"fmt"
	"io/ioutil"

	"google.golang.org/grpc"
	"github.com/bigfile/bigfile/rpc"
	"google.golang.org/grpc/credentials"
)

func createConnection() (*grpc.ClientConn, error) {
	var (
		err error
		conn *grpc.ClientConn
		cert tls.Certificate
		certPool *x509.CertPool
		rootCertBytes []byte
	)
	if cert, err = tls.LoadX509KeyPair("client.pem", "client.key"); err != nil {
		return nil, err
	}

	certPool = x509.NewCertPool()
	if rootCertBytes, err = ioutil.ReadFile("ca.pem"); err != nil {
		return nil, err
	}

	if !certPool.AppendCertsFromPEM(rootCertBytes) {
		return nil, err
	}

	if conn, err = grpc.Dial("192.168.0.103:10986", grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{
		Certificates: []tls.Certificate{cert},
		RootCAs: certPool,
	}))); err != nil {
		return nil, err
	}
	return conn, err
}

func main() {
	var (
		err error
		conn *grpc.ClientConn
	)

	if conn, err = createConnection(); err != nil {
		fmt.Println(err)
		return
	}
	defer conn.Close()
	grpcClient := rpc.NewTokenCreateClient(conn)
 	fmt.Println(grpcClient.TokenCreate(context.TODO(), &rpc.TokenCreateRequest{
 		AppUid: "42c4fcc1a620c9e97188f50b6f2ab199",
 		AppSecret: "f8f2ae1fe4f70b788254dcc991a35558",
 }))
}
```

### FTP Example

![ftp-example](https://bigfile.site/ftp-connect-succesfully.png)

[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile?ref=badge_large)
[![Get it from the Snap Store](https://snapcraft.io/static/images/badges/en/snap-store-white.svg)](https://snapcraft.io/bigfile)

bigfile

# copy

[![Go Reference](https://pkg.go.dev/badge/github.com/otiai10/copy.svg)](https://pkg.go.dev/github.com/otiai10/copy)
[![Actions Status](https://github.com/otiai10/copy/workflows/Go/badge.svg)](https://github.com/otiai10/copy/actions)
[![codecov](https://codecov.io/gh/otiai10/copy/branch/main/graph/badge.svg)](https://codecov.io/gh/otiai10/copy)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://github.com/otiai10/copy/blob/main/LICENSE)
[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fotiai10%2Fcopy.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2Fotiai10%2Fcopy?ref=badge_shield)
[![CodeQL](https://github.com/otiai10/copy/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/otiai10/copy/actions/workflows/codeql-analysis.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/otiai10/copy)](https://goreportcard.com/report/github.com/otiai10/copy)
[![GitHub tag (latest SemVer)](https://img.shields.io/github/v/tag/otiai10/copy?sort=semver)](https://pkg.go.dev/github.com/otiai10/copy)
[![Docker Test](https://github.com/otiai10/copy/actions/workflows/docker-test.yml/badge.svg)](https://github.com/otiai10/copy/actions/workflows/docker-test.yml)
[![Vagrant Test](https://github.com/otiai10/copy/actions/workflows/vagrant-test.yml/badge.svg)](https://github.com/otiai10/copy/actions/workflows/vagrant-test.yml)
[![GopherJS](https://github.com/otiai10/copy/actions/workflows/gopherjs.yml/badge.svg)](https://github.com/otiai10/copy/actions/workflows/gopherjs.yml)
[![Go WASM](https://github.com/otiai10/copy/actions/workflows/wasm.yml/badge.svg)](https://github.com/otiai10/copy/actions/workflows/wasm.yml)

`copy` copies directories recursively.

# Example Usage

```go
package main

import (
	"fmt"
	cp "github.com/otiai10/copy"
)

func main() {
	err := cp.Copy("your/src", "your/dest")
	fmt.Println(err) // nil
}
```

# Advanced Usage

```go
// Options specifies optional actions on copying.
type Options struct {

	// OnSymlink can specify what to do on symlink
	OnSymlink func(src string) SymlinkAction

	// OnDirExists can specify what to do when there is a directory already existing in destination.
	OnDirExists func(src, dest string) DirExistsAction

	// OnError can let users decide how to handle errors (e.g., you can suppress specific error).
	OnError func(src, dest, string, err error) error

	// Skip can specify which files should be skipped
	Skip func(srcinfo os.FileInfo, src, dest string) (bool, error)

	// RenameDestination can rename destination.
	// If not set, nil, it does nothing.
	RenameDestination func(src, dest string) (string, error)

	// PermissionControl can control permission of
	// every entry.
	// When you want to add permission 0222, do like
	//
	//		PermissionControl = AddPermission(0222)
	//
	// or if you even don't want to touch permission,
	//
	//		PermissionControl = DoNothing
	//
	// By default, PermissionControl = PreservePermission
	PermissionControl PermissionControlFunc

	// Sync file after copy.
	// Useful in case when file must be on the disk
	// (in case crash happens, for example),
	// at the expense of some performance penalty
	Sync bool

	// Preserve the atime and the mtime of the entries
	// On linux we can preserve only up to 1 millisecond accuracy
	PreserveTimes bool

	// Preserve the uid and the gid of all entries.
	PreserveOwner bool

	// The byte size of the buffer to use for copying files.
	// If zero, the internal default buffer of 32KB is used.
	// See https://golang.org/pkg/io/#CopyBuffer for more information.
	CopyBufferSize uint

	// If you want to add some limitation on reading src file,
	// you can wrap the src and provide new reader,
	// such as `RateLimitReader` in the test case.
	WrapReader func(src io.Reader) io.Reader

	// If given, copy.Copy refers to this fs.FS instead of the OS filesystem.
	// e.g., You can use embed.FS to copy files from embedded filesystem.
	FS fs.FS

	// NumOfWorkers represents the number of workers used for
	// concurrent copying contents of directories.
	// If 0 or 1, it does not use goroutine for copying directories.
	// Please refer to https://pkg.go.dev/golang.org/x/sync/semaphore for more details.
	NumOfWorkers int64

	// PreferConcurrent is a function to determine whether or not
	// to use goroutine for copying contents of directories.
	// If PreferConcurrent is nil, which is default, it does concurrent
	// copying for all directories.
	// If NumOfWorkers is 0 or 1, this function will be ignored.
	PreferConcurrent func(srcdir, destdir string) (bool, error)
}
```

```go
// For example...
opt := Options{
	Skip: func(info os.FileInfo, src, dest string) (bool, error) {
		return strings.HasSuffix(src, ".git"), nil
	},
}
err := Copy("your/directory", "your/directory.copy", opt)
```

# Issues

- https://github.com/otiai10/copy/issues


## License
[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fotiai10%2Fcopy.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fotiai10%2Fcopy?ref=badge_large)


copy

File operations library which aims to mirror feature parity with [GNU cp](https://www.gnu.org/software/coreutils/manual/html_node/cp-invocation.html)

# flop

[![GoDoc](https://godoc.org/github.com/homedepot/flop?status.svg)](https://godoc.org/github.com/homedepot/flop)
[![MIT license](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)
[![Go Report Card](https://goreportcard.com/badge/github.com/homedepot/flop)](https://goreportcard.com/report/github.com/homedepot/flop)

<img src="https://github.com/homedepot/flop/raw/master/doc/logo.png" width="200" title="design based on original gopher by Renee French">

----

flop aims to make copying files easier in Go, and is modeled after
[GNU cp](https://www.gnu.org/software/coreutils/manual/html_node/cp-invocation.html).
Most administrators and engineers interact with GNU utilities every day, so it makes sense to utilize
that knowledge and expectations for a library that does the same operation in code.

flop strategically diverges from cp where it is advantageous for the programmer to explicitly define the
behavior, like cp assuming that copying from a file path to a directory path means the file should be
created inside the directory. This behavior must be explicitly defined in flop by passing the option
AppendNameToPath, otherwise an error will be returned.

```BASH
go get -u github.com/homedepot/flop
```

## Usage

Basic file copy.

```go
err := flop.SimpleCopy("src_path", "dst_path")
handle(err)
```

Advanced file copy with [options](https://pkg.go.dev/github.com/homedepot/flop?tab=doc#Options).

```go
options := flop.Options{
 Recursive: true,
 MkdirAll: true,
}
err := flop.Copy("src_path", "dst_path", options)
handle(err)
```

## Logging

flop won't throw logs at you for no reason, but if you want to follow along with what's going on giving it a logger
can help expose the behavior, or aid in debugging if you are generous enough to contribute.

```go
// the logger just takes a string so format your favorite logger to accept one
import (
	"github.com/homedepot/flop"
	"github.com/rs/zerolog"
	zlog "github.com/rs/zerolog/log"
	llog "github.com/sirupsen/logrus"
)

func logDebug(msg string) {
	llog.WithFields(llog.Fields{
		"application": "stuffcopy",
	}).Info(msg)
}

func main() {
	zlog.Logger = zlog.Output(zerolog.ConsoleWriter{Out: os.Stderr})
	err := flop.Copy(src.Name(), dst.Name(), flop.Options{
		InfoLogFunc: zlog.Info().Msg, // Msg already accepts a string so we can just pass it directly
		DebugLogFunc: logDebug, // logrus Debug takes ...interface{} so we need to wrap it
	})
	handle(err)
}
```

flop

Disk usage analyzer with console interface

# go DiskUsage()

<img src="./gdu.png" alt="Gdu " width="200" align="right">

[![Codecov](https://codecov.io/gh/dundee/gdu/branch/master/graph/badge.svg)](https://codecov.io/gh/dundee/gdu)
[![Go Report Card](https://goreportcard.com/badge/github.com/dundee/gdu)](https://goreportcard.com/report/github.com/dundee/gdu)
[![Maintainability](https://api.codeclimate.com/v1/badges/30d793274607f599e658/maintainability)](https://codeclimate.com/github/dundee/gdu/maintainability)
[![CodeScene Code Health](https://codescene.io/projects/13129/status-badges/code-health)](https://codescene.io/projects/13129)

Pretty fast disk usage analyzer written in Go.

Gdu is intended primarily for SSD disks where it can fully utilize parallel processing.
However HDDs work as well, but the performance gain is not so huge.

[![asciicast](https://asciinema.org/a/382738.svg)](https://asciinema.org/a/382738)

<a href="https://repology.org/project/gdu/versions">
 <img src="https://repology.org/badge/vertical-allrepos/gdu.svg" alt="Packaging status" align="right">
</a>

## Installation

Head for the [releases page](https://github.com/dundee/gdu/releases) and download the binary for your system.

Using curl:

 curl -L https://github.com/dundee/gdu/releases/latest/download/gdu_linux_amd64.tgz | tar xz
 chmod +x gdu_linux_amd64
 mv gdu_linux_amd64 /usr/bin/gdu

See the [installation page](./INSTALL.md) for other ways how to install Gdu to your system.

Or you can use Gdu directly via Docker:

 docker run --rm --init --interactive --tty --privileged --volume /:/mnt/root ghcr.io/dundee/gdu /mnt/root

## Usage

```
 gdu [directory_to_scan] [flags]

Flags:
 --archive-browsing Enable browsing of zip/jar/tar archives (tar, tar.gz, tar.bz2, tar.xz)
 --collapse-path Collapse single-child directory chains
 --config-file string Read config from file (default is $HOME/.gdu.yaml)
 -D, --db string Store analysis in database (*.sqlite for SQLite, *.badger for BadgerDB)
 --depth int Show directory structure up to specified depth in non-interactive mode (0 means the flag is ignored)
 --enable-profiling Enable collection of profiling data and provide it on http://localhost:6060/debug/pprof/
 -E, --exclude-type strings File types to exclude (e.g., --exclude-type yaml,json)
 -L, --follow-symlinks Follow symlinks for files, i.e. show the size of the file to which symlink points to (symlinks to directories are not followed)
 -h, --help help for gdu
 -i, --ignore-dirs strings Paths to ignore (separated by comma). Can be absolute or relative to current directory (default [/proc,/dev,/sys,/run])
 -I, --ignore-dirs-pattern strings Path patterns to ignore (separated by comma)
 -X, --ignore-from string Read path patterns to ignore from file
 -f, --input-file string Import analysis from JSON file
 --interactive Force interactive mode even when output is not a TTY
 -l, --log-file string Path to a logfile (default "/dev/null")
 --max-age string Include files with mtime no older than DURATION (e.g., 7d, 2h30m, 1y2mo)
 -m, --max-cores int Set max cores that Gdu will use. 8 cores available (default 8)
 --min-age string Include files with mtime at least DURATION old (e.g., 30d, 1w)
 --mouse Use mouse
 -c, --no-color Do not use colorized output
 -x, --no-cross Do not cross filesystem boundaries
 --no-delete Do not allow deletions
 -H, --no-hidden Ignore hidden directories (beginning with dot)
 --no-prefix Show sizes as raw numbers without any prefixes (SI or binary) in non-interactive mode
 -p, --no-progress Do not show progress in non-interactive mode
 --no-spawn-shell Do not allow spawning shell
 -u, --no-unicode Do not use Unicode symbols (for size bar)
 --no-view-file Do not allow viewing file contents
 -n, --non-interactive Do not run in interactive mode
 -o, --output-file string Export all info into file as JSON
 -r, --read-from-storage Use existing database instead of re-scanning
 --reverse-sort Reverse sorting order (smallest to largest) in non-interactive mode
 --sequential Use sequential scanning (intended for rotating HDDs)
 -A, --show-annexed-size Use apparent size of git-annex'ed files in case files are not present locally (real usage is zero)
 -a, --show-apparent-size Show apparent size
 -d, --show-disks Show all mounted disks
 -k, --show-in-kib Show sizes in KiB (or kB with --si) in non-interactive mode
 -C, --show-item-count Show number of items in directory
 -M, --show-mtime Show latest mtime of items in directory
 -B, --show-relative-size Show relative size
 --si Show sizes with decimal SI prefixes (kB, MB, GB) instead of binary prefixes (KiB, MiB, GiB)
 --since string Include files with mtime >= WHEN. WHEN accepts RFC3339 timestamp (e.g., 2025-08-11T01:00:00-07:00) or date only YYYY-MM-DD (calendar-day compare; includes the whole day)
 -s, --summarize Show only a total in non-interactive mode
 -t, --top int Show only top X largest files in non-interactive mode
 -T, --type strings File types to include (e.g., --type yaml,json)
 --until string Include files with mtime <= WHEN. WHEN accepts RFC3339 timestamp or date only YYYY-MM-DD
 -v, --version Print version
 --write-config Write current configuration to file (default is $HOME/.gdu.yaml)

Basic list of actions in interactive mode (show help modal for more):
 ↑ or k Move cursor up
 ↓ or j Move cursor down
 → or Enter or l Go to highlighted directory
 ← or h Go to parent directory
 d Delete the selected file or directory
 e Empty the selected directory
 n Sort by name
 s Sort by size
 c Show number of items in directory
 ? Show help modal
```

## Examples

 gdu # analyze current dir
 gdu -a # show apparent size instead of disk usage
 gdu --no-delete # prevent write operations
 gdu --no-view-file # prevent viewing file contents
 gdu <some_dir_to_analyze> # analyze given dir
 gdu -d # show all mounted disks
 gdu -l ./gdu.log <some_dir> # write errors to log file
 gdu -i /sys,/proc / # ignore some paths
 gdu -I '.*[abc]+' # ignore paths by regular pattern
 gdu -X ignore_file / # ignore paths by regular patterns from file
 gdu -c / # use only white/gray/black colors

 gdu -n / # only print stats, do not start interactive mode
 gdu --interactive / | tee out.txt # force interactive mode even when stdout is piped
 gdu -p / # do not show progress, useful when using its output in a script
 gdu -ps /some/dir # show only total usage for given dir
 gdu -t 10 / # show top 10 largest files
 gdu --reverse-sort -n / # show files sorted from smallest to largest in non-interactive mode
 gdu / > file # write stats to file, do not start interactive mode

 gdu -o- / | gzip -c >report.json.gz # write all info to JSON file for later analysis
 zcat report.json.gz | gdu -f- # read analysis from file

 GOGC=10 gdu -g --use-storage / # use persistent key-value storage for saving analysis data
 gdu -r / # read saved analysis data from persistent key-value storage

## Modes

Gdu has three modes: interactive (default), non-interactive and export.

Non-interactive mode is started automatically when TTY is not detected (using [go-isatty](https://github.com/mattn/go-isatty)), for example if the output is being piped to a file, or it can be started explicitly by using a flag. Use `--interactive` to disable this automatic fallback and force interactive mode.

Export mode (flag `-o`) outputs all usage data as JSON, which can be later opened using the `-f` flag.

Hard links are counted only once.

## File flags

Files and directories may be prefixed by a one-character
flag with following meaning:

* `!` An error occurred while reading this directory.

* `.` An error occurred while reading a subdirectory, size may be not correct.

* `@` File is symlink or socket.

* `H` Same file was already counted (hard link).

* `e` Directory is empty.

## Configuration file

Gdu can read (and write) YAML configuration file.

`$HOME/.config/gdu/gdu.yaml` and `$HOME/.gdu.yaml` are checked for the presence of the config file by default.

See the [full list of all configuration options](configuration.md).

### Examples

* To configure gdu to permanently run in gray-scale color mode:

```
echo "no-color: true" >> ~/.gdu.yaml
```

* To set default sorting in configuration file:

```
sorting:
 by: name // size, name, itemCount, mtime
 order: desc
```

* To configure gdu to set CWD variable when browsing directories:

```
echo "change-cwd: true" >> ~/.gdu.yaml
```

* To save the current configuration

```
gdu --write-config
```

## Styling

There are wide options for how terminals can be colored.
Some gdu primitives (like basic text) adapt to different color schemas, but the selected/highlighted row does not.

If the default look is not sufficient, it can be changed in configuration file, e.g.:

```
style:
 selected-row:
 text-color: black
 background-color: "#ff0000"
```

## Deletion in background and in parallel (experimental)

Gdu can delete items in the background, thus not blocking the UI for additional work.
To enable:

```
echo "delete-in-background: true" >> ~/.gdu.yaml
```

Directory items can be also deleted in parallel, which might increase the speed of deletion.
To enable:

```
echo "delete-in-parallel: true" >> ~/.gdu.yaml
```

## Saving analysis data to database

Gdu can store the analysis data to a database file instead of just memory.
This allows you to save and reload analysis results later.
Both SQLite and BadgerDB are supported.

```
gdu --db analysis.sqlite / # saves analysis data to SQLite database
gdu --db analysis.badger / # saves analysis data to BadgerDB
gdu -r --db analysis.sqlite / # reads saved data, does not run analysis again
```

## Running tests

 make install-dev-dependencies
 make test

## Profiling

Gdu can collect profiling data when the `--enable-profiling` flag is set.
The data are provided via embedded http server on URL `http://localhost:6060/debug/pprof/`.

You can then use e.g. `go tool pprof -web http://localhost:6060/debug/pprof/heap`
to open the heap profile as SVG image in your web browser.

## Benchmarks

Benchmarks were performed on 50G directory (100k directories, 400k files) on 500 GB SSD using [hyperfine](https://github.com/sharkdp/hyperfine).
See `benchmark` target in [Makefile](Makefile) for more info.

### Cold cache

Filesystem cache was cleared using `sync; echo 3 | sudo tee /proc/sys/vm/drop_caches`.

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `diskus ~` | 3.074 ± 0.010 | 3.056 | 3.094 | 1.00 |
| `gdu -npc ~` | 3.133 ± 0.013 | 3.116 | 3.159 | 1.02 ± 0.01 |
| `gdu -gnpc ~` | 3.157 ± 0.013 | 3.139 | 3.180 | 1.03 ± 0.01 |
| `pdu ~` | 3.772 ± 0.149 | 3.630 | 4.071 | 1.23 ± 0.05 |
| `dust -d0 ~` | 4.001 ± 0.162 | 3.786 | 4.305 | 1.30 ± 0.05 |
| `dua ~` | 5.315 ± 3.210 | 4.068 | 14.447 | 1.73 ± 1.04 |
| `gdu -npc --use-storage ~` | 12.690 ± 0.527 | 11.325 | 13.091 | 4.13 ± 0.17 |
| `du -hs ~` | 14.940 ± 0.064 | 14.852 | 15.048 | 4.86 ± 0.03 |
| `duc index ~` | 15.501 ± 0.136 | 15.386 | 15.849 | 5.04 ± 0.05 |
| `ncdu -0 -o /dev/null ~` | 15.688 ± 0.053 | 15.610 | 15.789 | 5.10 ± 0.02 |

### Warm cache

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `diskus ~` | 211.4 ± 3.7 | 206.4 | 219.3 | 1.00 |
| `pdu ~` | 221.8 ± 2.4 | 219.3 | 226.3 | 1.05 ± 0.02 |
| `dust -d0 ~` | 363.6 ± 5.4 | 357.3 | 373.2 | 1.72 ± 0.04 |
| `gdu -npc ~` | 434.3 ± 3.4 | 426.0 | 437.8 | 2.05 ± 0.04 |
| `dua ~` | 451.2 ± 4.2 | 444.9 | 457.9 | 2.13 ± 0.04 |
| `gdu -gnpc ~` | 521.0 ± 14.0 | 510.9 | 558.5 | 2.46 ± 0.08 |
| `du -hs ~` | 809.4 ± 3.2 | 804.8 | 816.0 | 3.83 ± 0.07 |
| `duc index ~` | 952.3 ± 4.8 | 946.0 | 961.7 | 4.50 ± 0.08 |
| `ncdu -0 -o /dev/null ~` | 1432.8 ± 3.4 | 1428.0 | 1439.0 | 6.78 ± 0.12 |
| `gdu -npc --use-storage ~` | 9950.0 ± 474.1 | 9117.5 | 10647.4 | 47.07 ± 2.39 |

## Alternatives

* [ncdu](https://dev.yorhel.nl/ncdu) - NCurses based tool written in pure `C` (LTS) or `zig` (Stable)
* [godu](https://github.com/viktomas/godu) - Analyzer with a carousel like user interface
* [dua](https://github.com/Byron/dua-cli) - Tool written in `Rust` with interface similar to gdu (and ncdu)
* [diskus](https://github.com/sharkdp/diskus) - Very simple but very fast tool written in `Rust`
* [duc](https://duc.zevv.nl/) - Collection of tools with many possibilities for inspecting and visualising disk usage
* [dust](https://github.com/bootandy/dust) - Tool written in `Rust` showing tree like structures of disk usage
* [pdu](https://github.com/KSXGitHub/parallel-disk-usage) - Tool written in `Rust` showing tree like structures of disk usage

## Notes

[HDD icon created by Nikita Golubev - Flaticon](https://www.flaticon.com/free-icons/hdd)

Go bindings for ExifTool, the well-known library used to extract as much metadata as possible (EXIF, IPTC, ...) from files (pictures, PDF, office, ...)

go-exiftool

A package to convert an HTML template to a PDF file

[![PkgGoDev](https://pkg.go.dev/badge/github.com/SebastiaanKlippert/go-wkhtmltopdf)](https://pkg.go.dev/github.com/SebastiaanKlippert/go-wkhtmltopdf)
[![Go Report Card](https://goreportcard.com/badge/SebastiaanKlippert/go-wkhtmltopdf)](https://goreportcard.com/report/SebastiaanKlippert/go-wkhtmltopdf)
[![codecov](https://codecov.io/gh/SebastiaanKlippert/go-wkhtmltopdf/branch/master/graph/badge.svg)](https://codecov.io/gh/SebastiaanKlippert/go-wkhtmltopdf)

[![Build Status](https://github.com/SebastiaanKlippert/go-wkhtmltopdf/actions/workflows/ubuntu.yml/badge.svg?branch=master)](https://github.com/SebastiaanKlippert/go-wkhtmltopdf/actions/workflows/ubuntu.yml)
[![Build Status](https://github.com/SebastiaanKlippert/go-wkhtmltopdf/actions/workflows/macos.yml/badge.svg?branch=master)](https://github.com/SebastiaanKlippert/go-wkhtmltopdf/actions/workflows/macos.yml)

# go-wkhtmltopdf
Golang commandline wrapper for wkhtmltopdf

See http://wkhtmltopdf.org/index.html for wkhtmltopdf docs.

| :warning: WARNING |
|:---------------------------|
| wkhtmltopdf is no longer maintained and now archived on GitHub. See https://wkhtmltopdf.org/status.html |
| This go package is still maintained and will be for a while, but I recommend looking for alternatives for new projects. |


# What and why
We needed a way to generate PDF documents from Go. These vary from invoices with highly customizable lay-outs to reports with tables, graphs and images. In our opinion the best way to do this was by using HTML/CSS templates as source for our PDFs. Using CSS print media types and millimeters instead of pixel units we can generate very acurate PDF documents using wkhtmltopdf.

go-wkhtmltopdf is a pure Golang wrapper around the wkhtmltopdf command line utility.

It has all options typed out as struct members which makes it very easy to use if you use an IDE with
code completion and it has type safety for all options.
For example you can set general options like
```go
pdfg.Dpi.Set(600)
pdfg.NoCollate.Set(false)
pdfg.PageSize.Set(PageSizeA4)
pdfg.MarginBottom.Set(40)
``` 
The same goes for adding pages, settings page options, TOC options per page etc.

It takes care of setting the correct order of options as these can become very long with muliple pages where 
you have page and TOC options for each page.

Secondly it makes usage in server-type applications easier, every instance (PDF process) has its own output buffer 
which contains the PDF output and you can feed one input document from an io.Reader (using stdin in wkhtmltopdf).
You can combine any number of external HTML documents (HTTP(S) links) with at most one HTML document from stdin and set 
options for each input document.

Note: You can also ignore the internal buffer and let wkhtmltopdf write directly to disk if required for large files, or use the [SetOutput](https://godoc.org/github.com/SebastiaanKlippert/go-wkhtmltopdf#PDFGenerator.SetOutput) method to pass any `io.Writer`.

For us this is one of the easiest ways to generate PDF documents from Go(lang) and performance is very acceptable.

# Installation
go get or use a Go dependency manager of your liking.

```
go get -u github.com/SebastiaanKlippert/go-wkhtmltopdf
```

go-wkhtmltopdf finds the path to wkhtmltopdf by
* first looking in the current dir
* looking in the PATH and PATHEXT environment dirs
* using the WKHTMLTOPDF_PATH environment dir

**Warning**: Running executables from the current path is no longer possible in Go 1.19, see https://pkg.go.dev/os/exec@master#hdr-Executables_in_the_current_directory

If you need to set your own wkhtmltopdf path or want to change it during execution, you can call SetPath().

# Usage
See testfile ```wkhtmltopdf_test.go``` for more complex options, a common use case test is in ```simplesample_test.go``` 

```go
package wkhtmltopdf

import (
 "fmt"
 "log"
)

func ExampleNewPDFGenerator() {

 // Create new PDF generator
 pdfg, err := NewPDFGenerator()
 if err != nil {
 log.Fatal(err)
 }

 // Set global options
 pdfg.Dpi.Set(300)
 pdfg.Orientation.Set(OrientationLandscape)
 pdfg.Grayscale.Set(true)

 // Create a new input page from an URL
 page := NewPage("https://godoc.org/github.com/SebastiaanKlippert/go-wkhtmltopdf")

 // Set options for this page
 page.FooterRight.Set("[page]")
 page.FooterFontSize.Set(10)
 page.Zoom.Set(0.95)

 // Add to document
 pdfg.AddPage(page)

 // Create PDF document in internal buffer
 err = pdfg.Create()
 if err != nil {
 log.Fatal(err)
 }

 // Write buffer contents to file on disk
 err = pdfg.WriteFile("./simplesample.pdf")
 if err != nil {
 log.Fatal(err)
 }

 fmt.Println("Done")
 // Output: Done
}
```

As mentioned before, you can provide one document from stdin, this is done by using a [PageReader](https://godoc.org/github.com/SebastiaanKlippert/go-wkhtmltopdf#PageReader "GoDoc") object as input to AddPage. This is best constructed with [NewPageReader](https://godoc.org/github.com/SebastiaanKlippert/go-wkhtmltopdf#NewPageReader "GoDoc") and will accept any io.Reader so this can be used with files from disk (os.File) or memory (bytes.Buffer) etc. 
A simple example snippet:
```go
html := "<html>Hi</html>"
pdfgen.AddPage(NewPageReader(strings.NewReader(html)))
```

# Saving to and loading from JSON

The package now has the possibility to save the PDF Generator object as JSON and to create
a new PDF Generator from a JSON file.
All options and pages are saved in JSON, pages added using NewPageReader are read to memory before saving and then saved as Base64 encoded strings
in the JSON file.

This is useful to prepare a PDF file and generate the actual PDF elsewhere, for example on AWS Lambda.
To create PDF Generator on the client, where wkhtmltopdf might not be present, function `NewPDFPreparer` can be used.

Use `NewPDFPreparer` to create a PDF Generator object on the client and `NewPDFGeneratorFromJSON` to reconstruct it on the server.

```go 
// Client code
pdfg := NewPDFPreparer()
htmlfile, err := ioutil.ReadFile("testdata/htmlsimple.html")
if err != nil {
 log.Fatal(err)
}
 
pdfg.AddPage(NewPageReader(bytes.NewReader(htmlfile)))
pdfg.Dpi.Set(600)
 
// The contents of htmlsimple.html are saved as base64 string in the JSON file
jb, err := pdfg.ToJSON()
if err != nil {
 log.Fatal(err)
}
 
// Server code
pdfgFromJSON, err := NewPDFGeneratorFromJSON(bytes.NewReader(jb))
if err != nil {
 log.Fatal(err)
}
 
err = pdfgFromJSON.Create()
if err != nil {
 log.Fatal(err)
} 
```

For an example of running this in AWS Lambda see https://github.com/SebastiaanKlippert/go-wkhtmltopdf-lambda

# Speed 
The speed if pretty much determined by wkhtmltopdf itself, or if you use external source URLs, the time it takes to get and render the source HTML.

The go wrapper time is negligible with around 0.04ms for parsing an above average number of commandline options.

Benchmarks are included.

go-wkhtmltopdf

A cross-platform real-time file synchronization tool out of the box

gofs

A package for reading and creating ISO9660 disk images

## iso9660
[![Go Reference](https://pkg.go.dev/badge/github.com/kdomanski/iso9660.svg)](https://pkg.go.dev/github.com/kdomanski/iso9660)
[![codecov](https://codecov.io/gh/kdomanski/iso9660/branch/master/graph/badge.svg?token=14MJSZYZ24)](https://codecov.io/gh/kdomanski/iso9660)
[![Go Report Card](https://goreportcard.com/badge/github.com/kdomanski/iso9660)](https://goreportcard.com/report/github.com/kdomanski/iso9660)

A package for reading and creating ISO9660

Joliet extension is **NOT** supported.

Experimental support for reading Rock Ridge extension is currently in the works.
If you are experiencing issues, please use the v0.3 release, which ignores Rock Ridge.

## References for the format:
- [ECMA-119 1st edition (December 1986)](https://www.ecma-international.org/wp-content/uploads/ECMA-119_1st_edition_december_1986.pdf) ([Web Archive link](http://web.archive.org/web/20210122025258/https://www.ecma-international.org/wp-content/uploads/ECMA-119_1st_edition_december_1986.pdf))
- [ECMA-119 2nd edition (December 1987)](https://www.ecma-international.org/wp-content/uploads/ECMA-119_2nd_edition_december_1987.pdf) ([Web Archive link](http://web.archive.org/web/20210418211711/https://www.ecma-international.org/wp-content/uploads/ECMA-119_2nd_edition_december_1987.pdf))
- [ECMA-119 3rd edition (December 2017)](https://www.ecma-international.org/wp-content/uploads/ECMA-119_3rd_edition_december_2017.pdf) ([Web Archive link](http://web.archive.org/web/20210527165925/https://www.ecma-international.org/wp-content/uploads/ECMA-119_3rd_edition_december_2017.pdf))
- [ECMA-119 4th edition (June 2019)](https://www.ecma-international.org/wp-content/uploads/ECMA-119_4th_edition_june_2019.pdf) ([Web Archive link](https://www.ecma-international.org/wp-content/uploads/ECMA-119_4th_edition_june_2019.pdf))
- [Rock Ridge Interchange Protocol](http://www.nextcomputers.org/NeXTfiles/Projects/CD-ROM/Rock_Ridge_Interchange_Protocol.pdf) ([Web Archive link](http://web.archive.org/web/20071017082049/http://www.nextcomputers.org/NeXTfiles/Projects/CD-ROM/Rock_Ridge_Interchange_Protocol.pdf))
- [System Use Sharing Protocol v1.12](http://aminet.net/package/docs/misc/RRIP)

## Examples

### Extracting an ISO

```go
package main

import (
  "log"

  "github.com/kdomanski/iso9660/util"
)

func main() {
  f, err := os.Open("/home/user/myImage.iso")
  if err != nil {
    log.Fatalf("failed to open file: %s", err)
  }
  defer f.Close()

  if err = util.ExtractImageToDirectory(f, "/home/user/target_dir"); err != nil {
    log.Fatalf("failed to extract image: %s", err)
  }
}
```

### Creating an ISO

```go
package main

import (
  "log"
  "os"

  "github.com/kdomanski/iso9660"
)

func main() {
  writer, err := iso9660.NewWriter()
  if err != nil {
    log.Fatalf("failed to create writer: %s", err)
  }
  defer writer.Cleanup()

  f, err := os.Open("/home/user/myFile.txt")
  if err != nil {
    log.Fatalf("failed to open file: %s", err)
  }
  defer f.Close()

  err = writer.AddFile(f, "folder/MYFILE.TXT")
  if err != nil {
    log.Fatalf("failed to add file: %s", err)
  }

  outputFile, err := os.OpenFile("/home/user/output.iso", os.O_WRONLY | os.O_TRUNC | os.O_CREATE, 0644)
  if err != nil {
    log.Fatalf("failed to create file: %s", err)
  }

  err = writer.WriteTo(outputFile, "testvol")
  if err != nil {
    log.Fatalf("failed to write ISO image: %s", err)
  }

  err = outputFile.Close()
  if err != nil {
    log.Fatalf("failed to close output file: %s", err)
  }
}
```

### Recursively create an ISO image from the given directories

```go
package main

import (
  "fmt"
  "log"
  "os"
  "path/filepath"
  "strings"

  "github.com/kdomanski/iso9660"
)

func main() {
  writer, err := iso9660.NewWriter()
  if err != nil {
    log.Fatalf("failed to create writer: %s", err)
  }
  defer writer.Cleanup()

  isoFile, err := os.OpenFile("C:/output.iso", os.O_WRONLY|os.O_TRUNC|os.O_CREATE, 0644)
  if err != nil {
    log.Fatalf("failed to create file: %s", err)
  }
  defer isoFile.Close()

  prefix := "F:\\" // the prefix to remove in the output iso file
  sourceFolders := []string{"F:\\test1", "F:\\test2"} // the given directories to create an ISO file from

  for _, folderName := range sourceFolders {
    folderPath := strings.Join([]string{prefix, folderName}, "/")

    walk_err := filepath.Walk(folderPath, func(path string, info os.FileInfo, err error) error {
      if err != nil {
        log.Fatalf("walk: %s", err)
        return err
      }
      if info.IsDir() {
        return nil
      }
      outputPath := strings.TrimPrefix(path, prefix) // remove the source drive name
      fmt.Printf("Adding file: %s\n", outputPath)

      fileToAdd, err := os.Open(path)
      if err != nil {
        log.Fatalf("failed to open file: %s", err)
      }
      defer fileToAdd.Close()

      err = writer.AddFile(fileToAdd, outputPath)
      if err != nil {
        log.Fatalf("failed to add file: %s", err)
      }
      return nil
    })
    if walk_err != nil {
      log.Fatalf("%s", walk_err)
    }
  }

  err = writer.WriteTo(isoFile, "Test")
  if err != nil {
    log.Fatalf("failed to write ISO image: %s", err)
  }
}
```


iso9660

Read and write [parquet](https://parquet.apache.org) files

# Parquet

Parquet generates a parquet reader and writer based on a struct.  The struct
can be defined by you or it can be generated by reading an existing parquet file.

We (Parsyl) will respond to pull requests and issues to the best of our
abilities.  However, sometimes we will have higher priorities and the response
might not be immediate.

NOTE: If you generate the code based on a parquet file there are quite a few
limitations.  The PageType of each PageHeader must be DATA_PAGE and the Codec
(defined in ColumnMetaData) must be PLAIN or SNAPPY. Also, the parquet file's
schema must consist of the currently [supported types](#supported-types).  But
wait, there's more!  Some of the encodings, like DELTA_BINARY_PACKED, BIT_PACKED,
PLAIN_DICTIONARY, and DELTA_BYTE_ARRAY are also not supported.  I would guess
there are other parquet options that will cause problems since there are so many
possibilities.

## Installation
    
    go get -u github.com/parsyl/parquet/...

This will also install parquet's only two dependencies: thift and snappy

## Usage

First define a struct for the data to be written to parquet:

```go
type Person struct {
  	ID  int32  `parquet:"id"`
	Age *int32 `parquet:"age"`
}
```

Next, add a go:generate comment somewhere (in this example all code lives
in main.go):

```go
// go:generate parquetgen -input main.go -type Person -package main
```

Generate the code for the reader and writer:

```console
$ go generate
```

A new file (parquet.go) has now been written that defines ParquetWriter
and ParquetReader.  Next, make use of the writer and reader:

```go
package main

import (
    "bytes"
    "encoding/json"
)

func main() {
    var buf bytes.Buffer
    w, err := NewParquetWriter(&buf)
    if err != nil {
        log.Fatal(err)
    }

    w.Add(Person{ID: 1, Age: getAge(30)})
    w.Add(Person{ID: 2})

    // Each call to write creates a new parquet row group.
    if err := w.Write(); err != nil {
        log.Fatal(err)
    }

    // Close must be called when you are done.  It writes
    // the parquet metadata at the end of the file.
    if err := w.Close(); err != nil {
        log.Fatal(err)
    }

    r, err := NewParquetReader(bytes.NewReader(buf.Bytes()))
    if err != nil {
        log.Fatal(err)
    }

    enc := json.NewEncoder(os.Stdout)
    for r.Next() {
        var p Person
        r.Scan(&p)
        enc.Encode(p)
    }

    if err := r.Error(); err != nil {
        log.Fatal(err)
    }
}

func getAge(a int32) *int32 { return &a }
```

NewParquetWriter has a couple of optional arguments available: MaxPageSize,
Uncompressed, and Snappy.  For example, the following sets the page size (number
of rows in a page before a new one is created) and sets the page data compression
to snappy:

```go
w, err := NewParquetWriter(&buf, MaxPageSize(10000), Snappy)
```

See [this](./_examples/people) for a complete example of how to generate the code
based on an existing struct.

See [this](./_examples/via_parquet) for a complete example of how to generate the code
based on an existing parquet file.

## Supported Types 

The struct used to define the parquet data can have the following types:

```
int32
uint32
int64
uint64
float32
float64
string
bool
```

Each of these types may be a pointer to indicate that the data is optional.  The
struct can also embed another struct:

```go
type Being struct {
	ID  int32  `parquet:"id"`
	Age *int32 `parquet:"age"`
}

type Person struct {
	Being
	Username string `parquet:"username"`
}
```

Nested and repeated structs are supported too:

```go
type Being struct {
	ID  int32  `parquet:"id"`
	Age *int32 `parquet:"age"`
}

type Person struct {
	Being    Being
	Username string `parquet:"username"`
	Friends  []Being
}
```

If you want a field to be excluded from parquet you can tag
it with a dash or make it unexported like so:

```go
type Being struct {
  	ID  int32  `parquet:"id"`
	Password string`parquet:"-"` //will not be written to parquet
	age int32                    //will not be written to parquet
}
```

## Parquetgen

Parquetgen is the command that go generate should call in
order to generate the code for your custom type.  It also can
print the page headers and file metadata from a parquet file:

```console
$ parquetgen --help
Usage of parquetgen:
  -ignore
        ignore unsupported fields in -type, otherwise log.Fatal is called when an unsupported type is encountered (default true)
  -import string
        import statement of -type if it doesn't live in -package
  -input string
        path to the go file that defines -type
  -metadata
        print the metadata of a parquet file (-parquet) and exit
  -output string
        name of the file that is produced, defaults to parquet.go (default "parquet.go")
  -package string
        package of the generated code
  -pageheaders
        print the page headers of a parquet file (-parquet) and exit (also prints the metadata)
  -parquet string
        path to a parquet file (if you are generating code based on an existing parquet file or printing the file metadata or page headers)
  -struct-output string
        name of the file that is produced, defaults to parquet.go (default "generated_struct.go")
  -type string
        name of the struct that will used for writing and reading
```


parquet

# pdfcpu: a Go PDF processor and CLI

[![Test](https://github.com/pdfcpu/pdfcpu/workflows/Test/badge.svg)](https://github.com/pdfcpu/pdfcpu/actions)
[![Coverage Status](https://coveralls.io/repos/github/pdfcpu/pdfcpu/badge.svg?branch=master)](https://coveralls.io/github/pdfcpu/pdfcpu?branch=master)
[![GoDoc](https://godoc.org/github.com/pdfcpu/pdfcpu?status.svg)](https://pkg.go.dev/github.com/pdfcpu/pdfcpu)
[![Go Report Card](https://goreportcard.com/badge/github.com/pdfcpu/pdfcpu)](https://goreportcard.com/report/github.com/pdfcpu/pdfcpu)
[![Hex.pm](https://img.shields.io/hexpm/l/plug.svg)](https://opensource.org/licenses/Apache-2.0)
[![Gurubase](https://img.shields.io/badge/Gurubase-Ask%20pdfcpu%20Guru-006BFF)](https://gurubase.io/g/pdfcpu)

[![Latest release](https://img.shields.io/github/release/pdfcpu/pdfcpu.svg)](https://github.com/pdfcpu/pdfcpu/releases)
[![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/hhrutter)


<a href="https://pdfcpu.io"><img src="resources/logoSmall.png" width="150"></a>
<a href="https://pdfa.org"><img src="resources/pdfa.png" width="75"></a>

pdfcpu is a PDF processing library written in [Go](https://go.dev/) that supports encryption and offers both an API and a command-line interface (CLI). It is compatible with all PDF versions with basic support and ongoing improvement for PDF 2.0 (ISO-32000-2).


## Motivation

This is an effort to build a comprehensive PDF processing library from the ground up written in Go. Over time pdfcpu aims to support the standard range of PDF processing features and also any interesting use cases that may present themselves along the way.


 <kbd><a href="https://pdfcpu.io/generate/grid"><img src="resources/gridpdf.png" height="150"></a></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/core/watermark"><img src="resources/wmi1abs.png" height="150"></a></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/generate/nup"><img src="resources/nup9pdf.png" height="150"></a></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/fonts/fonts"><img src="resources/cjkv.png" height="150"></a></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/core/stamp"><img src="resources/4exp.png" height="150"></a></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/form/form"><img src="resources/form.png" height="150"></a></kbd> 
 <kbd><a href="https://pdfcpu.io/generate/create"><img src="resources/table.png" height="100"></a></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/core/stamp"><img src="resources/sti.png" height="150"></a></kbd>&nbsp;
 <kbd><img src="resources/hold3.png" height="150"></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/core/watermark"><img src="resources/wmi4.png" height="150"></a></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/generate/create"><img src="resources/imagebox.png" height="100"></a></kbd>&nbsp; 
 <kbd><a href="https://pdfcpu.io/generate/booklet"><img src="resources/book2A4p1.png" height="150"></a></kbd>
 <kbd><a href="https://pdfcpu.io/core/stamp"><img src="resources/stp.png" height="150"></a></kbd>&nbsp;
 <kbd><a href="https://pdfcpu.io/generate/grid"><img src="resources/gridimg.png" height="150"></a></kbd>
 <kbd><a href="https://pdfcpu.io/core/stamp"><img src="resources/stRoundBorder.png" height="150"></a></kbd>
 <kbd><a href="https://pdfcpu.io/generate/booklet"><img src="resources/book4A4p1.png" height="150"></a></kbd>


## Focus

The primary emphasis is on providing robust assistance for batch processing and scripting through a comprehensive command-line interface.
Simultaneously, pdfcpu aims to simplify the integration of PDF processing into your Go-based backend system by offering a versatile set of commands.

## Command Set

* [annotations](https://pdfcpu.io/annot/annot)
* [attachments](https://pdfcpu.io/attach/attach)
* [booklet](https://pdfcpu.io/generate/booklet)
* [bookmarks](https://pdfcpu.io/bookmarks/bookmarks)
* [boxes](https://pdfcpu.io/boxes/boxes)
* [certificates](https://pdfcpu.io/core/certs)
* [change owner password](https://pdfcpu.io/encrypt/change_opw)
* [change user password](https://pdfcpu.io/encrypt/change_upw)
* [collect](https://pdfcpu.io/core/collect)
* [config](https://pdfcpu.io/config/config)
* [create](https://pdfcpu.io/create/create)
* [crop](https://pdfcpu.io/core/crop)
* [cut](https://pdfcpu.io/generate/cut)
* [decrypt](https://pdfcpu.io/encrypt/decryptPDF)
* [encrypt](https://pdfcpu.io/encrypt/encryptPDF)
* [extract](https://pdfcpu.io/extract/extract)
* [fonts](https://pdfcpu.io/fonts/fonts)
* [form](https://pdfcpu.io/form/form)
* [grid](https://pdfcpu.io/generate/grid)
* [images](https://pdfcpu.io/images/images)
* [import](https://pdfcpu.io/generate/import)
* [info](https://pdfcpu.io/info)
* [keywords](https://pdfcpu.io/keywords/keywords)
* [merge](https://pdfcpu.io/core/merge)
* [ndown](https://pdfcpu.io/generate/ndown)
* [nup](https://pdfcpu.io/generate/nup)
* [optimize](https://pdfcpu.io/core/optimize)
* [pagelayout](https://pdfcpu.io/pagelayout/pagelayout)
* [pagemode](https://pdfcpu.io/pagemode/pagemode)
* [pages](https://pdfcpu.io/pages/pages)
* [permissions](https://pdfcpu.io/encrypt/perm_set)
* [portfolio](https://pdfcpu.io/portfolio/portfolio)
* [poster](https://pdfcpu.io/generate/poster)
* [properties](https://pdfcpu.io/properties/properties)
* [resize](https://pdfcpu.io/core/resize)
* [rotate](https://pdfcpu.io/core/rotate)
* [signatures](http://pdfcpu.io/core/sign)
* [split](https://pdfcpu.io/core/split)
* [stamp](https://pdfcpu.io/core/stamp)
* [trim](https://pdfcpu.io/core/trim)
* [validate](https://pdfcpu.io/core/validate)
* [viewerpref](https://pdfcpu.io/viewerpref/viewerpref)
* [watermark](https://pdfcpu.io/core/watermark)
* [zoom](https://pdfcpu.io/core/zoom)

## Documentation

* [pdfcpu.io](https://pdfcpu.io)
* [API tests](https://github.com/pdfcpu/pdfcpu/tree/master/pkg/api/test)
* [API samples](https://github.com/pdfcpu/pdfcpu/tree/master/pkg/samples) 
* CLI usage: `$ pdfcpu help cmd`

### GoDoc

* [pdfcpu package](https://pkg.go.dev/github.com/pdfcpu/pdfcpu)
* [pdfcpu API](https://pkg.go.dev/github.com/pdfcpu/pdfcpu/pkg/api)
* [pdfcpu CLI](https://pkg.go.dev/github.com/pdfcpu/pdfcpu/pkg/cli)

## Reminder

* Always make sure your work is based on the latest commit! 
* pdfcpu is still *Alpha* - bugfixes are committed on the fly and will be mentioned in the next release notes. 
* Follow [pdfcpu](https://twitter.com/pdfcpu) for news and release announcements.
* For quick questions or discussions get in touch on the [Gopher Slack](https://invite.slack.golangbridge.org/) in the #pdfcpu channel.


## Demo Screencast

(using older version with a smaller command set)

[![asciicast](resources/demo.png)](https://asciinema.org/a/P5jaAo9kgZXKj2iSA1OqIdLAU)

## Installation

### Download
Get the latest binary [here](https://github.com/pdfcpu/pdfcpu/releases).


### Using Go Modules

```
$ git clone https://github.com/pdfcpu/pdfcpu
$ cd pdfcpu/cmd/pdfcpu
$ go install
$ pdfcpu version
```
or directly through Go install:

```
$ go install github.com/pdfcpu/pdfcpu/cmd/pdfcpu@latest
```

### Using Homebrew (macOS)
```
$ brew install pdfcpu
$ pdfcpu version
```

### Using DNF/YUM (Fedora)
```
$ sudo dnf install golang-github-pdfcpu
$ pdfcpu version
```

### Run in a Docker container

```shell
$ docker build -t pdfcpu .
# mount current host folder into container as /app to process files in the local host folder
$ docker run -it -v "$(pwd)":/app pdfcpu validate a.pdf
```

## Contributing

### What

* Please [create](https://github.com/pdfcpu/pdfcpu/issues/new/choose) an issue if you find a bug or want to propose a change.
* Feature requests - always welcome!
* Bug fixes - always welcome!
* PRs - let's [discuss](https://github.com/pdfcpu/pdfcpu/discussions) first or [create](https://github.com/pdfcpu/pdfcpu/issues/new/choose) an issue.
* pdfcpu is stable but still *Alpha* and occasionally undergoing heavy changes.

### How

* The pdfcpu [discussion board](https://github.com/pdfcpu/pdfcpu/discussions) is open! Please engage in any form helpful for the community.
* If you want to report a bug please attach the *very verbose* (`pdfcpu cmd -vv ...`) output and ideally a test PDF that you can share.
* Always make sure your contribution is based on the latest commit.
* Please sign your commits.

### Reporting Crashes

Unfortunately crashes do happen :(
For the majority of the cases this is due to a diverse pool of PDF Writers out there and millions of PDF files using different versions waiting to be processed by pdfcpu. Sometimes these PDFs were written more than 20(!) years ago. Often there is an issue with validation - sometimes a bug in the parser. Many times even using relaxed validation with pdfcpu does not work. In these cases we need to extend relaxed validation and for this we are relying on your help. By reporting crashes you are helping to improve the stability of pdfcpu. If you happen to crash on any pdfcpu operation be it on the command line or in your Go backend these are the steps to report this:

Regardless of the pdfcpu operation, please start using the pdfcpu command line to validate your file:

``` sh
$ pdfcpu validate -v &> crash.log
```

 or to produce very verbose output

 ``` sh
 $ pdfcpu validate -vv &> crash.log
 ```

will produce what's needed to investigate a crash. Then open an issue and post `crash.log` or its contents. Ideally post a test PDF you can share to reproduce this. You can also email to hhrutter@gmail.com or if you prefer Slack you can get in touch on the Gopher slack #pdfcpu channel.

If processing your PDF with pdfcpu crashes during validation and can be opened by Adobe Reader and Mac Preview chances are we can extend relaxed validation and provide a fix. If the file in question cannot be opened by both Adobe Reader and Mac Preview we cannot help you!

## Contributors

Thanks 💚 goes to these wonderful people:


||||||||
| :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| [<img src="https://avatars1.githubusercontent.com/u/11322155?v=4" width="100px"/> Horst Rutter](https://github.com/hhrutter) | [<img src="https://avatars0.githubusercontent.com/u/5140211?v=4" width="100px"/> haldyr](https://github.com/haldyr) | [<img src="https://avatars3.githubusercontent.com/u/20608155?v=4" width="100px"/> Vyacheslav](https://github.com/SimePel) | [<img src="https://avatars1.githubusercontent.com/u/617459?v=4" width="100px"/> Erik Unger](https://github.com/ungerik) | [<img src="https://avatars1.githubusercontent.com/u/13079058?v=4" width="100px"/> Richard Wilkes](https://github.com/richardwilkes) | [<img src="https://avatars1.githubusercontent.com/u/16303386?s=400&v=4" width="100px"/> minenok-tutu](https://github.com/minenok-tutu) | [<img src="https://avatars0.githubusercontent.com/u/1965445?s=400&v=4" width="100px"/> Mateusz Burniak](https://github.com/matbur) |
| [<img src="https://avatars2.githubusercontent.com/u/1175110?s=400&v=4" width="100px"/> Dmitry Harnitski](https://github.com/dharnitski) | [<img src="https://avatars0.githubusercontent.com/u/1074083?s=400&v=4" width="100px"/> ryarnyah](https://github.com/ryarnyah) | [<img src="https://avatars0.githubusercontent.com/u/13267?s=400&v=4" width="100px"/> Sam Giffney](https://github.com/s01ipsist) | [<img src="https://avatars3.githubusercontent.com/u/32948066?s=400&v=4" width="100px"/> Carlos Eduardo Witte](https://github.com/cewitte) | [<img src="https://avatars1.githubusercontent.com/u/2374948?s=400&u=a36e5f8da8dc1c102bc4d283f25e4c61cae7f985&v=4" width="100px"/> minusworld](https://github.com/minusworld) | [<img src="https://avatars0.githubusercontent.com/u/18538487?s=400&u=b9e628dfc60f672a887be2ed04a791195829943e&v=4" width="100px"/> Witold Konior](https://github.com/jozuenoon) | [<img src="https://avatars0.githubusercontent.com/u/630151?s=400&v=4" width="100px"/> joonas.fi](https://github.com/joonas-fi) |
| [<img src="https://avatars3.githubusercontent.com/u/10349817?s=400&u=93bacb23bd2909d5b6c5b644a8d4cdd947422ee1&v=4" width="100px"/> Henrik Reinstädtler](https://github.com/henrixapp) | [<img src="https://avatars1.githubusercontent.com/u/72016286?s=400&v=4" width="100px"/> VMorozov-wh](https://github.com/VMorozov-wh) | [<img src="https://avatars0.githubusercontent.com/u/31929422?s=400&v=4" width="100px"/> Benoit KUGLER](https://github.com/benoitkugler) | [<img src="https://avatars.githubusercontent.com/u/704919?s=400&v=4" width="100px"/> Adam Greenhall](https://github.com/adamgreenhall) | [<img src="https://avatars.githubusercontent.com/u/5201812?s=400&u=8a0a9fca4560be71d4923299ddebf877854eea54&v=4" width="100px"/> moritamori](https://github.com/moritamori) | [<img src="https://avatars.githubusercontent.com/u/41904529?s=400&u=044396494285ad806e86d1936c390b3071ce57c0&v=4" width="100px"/> JanBaryla](https://github.com/JanBaryla) | [<img src="https://avatars.githubusercontent.com/u/43145244?s=400&u=89a689f1a854ce0f57ae2a0333c82bfdc5723bb9&v=4" width="100px"/> TheDiscordian](https://github.com/TheDiscordian) |
| [<img src="https://avatars.githubusercontent.com/u/15472552?v=4" width="100px"/> Rafael Garcia Argente](https://github.com/rgargente) | [<img src="https://avatars.githubusercontent.com/u/710057?v=4" width="100px"/> truyet](https://github.com/truyet) | [<img src="https://avatars.githubusercontent.com/u/5031217?v=4" width="100px"/> Christian Nicola](https://github.com/christiannicola) | [<img src="https://avatars.githubusercontent.com/u/3233970?v=4" width="100px"/> Benjamin Krill](https://github.com/kben) | [<img src="https://avatars.githubusercontent.com/u/26521615?v=4" width="100px"/> Peter Wyatt](https://github.com/petervwyatt) | [<img src="https://avatars.githubusercontent.com/u/3142701?v=4" width="100px"/> Kroum Tzanev](https://github.com/kpym) | [<img src="https://avatars.githubusercontent.com/u/992878?v=4" width="100px"/> Stefan Huber](https://github.com/signalwerk) |
| [<img src="https://avatars.githubusercontent.com/u/59667587?v=4" width="100px"/> Juan Iscar](https://github.com/juaismar) | [<img src="https://avatars.githubusercontent.com/u/20135478?v=4" width="100px"/> Eng Zer Jun](https://github.com/Juneezee) | [<img src="https://avatars.githubusercontent.com/u/28459131?v=4" width="100px"/> Dmitry Ivanov](https://github.com/hant0508)|[<img src="https://avatars.githubusercontent.com/u/16866547?v=4" width="100px"/> Rene Kaufmann](https://github.com/HeavyHorst)|[<img src="https://avatars.githubusercontent.com/u/26827864?v=4" width="100px"/> Christian Heusel](https://github.com/christian-heusel) | [<img src="https://avatars.githubusercontent.com/u/305673?v=4" width="100px"/> Chris](https://github.com/freshteapot) | [<img src="https://avatars.githubusercontent.com/u/2892794?v=4" width="100px"/> Lukasz Czaplinski](https://github.com/scoiatael) |
[<img src="https://avatars.githubusercontent.com/u/49206635?v=4" width="100px"/> Joel Silva Schutz](https://github.com/joelschutz) | [<img src="https://avatars.githubusercontent.com/u/28219076?v=4" width="100px"/> semvis123](https://github.com/semvis123) | [<img src="https://avatars.githubusercontent.com/u/8717479?v=4" width="100px"/> guangwu](https://github.com/testwill) | [<img src="https://avatars.githubusercontent.com/u/4014912?v=4" width="100px"/> Yoshiki Nakagawa](https://github.com/yyoshiki41) | [<img src="https://avatars.githubusercontent.com/u/432860?v=4" width="100px"/> Steve van Loben Sels](https://github.com/stevevls) | [<img src="https://avatars.githubusercontent.com/u/6083533?v=4" width="100px"/> Yaofu](https://github.com/mygityf) | [<img src="https://avatars.githubusercontent.com/u/15724278?v=4" width="100px"/> vsenko](https://github.com/vsenko) |
[<img src="https://avatars.githubusercontent.com/u/16507?v=4" width="100px"/> Alexis Hildebrandt](https://github.com/afh) | [<img src="https://avatars.githubusercontent.com/u/1395040?v=4" width="100px"/> Sivukhin Nikita](https://github.com/sivukhin) | [<img src="https://avatars.githubusercontent.com/u/247730?v=4" width="100px"/> Joachim Bauch](https://github.com/fancycode) | [<img src="https://avatars.githubusercontent.com/u/127291996?v=4" width="100px"/> kalimit](https://github.com/kalimit) | [<img src="https://avatars.githubusercontent.com/u/5080535?v=4" width="100px"/> Andreas Erhard](https://github.com/xelan) | [<img src="https://avatars.githubusercontent.com/u/32378535?v=4" width="100px"/> Matsumoto Toshi](https://github.com/toshi1127) | [<img src="https://avatars.githubusercontent.com/u/440634?v=4" width="100px"/> Carl Wilson](https://github.com/carlwilson) |
[<img src="https://avatars.githubusercontent.com/u/9918222?v=4" width="100px"/> LNAhri](https://github.com/LNAhri) | [<img src="https://avatars.githubusercontent.com/u/142796877?v=4" width="100px"/> vishal](https://github.com/vishal-at) | [<img src="https://avatars.githubusercontent.com/u/18169566?v=4" width="100px"/> Andreas Deininger](https://github.com/deining) | [<img src="https://avatars.githubusercontent.com/u/5825735?v=4" width="100px"/> Robert Raines](https://github.com/solintllc-robert) | [<img src="https://avatars.githubusercontent.com/u/316176?v=4" width="100px"/> Frank Anderson](https://github.com/frob) | [<img src="https://avatars.githubusercontent.com/u/20972350?v=4" width="100px"/> Sven Lilienthal](https://github.com/SveLil) | [<img src="https://avatars.githubusercontent.com/u/1900106?v=4" width="100px"/> Florian Kinder](https://github.com/fank)













## Code of Conduct

Please note that this project is released with a Contributor [Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.

## Disclaimer

Usage of pdfcpu assumes you know about and respect all copyrights of any PDF content you may be processing. This applies to the PDF files as such, their content and in particular all embedded resources like font files or images. Credit goes to [Renee French](https://instagram.com/reneefrench) for creating our beloved Gopher.

## License

Apache-2.0

pdfcpu

Go library for Gina Trapani's [_todo.txt_](http://todotxt.org/) files, supports parsing and manipulating of task lists in the [_todo.txt_ format](https://github.com/todotxt/todo.txt)

# todotxt

Yet another a Go library for Gina Trapani's todo.txt files. ✅

[![PkgGoDev](https://pkg.go.dev/badge/github.com/1set/todotxt)](https://pkg.go.dev/github.com/1set/todotxt)
[![License](https://img.shields.io/github/license/1set/todotxt)](https://github.com/1set/todotxt/blob/master/LICENSE)
[![GitHub Action Workflow](https://github.com/1set/todotxt/workflows/build/badge.svg)](https://github.com/1set/todotxt/actions?workflow=build)
[![Go Report Card](https://goreportcard.com/badge/github.com/1set/todotxt)](https://goreportcard.com/report/github.com/1set/todotxt)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/448bb14650984e21a7c41bcac221a1ba)](https://www.codacy.com/gh/1set/todotxt/dashboard)
[![Codecov](https://codecov.io/gh/1set/todotxt/branch/master/graph/badge.svg)](https://codecov.io/gh/1set/todotxt)

## Features

Based on [**go-todotxt**](https://github.com/JamesClonk/go-todotxt) from [Fabio Berchtold](https://github.com/JamesClonk) with:

- [x] Go mod support
- [x] Segments for task string
- [x] Task due today is not overdue
- [x] Negative `Due()` for overdue tasks
- [x] Support multiple options for sorting and filtering
- [x] More sorting options: by ID, text, context, project
- [x] Preset filters

## Usage

A quick start example:

```go
import (
	todo "github.com/1set/todotxt"
)

// ...

if tasklist, err := todo.LoadFromPath("todo.txt"); err != nil {
    log.Fatal(err)
} else {
    tasks := tasklist.Filter(todo.FilterNotCompleted).Filter(todo.FilterDueToday, todo.FilterHasPriority)
    _ = tasks.Sort(todo.SortPriorityAsc, todo.SortProjectAsc)
    for i, t := range tasks {
        fmt.Println(t.Todo)
        // oh really?
        tasks[i].Complete()
    }
    if err = tasks.WriteToPath("today-todo.txt"); err != nil {
        log.Fatal(err)
    }
}
```

For more examples and details, please check the [Go Doc](https://pkg.go.dev/github.com/1set/todotxt).

## Credits

- Original [**go-todotxt**](https://github.com/JamesClonk/go-todotxt) authored by [Fabio Berchtold](https://github.com/JamesClonk)
- Cool idea about [**todo.txt**](https://github.com/todotxt/todo.txt) from [Gina Trapani](http://todotxt.org/)


todotxt

A pluggable, extensible, and opinionated set of filesystem functionality for Go across a number of filesystem types such as os, S3, and GCS

<img src="logo.jpg" width="800">

[![GitHub tag](https://img.shields.io/github/tag/c2fo/vfs.svg?style=flat)](https://github.com/c2fo/vfs/releases)
[![Build Status](https://travis-ci.org/C2FO/vfs.svg?branch=master)](https://travis-ci.org/C2FO/vfs)
[![GoDoc](https://pkg.go.dev/badge/github.com/c2fo/vfs/v6?utm_source=godoc)](https://pkg.go.dev/github.com/c2fo/vfs/v7)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE.md)
[![Go Report Card](https://goreportcard.com/badge/github.com/c2fo/vfs)](https://goreportcard.com/report/github.com/c2fo/vfs)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)

## Overview

*vfs* provides a unified interface for working with different storage backends (local OS, Azure, S3, GCS, SFTP, etc.) in
a consistent way. Rather than handling each backend independently, you interact with a core set of interfaces that
abstract backend details. This allows you to:

* Seamlessly read, write, and manage files across multiple storages.
* Treat remote or cloud-based paths as if they were local (e.g. using standard `io.Reader`/`io.Writer` interfaces).
* Simplify environment-agnostic development by keeping most logic backend-agnostic.
* Plug in or mock storage functionality to facilitate testing and modular application design.

By focusing on *[FileSystem](https://pkg.go.dev/github.com/c2fo/vfs/v7#FileSystem)*,
*[Location](https://pkg.go.dev/github.com/c2fo/vfs/v7#Location)*, and
*[File](https://pkg.go.dev/github.com/c2fo/vfs/v7#File)* interfaces, you can build reusable flows for file operations without
needing to deeply understand each backend's specific APIs. Users can add or swap backends as needed, providing flexibility
for hybrid or evolving storage requirements.

## Installation
```bash
go get -u github.com/c2fo/vfs/v7
```

## Quick Start

```go
package main

import (
 "fmt"
 "strings"
 "io"
 "log"

 "github.com/c2fo/vfs/v7"
 "github.com/c2fo/vfs/v7/vfssimple"
)

func main() {
 // Create local OS file from a URI
 osFile, err := vfssimple.NewFile("file:///tmp/example.txt")
 if err != nil {
 log.Fatal(err)
 }
 defer osFile.Close()

 // Write to the file
 _, err = io.Copy(osFile, strings.NewReader("Hello from vfs!"))
 if err != nil {
 log.Fatal(err)
 }

 if err := osFile.Close(); err != nil {
 log.Fatal(err)
 }
 fmt.Println("File created and written:", osFile.URI())
}
```
This snippet shows the basic setup: an osFile is created from a URI and written to using standard libraries.

## Advanced Usage
* See the [GoDoc](https://pkg.go.dev/github.com/c2fo/vfs/v7) for deeper technical details, backend-specific options, and additional APIs.
* Specialized operations like copying between S3 locations or advanced configs (e.g. Azure credentials) are documented in each backend doc.

* [vfscp](docs/vfscp.md)
* [vfssimple](docs/vfssimple.md)
* [backend](docs/backend.md)
 * [os backend](docs/os.md)
 * [gs backend](docs/gs.md)
 * [s3 backend](docs/s3.md)
 * [in-memory backend](docs/mem.md)
 * [sftp backend](docs/sftp.md)
 * [ftp backend](docs/ftp.md)
 * [azure backend](docs/azure.md)
 * [conformance tests](docs/conformance_tests.md)
* [utils](docs/utils.md)

### Community Contributed Backends

These backends are maintained in `contrib/backend/` as separate Go modules:

* [Dropbox backend](contrib/backend/dropbox/README.md) - Dropbox cloud storage

See [contrib/backend/README.md](contrib/backend/README.md) for guidelines on contributing new backends.

## Additional Tools

* [lockfile](contrib/lockfile/README.md): Provides distributed locking mechanisms using vfs backends.
* [vfsevents](contrib/vfsevents/README.md): Production-ready file event monitoring with support for real-time notifications, retry logic, and multiple watcher types (VFS polling, S3 events, GCS events).

## FAQ

**Q**: Why am I seeing an empty file when using `io.Copy` on some backends if my source is empty?
**A**: An empty `Reader` often means the backend doesn't write a file until data is actually written. Use `utils.TouchCopy` if you need to ensure a zero-byte file is created.

**Q**: Will vfs v6 still be supported?
**A**: Yes and no. We will continue to provide security patches and bug fixes for v6, but new features and enhancements
will be added to v7.

**Q**: How long will v6 be supported?
**A**: We will support v6 until the next major release, v8, is released. Then v7 will be supported until v9 is released.

## Upgrading

### Upgrading from v6 to v7

Please review these changes and update your code accordingly to ensure compatibility with v7.

#### S3 Backend

The project now uses the `aws-sdk-go-v2` library instead of the deprecated, EOL `aws-sdk-go`. This update necessitated
these changes to the S3 backend:

- The S3 backend's `s3fs.Client()` function now returns an `s3.Client` which is a subset of AWS's sdk v2 functionality.
 This change may require updates to your code if you were relying on client functionality not directly required by the
 s3 vfs backend.
- The `Option.Retry` field is now an `aws.Retryer` instead of a `request.Retry`. Ensure that your Option logic is
 compatible with the new type.

#### Azure Backend

- Scheme for Azure has been updated from `https` to `az`. Update your code to use the new scheme.
- Authority for Azure has been updated from `blob.core.windows.net` to `my-container-name`, such that the full URI
 is `az://my-blob-name/path/to/file.txt` rather than
 `https://my-storage-account-name.core.windows.net/my-container-name/path/to/file.txt`.

#### GS Backend
- The `Options.Retry` field, with the now deprecated `vfs.Retry` type, has been moved to `gs.FileSystem` as the new `gs.Retyer`
 type. It is now set via `gs.NewFileSystem` function using functional option `gs.WithRetryer`. All previous `vfs.Retry`
 usage has been replaced with `gs.WithRetryer`. Update your code to use the new `gs.WithRetryer` functional option.

#### All Backends
Some methods in the *Location* and *FileSystem* interfaces have been deprecated because they use terminology that doesn't apply to all backends.
They will be removed in a future release. Update your code to use the new methods. See [#235](https://github.com/C2FO/vfs/issues/235).
- `location.Volume()` method which returns the authority as a string has been deprecated in favor of the `location.Authority()` method which
 returns an `authority.Authority` struct. Update your code to use the `Authority().String()` method instead of `Volume()`.
- `location.ChangeDir()` method ash been deprecated in favor of the existing `location.NewLocation()` method. Update your code to use the
 `NewLocation()` method instead of `ChangeDir()`.
- `vfs.Options` struct has been deprecated in favor of using backend-specific structs.
- `filesystem.Retry()` method has been deprecated in favor of using backend-specific functional options, such as `gs.WithRetryer`.

Additionally, we have added functional option interface, `FileSystemOption`, to allow for more flexible configuration
of backends. This interface allows for more complex configuration options to be passed to the via the `NewFileSystem` function.
This will replace backend-specific chainable functions that require casting the filesystem to the backend type first. See [#238](https://github.com/C2FO/vfs/issues/238).

### Upgrading from v5 to v6

With `v6`, sftp.Options struct changed to accept an array of Key Exchange algorithms rather than a string. To update,
change the syntax of the auth commands.

```
 "keyExchanges":"diffie-hellman-group-a256"
```

becomes

```
 "keyExchanges":["diffie-hellman-group-a256"]
```

## Contributing

1. Fork it (`git clone https://github.com/c2fo/vfs.git`)
2. Create your feature branch (`git checkout -b feature/fooBar`)
3. Commit your changes (`git commit -am "Add some fooBar"`)
4. Push to the branch (`git push origin feature/fooBar`)
5. Create a new Pull Request

## License
Distributed under the MIT License. See the [LICENSE.md](LICENSE.md) file for details.

Compute message digest, like MD5, SHA256, SHA1, CRC or BLAKE2s, for large files

checksum
==
[![PkgGoDev](https://pkg.go.dev/badge/github.com/codingsince1985/checksum)](https://pkg.go.dev/github.com/codingsince1985/checksum)
[![Go Report Card](https://goreportcard.com/badge/codingsince1985/checksum)](https://goreportcard.com/report/codingsince1985/checksum)
[![codecov](https://codecov.io/gh/codingsince1985/checksum/branch/master/graph/badge.svg?token=BO1wAQwc85)](https://codecov.io/gh/codingsince1985/checksum)

Compute message digest, like MD5, SHA256, SHA1, CRC or BLAKE2s-256 in Golang for potentially large files.

Usage
--
```go
package main

import (
	"fmt"
	"github.com/codingsince1985/checksum"
)

func main() {
	file := "/home/jerry/Downloads/ubuntu-20.04.2.0-desktop-amd64.iso"

	md5, _ := checksum.MD5sum(file)
	fmt.Println(md5)

	sha256, _ := checksum.SHA256sum(file)
	fmt.Println(sha256)

	sha1, _ := checksum.SHA1sum(file)
	fmt.Println(sha1)

	crc32, _ := checksum.CRC32(file)
	fmt.Println(crc32)

	blake2s256, _ := checksum.Blake2s256(file)
	fmt.Println(blake2s256)
}
```
License
==
checksum is distributed under the terms of the MIT license. See LICENSE for details.


checksum

# go-csv-tag

Read csv file from Go using tags

[![godoc for artonge/go-csv-tag](https://godoc.org/github.com/artonge/go-csv-tag?status.svg)](http://godoc.org/github.com/artonge/go-csv-tag)

![Go](https://github.com/artonge/go-csv-tag/workflows/Go/badge.svg)
[![goreportcard for artonge/go-csv-tag](https://goreportcard.com/badge/github.com/artonge/go-csv-tag)](https://goreportcard.com/report/artonge/go-csv-tag)

[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)

**The project is in maintenance mode.**

It is kept compatible with changes in the Go ecosystem but no new features will be developed. PR could be accepted.

# Install

`go get github.com/artonge/go-csv-tag/v2`

# Example

## Load

The csv file:

```csv
name, ID, number
name1, 1, 1.2
name2, 2, 2.3
name3, 3, 3.4
```

Your Go code:

```go
type Demo struct {                                // A structure with tags
	Name string  `csv:"name"`
	ID   int     `csv:"ID"`
	Num  float64 `csv:"number"`
}

tab := []Demo{}                                   // Create the slice where to put the content
err  := csvtag.LoadFromPath(
	"file.csv",                                   // Path of the csv file
	&tab,                                         // A pointer to the create slice
	csvtag.CsvOptions{                            // Load your csv with optional options
		Separator: ';',                           // changes the values separator, default to ','
		Header: []string{"name", "ID", "number"}, // specify custom headers
		TagKey: "csv",                            // specify a custom tag key, default to 'csv'
})
```

You can also load the data from an io.Reader with:

```go
csvtag.LoadFromReader(youReader, &tab)
```

Or from a string with:

```go
csvtag.LoadFromString(yourString, &tab)
```

## Dump

Your Go code:

```go
type Demo struct {                         // A structure with tags
	Name string  `csv:"name"`
	ID   int     `csv:"ID"`
	Num  float64 `csv:"number"`
}

tab := []Demo{                             // Create the slice where to put the content
	Demo{
		Name: "some name",
		ID: 1,
		Num: 42.5,
	},
}

err := csvtag.DumpToFile(tab, "csv_file_name.csv")
```

You can also dump the data into an io.Writer with:

```go
err := csvtag.DumpToWriter(tab, yourIOWriter)
```

Or dump to a string with:

```go
str, err := csvtag.DumpToString(tab)
```

The csv file written:

```csv
name,ID,number
some name,1,42.5
```


go-csv-tag

[![GoDoc](https://godoc.org/github.com/hugocarreira/go-decent-copy?status.svg)](https://godoc.org/github.com/hugocarreira/go-decent-copy) [![Build Status](https://travis-ci.org/hugocarreira/go-decent-copy.svg?branch=master)](https://travis-ci.org/hugocarreira/go-decent-copy) [![Go Report Card](https://goreportcard.com/badge/github.com/hugocarreira/go-decent-copy)](https://goreportcard.com/report/github.com/hugocarreira/go-decent-copy) [![codecov](https://codecov.io/gh/hugocarreira/go-decent-copy/branch/master/graph/badge.svg)](https://codecov.io/gh/hugocarreira/go-decent-copy)

# Go-Decent-Copy

go-decent-copy provides a copy file for humans

## Usage

```go
package main

import "github.com/hugocarreira/go-decent-copy"

func main() {
    execPath, _ := os.Getwd()

    fileOrigin := fmt.Sprintf(execPath + "/testCopy.txt")
    fileDestiny := fmt.Sprintf(execPath + "/testCopied.txt")
    err := Copy(fileOrigin, fileDestiny)
    if err != nil {
        t.Fatalf("Error in copy file : %#v ", err.Error())
    }
}
```

## License

This project is licensed under the [MIT License](LICENSE).


go-decent-copy

# go-gtfs
Load GTFS files in Go.

[![godoc for artonge/go-gtfs](https://godoc.org/github.com/artonge/go-gtfs?status.svg)](http://godoc.org/github.com/artonge/go-gtfs)

![Go](https://github.com/artonge/go-gtfs/workflows/Go/badge.svg)
[![goreportcard for artonge/go-gtfs](https://goreportcard.com/badge/github.com/artonge/go-gtfs)](https://goreportcard.com/report/artonge/go-gtfs)

[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)

**The project is in maintenance mode.**

It is kept compatible with changes in the Go ecosystem but no new features will be developed. PR could be accepted.
# Install
`go get github.com/artonge/go-gtfs`

# Examples
## Load one directory containing GTFS files:
```bash
path/to/gtfs_files
├── agency.txt
├── attributions.txt
├── calendar_dates.txt
├── calendar.txt
├── fare_attributes.txt
├── fare_rules.txt
├── feed_info.txt
├── frequencies.txt
├── levels.txt
├── pathways.txt
├── routes.txt
├── shapes.txt
├── stops.txt
├── stop_times.txt
├── transfers.txt
└── trips.txt
```
```go
g, err := gtfs.Load("path/to/gtfs_files", nil)
```

## Load a directory containing sub directories containing GTFS files:
```bash
path/to/gtfs_directories
├── gtfs1
│   ├── agency.txt
│   ├── attributions.txt
│   ├── calendar_dates.txt
│   ├── calendar.txt
│   ├── fare_attributes.txt
│   ├── fare_rules.txt
│   ├── feed_info.txt
│   ├── frequencies.txt
│   ├── levels.txt
│   ├── pathways.txt
│   ├── routes.txt
│   ├── shapes.txt
│   ├── stops.txt
│   ├── stop_times.txt
│   ├── transfers.txt
│   └── trips.txt
└── gtfs2
    ├── agency.txt
    ├── attributions.txt
    ├── calendar_dates.txt
    ├── calendar.txt
    ├── fare_attributes.txt
    ├── fare_rules.txt
    ├── feed_info.txt
    ├── frequencies.txt
    ├── levels.txt
    ├── pathways.txt
    ├── routes.txt
    ├── shapes.txt
    ├── stops.txt
    ├── stop_times.txt
    ├── transfers.txt
    └── trips.txt

```
```go
gs, err := gtfs.LoadSplitted("path/to/gtfs_directories", nil)
```

You can then access the data through the GTFS structure.
That structure contains arrays of approriate structures for each files.
```go
type GTFS struct {
	Path           string // The path to the containing directory
	Agency         Agency
	Agencies       []Agency
	Attributions   []Attribution
	Calendars      []Calendar
	CalendarDates  []CalendarDate
	FareAttributes []FareAttribute
	FareRules      []FareRule
	FeedInfos      []FeedInfo
	Frequencies    []Frequency
	Levels         []Level
	Routes         []Route
	Pathways       []Pathway
	Shapes         []Shape
	Stops          []Stop
	StopsTimes     []StopTime
	Trips          []Trip
	Transfers      []Transfer
}

type Route struct {
	ID        string `csv:"route_id"`
	AgencyID  string `csv:"agency_id"`
	ShortName string `csv:"route_short_name"`
	LongName  string `csv:"route_long_name"`
	Type      int    `csv:"route_type"`
	Desc      string `csv:"route_desc"`
	URL       string `csv:"route_url"`
	Color     string `csv:"route_color"`
	TextColor string `csv:"route_text_color"`
}

...
```

# Contributions
Pull requests are welcome ! :)


go-gtfs

Simple and reliable package for file operations like copy/move/diff/list on files, directories and symbolic links

# gut 🍱

Yet another collection of **G**o **u**tilities & **t**ools.  
A simple one. Just **go** with your **gut** feeling.

---

## Shortcuts

| Symbol            | 🍱 弁当                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| :---------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Document**      | [![PkgGoDev](https://pkg.go.dev/badge/github.com/1set/gut)](https://pkg.go.dev/github.com/1set/gut)                                                                                                                                                                                                                                                                                                                                          |
| **License**       | [![License](https://img.shields.io/github/license/1set/gut)](https://github.com/1set/gut/blob/master/LICENSE)                                                                                                                                                                                                                                                                                                                                |
| **Build Status**  | [![GitHub Action - Ubuntu Build](https://github.com/1set/gut/workflows/Ubuntu/badge.svg)](https://github.com/1set/gut/actions?workflow=Ubuntu) [![GitHub Action - macOS Build](https://github.com/1set/gut/workflows/macOS/badge.svg)](https://github.com/1set/gut/actions?workflow=macOS) [![GitHub Action - Windows Build](https://github.com/1set/gut/workflows/Windows/badge.svg)](https://github.com/1set/gut/actions?workflow=Windows) |
| **Code Quality**  | [![Go Report Card](https://goreportcard.com/badge/github.com/1set/gut)](https://goreportcard.com/report/github.com/1set/gut) [![Codacy](https://api.codacy.com/project/badge/Grade/f70fcc271c3e4785a3dfb87739a44cd0)](https://www.codacy.com/manual/an9an63/gut) [![Code Climate](https://api.codeclimate.com/v1/badges/6217d80425ed0b096664/maintainability)](https://codeclimate.com/github/1set/gut/maintainability)                      |
| **Test Coverage** | [![Codecov](https://codecov.io/gh/1set/gut/branch/master/graph/badge.svg)](https://codecov.io/gh/1set/gut/branch/master)                                                                                                                                                                                                                                                                                                                     |

## Packages

[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)

- [`yos`](https://godoc.org/github.com/1set/gut/yos): yet another wrapper of platform-independent interface to operating system functionality (esp. file operations)
- [`yrand`](https://godoc.org/github.com/1set/gut/yrand): yet another wrapper of **crypto/rand** with func like **math/rand**
- [`yhash`](https://godoc.org/github.com/1set/gut/yhash): yet another collection of methods to calculate checksum of file/string/bytes
- [`ystring`](https://godoc.org/github.com/1set/gut/ystring): yet another collection of methods to manipulate UTF-8 encoded strings


gut/yos

A tiny cross-platform Go library to hide/unhide files and directories

# higgs

A tiny cross-platform Go library to hide or unhide files and directories.

**Supported OSs:** All unix based OSs (Tested on Ubuntu, MacOS), Windows

[![unix](https://github.com/dastoori/higgs/actions/workflows/unix.yml/badge.svg)](https://github.com/dastoori/higgs/actions/workflows/unix.yml)
[![windows](https://github.com/dastoori/higgs/actions/workflows/windows.yml/badge.svg)](https://github.com/dastoori/higgs/actions/workflows/windows.yml)
[![codecov](https://codecov.io/gh/dastoori/higgs/branch/master/graph/badge.svg?token=T1AJXSWI3F)](https://codecov.io/gh/dastoori/higgs)
[![GitHub release](https://img.shields.io/github/v/release/dastoori/higgs?sort=semver)](https://github.com/dastoori/higgs/releases) 
[![Go Reference](https://pkg.go.dev/badge/github.com/dastoori/higgs.svg)](https://pkg.go.dev/github.com/dastoori/higgs)
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/dastoori/higgs)
[![Go Report Card](https://goreportcard.com/badge/github.com/dastoori/higgs)](https://goreportcard.com/report/github.com/dastoori/higgs)
[![GitHub](https://img.shields.io/github/license/dastoori/higgs)](https://github.com/dastoori/higgs/blob/master/LICENSE)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go) 

## Installation

```sh
$ go get github.com/dastoori/higgs
```

## Usage

```go
package main

import (
	"fmt"
	"github.com/dastoori/higgs"
)

func main() {
	path := "foo.txt"

	// Hiding a file
	newPath, err := higgs.Hide(path)
	// NOTE: On Unix after hiding the file the file name
	// will be changed to `.foo.txt` and `newPath` contains
	// the new file name

	if err != nil {
		fmt.Println(err)
	}

	// Unhiding a file
	newPath, err = higgs.Unhide(newPath)
	
	if err != nil {
		fmt.Println(err)
	}

	// Setting unix overwrite option (disable by default)
	fh := NewFileHide(".bar.txt", UnixOverwriteOption(true))

	// NOTE: On Unix if a `bar.txt` file exists, it will be
	// overwritten after unhiding `.bar.txt`
	err := fh.Unhide()
	
	if err != nil {
		fmt.Println(err)
	}

	// NOTE: `fh.Path` contains the new file name
}
```

## License

MIT

higgs

File system event notification library with simple API, similar to os/signal

notify [![GoDoc](https://godoc.org/github.com/rjeczalik/notify?status.svg)](https://godoc.org/github.com/rjeczalik/notify) [![Build Status](https://img.shields.io/travis/rjeczalik/notify/master.svg)](https://travis-ci.org/rjeczalik/notify "inotify + FSEvents + kqueue") [![Build status](https://img.shields.io/appveyor/ci/rjeczalik/notify-246.svg)](https://ci.appveyor.com/project/rjeczalik/notify-246 "ReadDirectoryChangesW") [![Coverage Status](https://img.shields.io/coveralls/rjeczalik/notify/master.svg)](https://coveralls.io/r/rjeczalik/notify?branch=master)
======

Filesystem event notification library on steroids.

*Documentation*

[godoc.org/github.com/rjeczalik/notify](https://godoc.org/github.com/rjeczalik/notify)

*Installation*

```
~ $ go get -u github.com/rjeczalik/notify
```

*Projects using notify*

- [github.com/rjeczalik/cmd/notify](https://godoc.org/github.com/rjeczalik/cmd/notify)
- [github.com/cortesi/devd](https://github.com/cortesi/devd)
- [github.com/cortesi/modd](https://github.com/cortesi/modd)
- [github.com/syncthing/syncthing](https://github.com/syncthing/syncthing)
- [github.com/OrlovEvgeny/TinyJPG](https://github.com/OrlovEvgeny/TinyJPG)
- [github.com/mitranim/gow](https://github.com/mitranim/gow)


notify

Load Open Packaging Conventions (OPC) files for Go

# opc

[![PkgGoDev](https://pkg.go.dev/badge/github.com/qmuntal/opc)](https://pkg.go.dev/github.com/qmuntal/opc)
[![Build Status](https://github.com/qmuntal/opc/actions/workflows/test.yml/badge.svg)](https://github.com/qmuntal/opc/actions/workflows/test.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/qmuntal/opc)](https://goreportcard.com/report/github.com/qmuntal/opc)
[![codecov](https://coveralls.io/repos/github/qmuntal/opc/badge.svg)](https://coveralls.io/github/qmuntal/opc?branch=master)
[![codeclimate](https://codeclimate.com/github/qmuntal/opc/badges/gpa.svg)](https://codeclimate.com/github/qmuntal/opc)
[![License](https://img.shields.io/badge/License-BSD%202--Clause-orange.svg)](https://opensource.org/licenses/BSD-2-Clause)
[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)  

Package opc implements the ISO/IEC 29500-2, also known as the [Open Packaging Convention](https://en.wikipedia.org/wiki/Open_Packaging_Conventions).

The Open Packaging specification describes an abstract model and physical format conventions for the use of XML, Unicode, ZIP, and other openly available technologies and specifications to organize the content and resources of a document within a package.

The OPC is the foundation technology for many new file formats: .docx, .pptx, .xlsx, .3mf, .dwfx, ...

## Features
- [x] Package reader and writer
- [x] Package core properties and relationships
- [x] Part relationships
- [x] ZIP mapping
- [x] Package, relationships and parts validation against specs
- [ ] Part interleaved pieces
- [ ] Digital signatures

## Examples
### Write
```go
// Create a file to write our archive to.
f, _ := os.Create("example.xlsx")

// Create a new OPC archive.
w := opc.NewWriter(f)

// Create a new OPC part.
name := opc.NormalizePartName("docs\\readme.txt")
part, _ := w.Create(name, "text/plain")

// Write content to the part.
part.Write([]byte("This archive contains some text files."))

// Make sure to check the error on Close.
w.Close()
```

### Read
```go
r, _ := opc.OpenReader("testdata/test.xlsx")
defer r.Close()

// Iterate through the files in the archive,
// printing some of their contents.
for _, f := range r.Files {
  fmt.Printf("Contents of %s with type %s :\n", f.Name, f.ContentType)
  rc, _ := f.Open()
  io.CopyN(os.Stdout, rc, 68)
  rc.Close()
  fmt.Println()
}
```


Treat paths as their own type instead of using strings

# pathtype

Treat paths as their own type instead of using strings. This small package wraps functions from the standard library to create a new Path type.

## Example

### Code

```go
package main

import (
	"fmt"
	"log"

	pt "github.com/jonchun/pathtype"
)

type path = pt.Path

func main() {
	myFile := path("myfile.txt")
	exampleFile := path("example/example.txt")
	fmt.Println(exampleFile.Dir())
	fmt.Println(exampleFile.Dir().Join(myFile))

	res, err := exampleFile.Dir().Join(myFile).Dir().Abs()
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(res)

	fmt.Println("=========================")
	listBase(res)
	fmt.Println("=========================")
	listExt(res)
}

// list all Base for files in p
func listBase(p path) {
	if glob, err := p.Glob("*"); err != nil {
		log.Fatal(err)
	} else {
		for _, match := range glob {
			fmt.Println(match.Base())
		}
	}
}

// list all extensions for files in p
func listExt(p path) {
	if glob, err := p.Glob("*"); err != nil {
		log.Fatal(err)
	} else {
		for _, match := range glob {
			fmt.Println(match.Ext())
		}
	}
}
```

### Output

```
example
example/myfile.txt
/home/jonchun/example_module/example
=========================
1.log
2.log
example.txt
=========================
.log
.log
.txt
```

See [GoDoc](https://godoc.org/github.com/jonchun/pathtype) for documentation, but it should be pretty self-explanatory.

## TODO

- Update examples for all the methods and have 1-to-1 coverage of the official examples/docs.


pathtype

Package to allow one to concurrently go through a filesystem with ease

# skywalker [![GoDoc](https://godoc.org/github.com/dixonwille/skywalker?status.svg)](https://godoc.org/github.com/dixonwille/skywalker) [![Build Status](https://travis-ci.org/dixonwille/skywalker.svg?branch=master)](https://travis-ci.org/dixonwille/skywalker) [![Build status](https://ci.appveyor.com/api/projects/status/d1h7lpf0pv546amh?svg=true)](https://ci.appveyor.com/project/dixonwille/skywalker) [![codecov](https://codecov.io/gh/dixonwille/skywalker/branch/master/graph/badge.svg)](https://codecov.io/gh/dixonwille/skywalker) [![Go Report Card](https://goreportcard.com/badge/github.com/dixonwille/skywalker)](https://goreportcard.com/report/github.com/dixonwille/skywalker)

Skywalker is a package to allow one to concurrently go through a filesystem with ease.

## Features

- Concurrency
- BlackList filtering
- WhiteList filtering
- Filter by Directory
- Filter by Extension
- Glob Filtering (provided by [gobwas/glob](https://github.com/gobwas/glob))

> For matching to work properly across platforms. Please use `/`. In [gobwas/glob](https://github.com/gobwas/glob) the `\` is an escape character (so you can escape `*`, `?`, etc...) making it difficult to know if you want to escape a character or go into directory.

## Example

```go
package main

import (
    "fmt"
    "sort"
    "strings"
    "sync"

    "github.com/dixonwille/skywalker"
)

type ExampleWorker struct {
    *sync.Mutex
    found []string
}

func (ew *ExampleWorker) Work(path string) {
    //This is where the necessary work should be done.
    //This will get concurrently so make sure it is thread safe if you need info across threads.
    ew.Lock()
    defer ew.Unlock()
    ew.found = append(ew.found, path)
}

func main() {
    //Following two functions are only to create and destroy data for the example
    defer teardownData()
    standupData()

    ew := new(ExampleWorker)
    ew.Mutex = new(sync.Mutex)

    //root is the root directory of the data that was stood up above
    sw := skywalker.New(root, ew)
    sw.DirListType = skywalker.LTBlacklist
    sw.DirList = []string{"sub"}
    sw.ExtListType = skywalker.LTWhitelist
    sw.ExtList = []string{".pdf"}
    err := sw.Walk()
    if err != nil {
        fmt.Println(err)
        return
    }
    sort.Sort(sort.StringSlice(ew.found))
    for _, f := range ew.found {
        fmt.Println(strings.Replace(f, sw.Root, "", 1))
    }
}
```


skywalker

A library to process http file uploads easily

<div align="center">
	<pre> 
 ▄▄▄▄ ▄▄▄ ██▀███ ▄▄▄ ██ ▄█▀▄▄▄ 
▓█████▄ ▒████▄ ▓██ ▒ ██▒▒████▄ ██▄█▒▒████▄ 
▒██▒ ▄██▒██ ▀█▄ ▓██ ░▄█ ▒▒██ ▀█▄ ▓███▄░▒██ ▀█▄ 
▒██░█▀ ░██▄▄▄▄██ ▒██▀▀█▄ ░██▄▄▄▄██ ▓██ █▄░██▄▄▄▄██ 
░▓█ ▀█▓ ▓█ ▓██▒░██▓ ▒██▒ ▓█ ▓██▒▒██▒ █▄▓█ ▓██▒
░▒▓███▀▒ ▒▒ ▓▒█░░ ▒▓ ░▒▓░ ▒▒ ▓▒█░▒ ▒▒ ▓▒▒▒ ▓▒█░
▒░▒ ░ ▒ ▒▒ ░ ░▒ ░ ▒░ ▒ ▒▒ ░░ ░▒ ▒░ ▒ ▒▒ ░
 ░ ░ ░ ▒ ░░ ░ ░ ▒ ░ ░░ ░ ░ ▒ 
 ░ ░ ░ ░ ░ ░░ ░ ░ ░
 ░ 
	</pre>
 
[![Go Report Card](https://goreportcard.com/badge/github.com/xis/baraka)](https://goreportcard.com/report/github.com/xis/baraka)
[![codecov](https://codecov.io/gh/xis/baraka/branch/master/graph/badge.svg)](https://codecov.io/gh/xis/baraka)
[![Build Status](https://travis-ci.org/xis/baraka.svg?branch=master)](https://travis-ci.org/xis/baraka) 
[![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/github.com/xis/baraka/v2)
 
a tool for handling file uploads for http servers

makes it easier to make operations with files from the http request.
</div>

## Contents
 - [Install](#Install)
 - [Simple Usage](#Simple-Usage)
 - [Filtering Parts](#Filtering-Parts)
 - [More](#More)
 - [Contribute](#Contribute)
 - [License](#License)

## Install
```bash
go get github.com/xis/baraka/v2
```

## Simple Usage
```go
func main() {
	// create a parser
	parser := baraka.NewParser(baraka.ParserOptions{
		MaxFileSize: 5 << 20,
		MaxFileCount: 5,
		MaxParseCount: 5,
	})

	store := baraka.NewFilesystemStorage("./files")

	router := gin.Default()
	router.POST("/upload", func(c *gin.Context) {
		// parse
		request, err := parser.Parse(c.Request)
		if err != nil {
			fmt.Println(err)
		}

		// get the form
		images, err := request.GetForm("images")
		if err != nil {
			fmt.Println(err)
		}

		// save
		for key, image := range images {
			err = store.Save("images", "image_"+strconv.Itoa(key), image)
			if err != nil {
				fmt.Println(err)
			}
		}
	})
	router.Run()
}
```
You can use baraka with the other http server libraries, just pass the http.Request to the parser.Parse function.

## Filtering Parts
You can filter parts by their properties, like part's content type. Parser can inspect the part's bytes and detect the type of the part with the Inspector.

```go
// create a parser
parser := baraka.NewParser(baraka.ParserOptions{
	MaxFileSize: 5 << 20,
	MaxFileCount: 5,
	MaxParseCount: 5,
})

// give parser an inspector
parser.SetInspector(baraka.NewDefaultInspector(512))
// give parser a filter
parser.SetFilter(baraka.NewExtensionFilter(".jpg"))
```

Now parser will inspect the each part and it will just return the jpeg ones from the Parse function. You can make your own Inspector and Filter.

## Contribute
Pull requests are welcome. please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

## License
[MIT](https://choosealicense.com/licenses/mit/)

baraka

Fast parallel directory traversal library (used by [fzf](https://github.com/junegunn/fzf))

[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/charlievieth/fastwalk)
[![Test fastwalk on macOS](https://github.com/charlievieth/fastwalk/actions/workflows/macos.yml/badge.svg)](https://github.com/charlievieth/fastwalk/actions/workflows/macos.yml)
[![Test fastwalk on Linux](https://github.com/charlievieth/fastwalk/actions/workflows/linux.yml/badge.svg)](https://github.com/charlievieth/fastwalk/actions/workflows/linux.yml)
[![Test fastwalk on Windows](https://github.com/charlievieth/fastwalk/actions/workflows/windows.yml/badge.svg)](https://github.com/charlievieth/fastwalk/actions/workflows/windows.yml)

# fastwalk

Fast parallel directory traversal for Golang.

Package fastwalk provides a fast parallel version of [`filepath.WalkDir`](https://pkg.go.dev/io/fs#WalkDirFunc)
that is \~2.5x faster on macOS, \~4x faster on Linux, \~6x faster on Windows,
allocates 50% less memory, and requires 25% fewer memory allocations.
Additionally, it is \~4-5x faster than [godirwalk](https://github.com/karrick/godirwalk)
across OSes.

Inspired by and based off of [golang.org/x/tools/internal/fastwalk](https://pkg.go.dev/golang.org/x/tools@v0.1.9/internal/fastwalk).

## Features

* Fast: multiple goroutines stat the filesystem and call the
 [`filepath.WalkDirFunc`](https://pkg.go.dev/io/fs#WalkDirFunc) callback concurrently
* Safe symbolic link traversal ([`Config.Follow`](https://pkg.go.dev/github.com/charlievieth/fastwalk#Config))
* Same behavior and callback signature as [`filepath.WalkDir`](https://pkg.go.dev/path/filepath#WalkDir)
* Wrapper functions are provided to ignore duplicate files and directories:
	[`IgnoreDuplicateFiles()`](https://pkg.go.dev/github.com/charlievieth/fastwalk#IgnoreDuplicateFiles)
	and
	[`IgnoreDuplicateDirs()`](https://pkg.go.dev/github.com/charlievieth/fastwalk#IgnoreDuplicateDirs)
* Extensively tested on macOS, Linux, and Windows

## Usage

Usage is the same as [`filepath.WalkDir`](https://pkg.go.dev/io/fs#WalkDirFunc),
but the [`walkFn`](https://pkg.go.dev/path/filepath@go1.17.7#WalkFunc)
argument to [`fastwalk.Walk`](https://pkg.go.dev/github.com/charlievieth/fastwalk#Walk)
must be safe for concurrent use.

Examples can be found in the [examples](./examples) directory.



The below example is a very simple version of the POSIX
[find](https://pubs.opengroup.org/onlinepubs/007904975/utilities/find.html) utility:
```go
// fwfind is a an example program that is similar to POSIX find,
// but faster and worse (it's an example).
package main

import (
	"flag"
	"fmt"
	"io/fs"
	"os"
	"path/filepath"

	"github.com/charlievieth/fastwalk"
)

const usageMsg = `Usage: %[1]s [-L] [-name] [PATH...]:

%[1]s is a poor replacement for the POSIX find utility

`

func main() {
	flag.Usage = func() {
		fmt.Fprintf(os.Stdout, usageMsg, filepath.Base(os.Args[0]))
		flag.PrintDefaults()
	}
	pattern := flag.String("name", "", "Pattern to match file names against.")
	followLinks := flag.Bool("L", false, "Follow symbolic links")
	flag.Parse()

	// If no paths are provided default to the current directory: "."
	args := flag.Args()
	if len(args) == 0 {
		args = append(args, ".")
	}

	// Follow links if the "-L" flag is provided
	conf := fastwalk.Config{
		Follow: *followLinks,
	}

	walkFn := func(path string, d fs.DirEntry, err error) error {
		if err != nil {
			fmt.Fprintf(os.Stderr, "%s: %v\n", path, err)
			return nil // returning the error stops iteration
		}
		if *pattern != "" {
			if ok, err := filepath.Match(*pattern, d.Name()); !ok {
				// invalid pattern (err != nil) or name does not match
				return err
			}
		}
		_, err = fmt.Println(path)
		return err
	}
	for _, root := range args {
		if err := fastwalk.Walk(&conf, root, walkFn); err != nil {
			fmt.Fprintf(os.Stderr, "%s: %v\n", root, err)
			os.Exit(1)
		}
	}
}
```

## Benchmarks

Benchmarks were created using `go1.17.6` and can be generated with the `bench_comp` make target:
```sh
$ make bench_comp
```

### Darwin

**Hardware:**
```
goos: darwin
goarch: arm64
cpu: Apple M1 Max
```

#### [`filepath.WalkDir`](https://pkg.go.dev/path/filepath@go1.17.7#WalkDir) vs. [`fastwalk.Walk()`](https://pkg.go.dev/github.com/charlievieth/fastwalk#Walk):
```
 filepath fastwalk delta
time/op 27.9ms ± 1% 13.0ms ± 1% -53.33%
alloc/op 4.33MB ± 0% 2.14MB ± 0% -50.55%
allocs/op 50.9k ± 0% 37.7k ± 0% -26.01%
```

#### [`godirwalk.Walk()`](https://pkg.go.dev/github.com/karrick/godirwalk@v1.16.1#Walk) vs. [`fastwalk.Walk()`](https://pkg.go.dev/github.com/charlievieth/fastwalk#Walk):
```
 godirwalk fastwalk delta
time/op 58.5ms ± 3% 18.0ms ± 2% -69.30%
alloc/op 25.3MB ± 0% 2.1MB ± 0% -91.55%
allocs/op 57.6k ± 0% 37.7k ± 0% -34.59%
```

### Linux

**Hardware:**
```
goos: linux
goarch: amd64
cpu: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
drive: Samsung SSD 970 PRO 1TB
```

#### [`filepath.WalkDir`](https://pkg.go.dev/path/filepath@go1.17.7#WalkDir) vs. [`fastwalk.Walk()`](https://pkg.go.dev/github.com/charlievieth/fastwalk#Walk):

```
 filepath fastwalk delta
time/op 10.1ms ± 2% 2.8ms ± 2% -72.83%
alloc/op 2.44MB ± 0% 1.70MB ± 0% -30.46%
allocs/op 47.2k ± 0% 36.9k ± 0% -21.80%
```

#### [`godirwalk.Walk()`](https://pkg.go.dev/github.com/karrick/godirwalk@v1.16.1#Walk) vs. [`fastwalk.Walk()`](https://pkg.go.dev/github.com/charlievieth/fastwalk#Walk):

```
 filepath fastwalk delta
time/op 13.7ms ±16% 2.8ms ± 2% -79.88%
alloc/op 7.48MB ± 0% 1.70MB ± 0% -77.34%
allocs/op 53.8k ± 0% 36.9k ± 0% -31.38%
```

### Windows

**Hardware:**
```
goos: windows
goarch: amd64
pkg: github.com/charlievieth/fastwalk
cpu: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
```

#### [`filepath.WalkDir`](https://pkg.go.dev/path/filepath@go1.17.7#WalkDir) vs. [`fastwalk.Walk()`](https://pkg.go.dev/github.com/charlievieth/fastwalk#Walk):

```
 filepath fastwalk delta
time/op 88.0ms ± 1% 14.6ms ± 1% -83.47%
alloc/op 5.68MB ± 0% 6.76MB ± 0% +19.01%
allocs/op 69.6k ± 0% 90.4k ± 0% +29.87%
```

#### [`godirwalk.Walk()`](https://pkg.go.dev/github.com/karrick/godirwalk@v1.16.1#Walk) vs. [`fastwalk.Walk()`](https://pkg.go.dev/github.com/charlievieth/fastwalk#Walk):

```
 filepath fastwalk delta
time/op 87.4ms ± 1% 14.6ms ± 1% -83.34%
alloc/op 6.14MB ± 0% 6.76MB ± 0% +10.24%
allocs/op 100k ± 0% 90k ± 0% -9.59%
```

fastwalk

A simple HTTP middleware to automatically handle all your file upload needs

# Gulter

[![Go Reference](https://pkg.go.dev/badge/github.com/adelowo/gulter.svg)](https://pkg.go.dev/github.com/adelowo/gulter)
[![Go Report Card](https://goreportcard.com/badge/github.com/adelowo/gulter)](https://goreportcard.com/report/github.com/adelowo/gulter)

Gulter is a Go HTTP middleware designed to simplify the process of uploading files
for your web apps. It follows the standard
`http.Handler` and `http.HandlerFunc` interfaces so you can
always use with any of framework or the standard library router.

> [!NOTE]
> Name and idea was gotten from the insanely popular multer package in NodeJS that does the same

- [Installation](#installation)
- [Usage](#usage)
 - [Standard HTTP router](#standard-http-router)
 - [Chi Router and others](#chi-router-and-other-compatible-http-handlers)
- [API](#api)
- [FAQs](#faqs)
 - [customizing http error](#customizing-the-error-response)
 - [ignoring keys](#ignoring-non-existent-keys-in-the-multipart-request)
 - [custom validation logic](#writing-your-custom-validator-logic)

## Installation

```sh

go get -u -v github.com/adelowo/gulter

```

## Usage

Assuming you have a HTML form like this:

```html
<form action="/" method="post" enctype="multipart/form-data">
 <input type="file" name="form-field-1" />
 <input type="file" name="form-field-2" />
</form>
```

To create a new Gulter instance, you can do something like this:

```go
 handler, _ := gulter.New(
 gulter.WithMaxFileSize(10<<20),
 gulter.WithValidationFunc(
 gulter.ChainValidators(gulter.MimeTypeValidator("image/jpeg", "image/png"),
 func(f gulter.File) error {
 // Your own custom validation function on the file here
 // Else you can really just drop the ChainValidators and use only the MimeTypeValidator or just
 // one custom validator alone
 return nil
 })),
 gulter.WithStorage(s3Store),
 )
```

The `handler` is really just a HTTP middleware with the following signature
`Upload(keys ...string) func(next http.Handler) http.Handler`. `keys` here
are the input names from the HTML form, so you can chain this into almost any HTTP
router.

### Standard HTTP router

```go
package main

import (
 "fmt"
 "net/http"

 "github.com/adelowo/gulter"
 "github.com/adelowo/gulter/storage"
)

func main() {
 s3Store, err := storage.NewS3FromEnvironment(storage.S3Options{
 Bucket: "std-router",
 })
 if err != nil {
 panic(err.Error())
 }

 handler, err := gulter.New(
 gulter.WithMaxFileSize(10<<20),
 gulter.WithStorage(s3Store),
 )

 mux := http.NewServeMux()

 // upload all files with the "name" and "lanre" fields on this route
 mux.Handle("/", handler.Upload("name", "lanre")(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 fmt.Println("Uploaded file")

 // return all uploaded files
 f, err := gulter.FilesFromContext(r)
 if err != nil {
 fmt.Println(err)
 return
 }

 // return uploaded files with the form field "lanre"
 ff, err := gulter.FilesFromContextWithKey(r, "lanre")
 if err != nil {
 fmt.Println(err)
 return
 }

 fmt.Printf("%+v", ff)

 for _, v := range f {
 fmt.Printf("%+v", v)
 fmt.Println()
 }
 })))

 http.ListenAndServe(":3300", mux)

```

### Chi router and other compatible HTTP handlers

```go

 s3Store, err := storage.NewS3FromEnvironment(storage.S3Options{
 Bucket: "chi-router",
 })
 if err != nil {
 panic(err.Error())
 }

 handler := gulter.New(
 gulter.WithMaxFileSize(10<<20),
 gulter.WithValidationFunc(gulter.ChainValidators(gulter.MimeTypeValidator("image/jpeg", "image/png"))),
 gulter.WithStorage(s3Store),
 )

 router := chi.NewMux()

 // upload all files in the form fields called "form-field-1" and "form-field-2"
 router.With(handler.Upload("form-field-1", "form-field-2")).Post("/", func(w http.ResponseWriter, r *http.Request) {
 fmt.Println("Uploaded file")

 f, err := gulter.FilesFromContext(r)
 if err != nil {
 fmt.Println(err)
 return
 }

 ff, err := gulter.FilesFromContextWithKey(r, "form-field-1") // or form-field-2
 if err != nil {
 fmt.Println(err)
 return
 }

 fmt.Printf("%+v", ff)

 for _, v := range f {
 fmt.Printf("%+v", v)
 fmt.Println()
 }
 })

```

## API

While this middleware automatically uploads your files, sometimes you need
details about the uploaded file to show to the user, this could be making up the
image url or path to the image. To get that in your HTTP handler, you can use either:

- `FilesFromContextWithKey`: retrieve a named input uploaded file.
- `FilesFromContext`: retrieve all uploaded files

Gulter also ships with two storage implementations at the moment:

- `S3Store` : supports S3 or any compatible service like Minio,R2 & others
- `DiskStore`: uses a local filesystem backed store to upload files
- `CloudinaryStore`: uploads file to cloudinary

## FAQs

### Ignoring non existent keys in the multipart Request

Sometimes, the keys you have configured the middleware might get dropped from the
frontend for some reason, ideally the middleware fails if it cannot find a
configured key in the request. To disable this behavior and ignore the missing
key, you can make use of the `WithIgnoreNonExistentKey(true)` option to prevent the
middleware from causing an error when such keys do not exists

### Customizing the error response

Since Gulter is a middleware that runs, it returns an error to the client if found,
this might not match your existing structure, so to configure the response, use the
`WithErrorResponseHandler`. The default is shown below and can be used as a template
to define yours.

```go

 errHandler ErrResponseHandler = func(err error) http.HandlerFunc {
 return func(w http.ResponseWriter, _ *http.Request) {
 w.Header().Set("Content-Type", "application/json")
 w.WriteHeader(http.StatusInternalServerError)
 fmt.Fprintf(w, `{"message" : "could not upload file", "error" : %s}`, err.Error())
 }
 }
```

### Writing your custom validator logic

Sometimes, you could have some custom logic to validate uploads, in this example
below, we limit the size of the upload based on the mimeypes of the uploaded files

```go

var customValidator gulter.ValidationFunc = func(f gulter.File) error {
 switch f.MimeType {
 case "image/png":
 if f.Size > 4096 {
 return errors.New("file size too large")
 }

 return nil

 case "application/pdf":
 if f.Size > (1024 * 10) {
 return errors.New("file size too large")
 }

 return nil
 default:
 return nil
 }
}

```

gulter

Modern full-lifecycle PDF library for Go — parse, extract tables, generate, and sign documents with zero CGO dependencies

# GxPDF - Enterprise-Grade PDF Library for Go

![GxPDF Cover](assets/gh_cover.png)

[![GitHub Release](https://img.shields.io/github/v/release/coregx/gxpdf)](https://github.com/coregx/gxpdf/releases/latest)
[![Go Version](https://img.shields.io/badge/Go-1.25+-00ADD8?style=flat&logo=go)](https://go.dev/)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![CI](https://github.com/coregx/gxpdf/actions/workflows/test.yml/badge.svg)](https://github.com/coregx/gxpdf/actions/workflows/test.yml)
[![codecov](https://codecov.io/gh/coregx/gxpdf/branch/main/graph/badge.svg)](https://codecov.io/gh/coregx/gxpdf)
[![GoDoc](https://pkg.go.dev/badge/github.com/coregx/gxpdf)](https://pkg.go.dev/github.com/coregx/gxpdf)
[![Go Report Card](https://goreportcard.com/badge/github.com/coregx/gxpdf)](https://goreportcard.com/report/github.com/coregx/gxpdf)
[![GitHub Stars](https://img.shields.io/github/stars/coregx/gxpdf)](https://github.com/coregx/gxpdf/stargazers)

**GxPDF** is a modern, high-performance PDF library for Go, built with clean architecture and Go 1.25+ best practices.

**[View Example PDF](assets/gxpdf_enterprise_brochure.pdf)** - See what GxPDF can create!

## Key Features

### Declarative Builder API (NEW)

QuestPDF-inspired declarative API with 12-column grid, automatic pagination, and composable components. Define documents as nested closures — the engine handles page breaks, header/footer repetition, and two-pass page number resolution automatically.

```go
import "github.com/coregx/gxpdf/builder"

doc := builder.NewBuilder(
    builder.WithPageSize(builder.A4),
    builder.WithMargins(builder.Mm(20), builder.Mm(15), builder.Mm(20), builder.Mm(15)),
    builder.WithTitle("Invoice"),
    builder.WithDefaultFontSize(11),
)

doc.Page(func(page *builder.PageBuilder) {
    page.Header(func(h *builder.Container) {
        h.Row(func(r *builder.RowBuilder) {
            r.Col(8, func(c *builder.ColBuilder) {
                c.Text("ACME Corporation", builder.Bold(), builder.FontSize(16))
            })
            r.Col(4, func(c *builder.ColBuilder) {
                c.Text("INVOICE", builder.Bold(), builder.FontSize(20),
                    builder.AlignRight(), builder.TextColor(builder.Navy))
            })
        })
        h.Line()
    })

    page.Content(func(content *builder.Container) {
        content.Spacer(builder.Mm(10))
        content.Row(func(r *builder.RowBuilder) {
            r.Col(6, func(c *builder.ColBuilder) {
                c.Text("Bill To:", builder.Bold())
                c.Text("John Doe")
                c.Text("123 Main Street")
            })
            r.Col(6, func(c *builder.ColBuilder) {
                c.Text("Invoice #: INV-2026-001", builder.AlignRight())
                c.Text("Date: March 23, 2026", builder.AlignRight())
            })
        })
    })

    page.Footer(func(f *builder.Container) {
        f.PageNumber(builder.PageNum+" of "+builder.TotalPages,
            builder.AlignCenter(), builder.FontSize(8), builder.TextColor(builder.Gray))
    })
})

pdfBytes, err := doc.Build()
```

### Digital Signatures (NEW)

Sign and verify PDF documents with zero external dependencies:

```go
import "github.com/coregx/gxpdf/signature"

// Generate or load your certificate
key, cert, _ := signature.GenerateTestCertificate()
signer, _ := signature.NewLocalSigner(key, []*x509.Certificate{cert})

// Sign a PDF
signed, err := signature.SignDocument(pdfBytes, signer,
    signature.WithReason("Approved"),
    signature.WithLocation("Moscow"),
)

// Verify signatures
infos, err := signature.Verify(signed)
fmt.Println(infos[0].SignedBy, infos[0].Valid) // "CN=Test" true
```

- **PAdES B-B** — CMS/PKCS#7 with ESS signing-certificate-v2
- **PAdES B-T** — RFC 3161 timestamping via `WithTimestamp(tsaURL)`
- **RSA + ECDSA** — SHA-256 by default, SHA-384/512 configurable
- **Verification** — ByteRange hash + CMS cryptographic verification
- **Incremental update** — preserves existing content and prior signatures

### PDF Creation (Creator API)
- **Text & Typography** - Rich text with multiple fonts, styles, and colors
- **Graphics** - Lines, rectangles, circles, polygons, ellipses, arcs (wedge/chord), cubic and quadratic Bezier curves
- **Gradients** - Linear and radial gradient fills with full PDF Shading (multi-stop support)
- **Color Spaces** - RGB and CMYK support
- **Tables** - Complex tables with merged cells, borders, backgrounds
- **Images** - JPEG and PNG with transparency support
- **Fonts** - Standard 14 PDF fonts + TTF/OTF embedding with full Unicode support (Cyrillic, CJK, symbols)
- **Document Structure** - Chapters, auto-generated Table of Contents
- **Annotations** - Sticky notes, highlights, underlines, stamps
- **Interactive Forms** - Text fields, checkboxes, radio buttons, dropdowns
- **Security** - RC4 (40/128-bit) and AES (128/256-bit) encryption
- **Watermarks** - Text watermarks with rotation and opacity
- **Page Sizes** - 35+ built-in sizes (ISO A/B/C, ANSI, photo, book, JIS, envelopes, slides)
- **Custom Dimensions** - Arbitrary page sizes with unit conversion helpers (inches, mm, cm)
- **Landscape/Portrait** - True landscape via `NewPageWithSize(A4, Landscape)` — industry-standard swapped-MediaBox
- **Text Rotation** - Rotated text at any angle via `Tm` operator (standard 14 + custom TTF fonts)
- **Opacity/Transparency** - Text and shape opacity via ExtGState (`AddTextColorAlpha`, shape `Opacity` option)
- **Page Operations** - Merge, split, rotate, append

### PDF Reading & Extraction
- **Encrypted PDF Reading** - Open password-protected PDFs (RC4-40/128, AES-128)
- **Table Extraction** - Industry-leading accuracy (100% on bank statements)
- **Text Extraction** - Full text with positions and Unicode support
- **Image Extraction** - Extract embedded images
- **Export Formats** - CSV, JSON, Excel

## Installation

```bash
go get github.com/coregx/gxpdf
```

**Requirements**: Go 1.25 or later

## Quick Start

### Creating a PDF with the Builder API

```go
package main

import (
    "log"
    "github.com/coregx/gxpdf/builder"
)

func main() {
    doc := builder.NewBuilder(
        builder.WithPageSize(builder.A4),
        builder.WithTitle("Hello World"),
    )

    doc.Page(func(page *builder.PageBuilder) {
        page.Content(func(c *builder.Container) {
            c.Text("Hello, GxPDF!", builder.Bold(), builder.FontSize(24))
            c.Spacer(builder.Mm(5))
            c.Text("Professional PDF creation in Go.")
        })
    })

    if err := doc.BuildToFile("output.pdf"); err != nil {
        log.Fatal(err)
    }
}
```

### Creating a PDF Document (Creator API)

```go
package main

import (
    "log"
    "github.com/coregx/gxpdf/creator"
)

func main() {
    c := creator.New()
    c.SetTitle("My Document")
    c.SetAuthor("GxPDF")

    page, _ := c.NewPage()

    // Add text
    page.AddText("Hello, GxPDF!", 100, 750, creator.HelveticaBold, 24)
    page.AddText("Professional PDF creation in Go", 100, 720, creator.Helvetica, 12)

    // Draw shapes
    page.DrawRectangle(100, 600, 200, 100, &creator.RectangleOptions{
        FillColor:   &creator.Blue,
        StrokeColor: &creator.Black,
        StrokeWidth: 2,
    })

    if err := c.WriteToFile("output.pdf"); err != nil {
        log.Fatal(err)
    }
}
```

### Unicode Text with Custom Fonts

```go
c := creator.New()
page, _ := c.NewPage()

// Load custom font with Unicode support
font, _ := c.LoadFont("/path/to/arial.ttf")

// Cyrillic text
page.AddTextCustomFont("Привет, мир!", 100, 700, font, 18)

// CJK text (requires appropriate font like Malgun Gothic)
cjkFont, _ := c.LoadFont("/path/to/malgun.ttf")
page.AddTextCustomFont("你好世界 • 안녕하세요", 100, 670, cjkFont, 16)

c.WriteToFile("unicode.pdf")
```

### Creating Encrypted PDFs

```go
c := creator.New()

c.SetEncryption(creator.EncryptionOptions{
    UserPassword:  "user123",
    OwnerPassword: "owner456",
    Permissions:   creator.PermissionPrint | creator.PermissionCopy,
    Algorithm:     creator.EncryptionAES256,
})

page, _ := c.NewPage()
page.AddText("This document is encrypted!", 100, 750, creator.Helvetica, 14)
c.WriteToFile("encrypted.pdf")
```

### Creating Documents with Chapters and TOC

```go
c := creator.New()
c.EnableTOC()

ch1 := creator.NewChapter("Introduction")
ch1.Add(creator.NewParagraph("Introduction content..."))

ch1_1 := ch1.NewSubChapter("Background")
ch1_1.Add(creator.NewParagraph("Background information..."))

ch2 := creator.NewChapter("Methods")
ch2.Add(creator.NewParagraph("Methods description..."))

c.AddChapter(ch1)
c.AddChapter(ch2)

c.WriteToFile("document_with_toc.pdf")
```

### Interactive Forms (AcroForm)

```go
import "github.com/coregx/gxpdf/creator/forms"

c := creator.New()
page, _ := c.NewPage()

// Text field
nameField := forms.NewTextField("name", 100, 700, 200, 20)
nameField.SetLabel("Full Name:")
nameField.SetRequired(true)
page.AddField(nameField)

// Checkbox
agreeBox := forms.NewCheckbox("agree", 100, 660, 15, 15)
agreeBox.SetLabel("I agree to the terms")
page.AddField(agreeBox)

// Dropdown
countryDropdown := forms.NewDropdown("country", 100, 620, 150, 20)
countryDropdown.AddOption("us", "United States")
countryDropdown.AddOption("uk", "United Kingdom")
page.AddField(countryDropdown)

c.WriteToFile("form.pdf")
```

### Reading and Filling Forms

```go
// Read form fields from existing PDF
doc, _ := gxpdf.Open("form.pdf")
defer doc.Close()

// Check if document has a form
if doc.HasForm() {
    fields, _ := doc.GetFormFields()
    for _, f := range fields {
        fmt.Printf("%s (%s): %v\n", f.Name(), f.Type(), f.Value())
    }
}

// Fill form fields
app, _ := creator.NewAppender("form.pdf")
defer app.Close()

app.SetFieldValue("name", "John Doe")
app.SetFieldValue("email", "john@example.com")
app.SetFieldValue("agree", true)  // Checkbox
app.SetFieldValue("country", "USA")  // Dropdown

app.WriteToFile("filled_form.pdf")
```

### Flattening Forms

```go
// Convert form fields to static content (non-editable)
app, _ := creator.NewAppender("filled_form.pdf")
defer app.Close()

app.FlattenForm()  // Flatten all fields
// Or: app.FlattenFields("signature", "date")  // Specific fields

app.WriteToFile("flattened.pdf")
```

### Extracting Text from PDFs

```go
doc, _ := gxpdf.Open("document.pdf")
defer doc.Close()

// Extract text from a specific page (1-based)
text, err := doc.ExtractTextFromPage(1)
if err != nil {
    log.Fatal(err)
}
fmt.Println(text)

// Or iterate all pages
for _, page := range doc.Pages() {
    fmt.Println(page.ExtractText())
}
```

### Extracting Tables from PDFs

```go
doc, _ := gxpdf.Open("bank_statement.pdf")
defer doc.Close()

tables := doc.ExtractTables()
for _, table := range tables {
    fmt.Printf("Table: %d rows x %d cols\n", table.RowCount(), table.ColumnCount())

    // Export to CSV
    csv, _ := table.ToCSV()
    fmt.Println(csv)
}
```

### Reading Encrypted PDFs

```go
// PDFs with empty user password (permissions-only) open transparently
doc, _ := gxpdf.Open("bank_statement_encrypted.pdf")
defer doc.Close()

// PDFs requiring a password
doc, err := gxpdf.OpenWithPassword("protected.pdf", "secret")
if errors.Is(err, gxpdf.ErrPasswordRequired) {
    log.Fatal("Wrong password")
}
defer doc.Close()

fmt.Printf("Pages: %d\n", doc.PageCount())
```

## Package Structure

```
github.com/coregx/gxpdf
├── gxpdf.go          # Main entry point (Open, OpenWithPassword)
├── builder/          # Declarative Builder API (12-col grid, tables, rich text)
├── layout/           # Pure computation layout engine (zero PDF deps)
├── signature/        # Digital signatures (PAdES B-B/B-T, verify)
├── export/           # Export formats (CSV, JSON, Excel)
├── creator/          # Low-level PDF creation API
│   └── forms/        # Interactive form fields
├── logging/          # Configurable debug logging
└── internal/         # Private implementation
    ├── application/  # Use cases (extraction, reading)
    └── infrastructure/ # PDF parsing, encoding, writing
```

## Documentation

- **[API Reference](https://pkg.go.dev/github.com/coregx/gxpdf)** - Full API documentation
- **[Examples](examples/)** - Code examples for all features
- **[Enterprise Brochure (PDF)](assets/gxpdf_enterprise_brochure.pdf)** - Sample PDF created with GxPDF
- **[Architecture](docs/ARCHITECTURE.md)** - DDD architecture overview
- **[Contributing](CONTRIBUTING.md)** - Contribution guidelines
- **[Security](SECURITY.md)** - Security policy

## Testing

```bash
# Run all tests
go test ./...

# Run with race detector
go test -race ./...

# Run with coverage
go test -cover ./...
```

### Debug Logging

Debug output is disabled by default. To enable it, configure a logger via the `logging` package:

```go
import (
    "log/slog"
    "os"
    "github.com/coregx/gxpdf/logging"
)

logging.SetLogger(slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{
    Level: slog.LevelWarn,
})))
```

Convenience methods like `ExtractText()`, `ExtractTables()`, and `GetImages()` log errors via slog instead of returning them. Enable logging to see why these methods return empty results.

## Roadmap

See [ROADMAP.md](ROADMAP.md) for the full development plan and version history.

## License

GxPDF is released under the **MIT License**. See [LICENSE](LICENSE) for details.

## Support

- **Issues**: [GitHub Issues](https://github.com/coregx/gxpdf/issues)
- **Discussions**: [GitHub Discussions](https://github.com/coregx/gxpdf/discussions)

---

**Built with Go 1.25+ and Domain-Driven Design**


The Best Go Libraries For File Handling (28)