Awesome Go

Scrapers

10 libraries in Text Processing / Scrapers

colly

Fast and Elegant Scraping Framework for Gophers

25,551Text ProcessingScrapers
GoQuery

GoQuery brings a syntax and a set of features similar to jQuery to the Go language

14,988Text ProcessingScrapers
xurls

Extract urls from text

1,266Text ProcessingScrapers
dataflowkit

Web scraping Framework to turn websites into structured data

713Text ProcessingScrapers
pagser

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

112Text ProcessingScrapers
doc-scraper

Web crawler that converts documentation sites to clean Markdown and JSONL for LLM ingestion (RAG, training data)

106Text ProcessingScrapers
Tagify

Produces a set of tags from given source

39Text ProcessingScrapers
go-recipe

A package for scraping recipes from websites

33Text ProcessingScrapers
walker

Seamlessly fetch paginated data from any source. Simple and high performance API scraping included

15Text ProcessingScrapers
go-sitemap-parser

Go language library for parsing Sitemaps

8Text ProcessingScrapers