parsefields

Tool for parse JSON-like logs for collecting unique fields. Main purpose to collect JSON-data with typical events and fields it is useful when you want to create mapping schema for database and you want to reduce the risks of forgotten fields. By default separator between to nested structs is "->", but you can change it with environment.

API consist:

-  POST /v1/json/
-  POST /v1/mjson/
-  GET /v1/fileds/
-  GET  /v1/events/
-  GET /v1/events/:logname/:eventid
-  DELETE /v1/events/:logname/:eventid
-  DELETE /v1/fields/:field

P.S. additionally info about all new events/fields will be show in stdout.

Deploy

docker build . -t parsefield
docker run -d -p 8000:8000 parsefield

or

docker-compose -p 8000:8000 -d up 

Usage

Push new log for parse

Single message per request

curl -X POST -d '{"process_name": "calc.exe", "process_path":"C:\\windows\\system32"}'  127.0.0.1:8000/v1/json/

Multiple message per request

curl -X POST -d '[{"process_name": "calc.exe", "process_path":"C:\\windows\\system32"},{"process_image": "calc.exe", "process_path":"C:\\windows\\system32"},{"pid":"1"}]'  127.0.0.1:8000/v1/mjson/

All unique fields

curl 127.0.0.1:8000/v1/fields/

All unique events

curl 127.0.0.1:8000/v1/events/

Show body of event

curl 127.0.0.1:8000/v1/events/Sysmon/999 

Delete events, fields

curl -X DELETE 127.0.0.1:8000/v1/events/Sysmon/999 - delete events with logname Sysmon and eventId 999
curl -X DELETE 127.0.0.1:8000/v1/fields/key - delete field with name key