{"id":13411705,"url":"https://github.com/MonaxGT/parsefields","last_synced_at":"2025-03-14T17:31:04.180Z","repository":{"id":57496805,"uuid":"181090409","full_name":"MonaxGT/parsefields","owner":"MonaxGT","description":"Tools for parse JSON-like logs for collecting unique fields and events","archived":false,"fork":false,"pushed_at":"2019-05-05T18:55:53.000Z","size":33,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-11T12:43:01.132Z","etag":null,"topics":["event","fields","go","golang","json","log","logging","mapping","schema"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MonaxGT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-12T22:15:10.000Z","updated_at":"2024-10-29T06:21:08.000Z","dependencies_parsed_at":"2022-09-03T02:30:44.363Z","dependency_job_id":null,"html_url":"https://github.com/MonaxGT/parsefields","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonaxGT%2Fparsefields","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonaxGT%2Fparsefields/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonaxGT%2Fparsefields/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonaxGT%2Fparsefields/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MonaxGT","download_url":"https://codeload.github.com/MonaxGT/parsefields/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243618656,"owners_count":20320272,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["event","fields","go","golang","json","log","logging","mapping","schema"],"created_at":"2024-07-30T20:01:15.998Z","updated_at":"2025-03-14T17:31:03.782Z","avatar_url":"https://github.com/MonaxGT.png","language":"Go","readme":"[![Codacy Badge](https://api.codacy.com/project/badge/Grade/dc9f7432a97a4d5296045487e1db44f1)](https://app.codacy.com/app/MonaxGT/parsefields?utm_source=github.com\u0026utm_medium=referral\u0026utm_content=MonaxGT/parsefields\u0026utm_campaign=Badge_Grade_Dashboard)\n[![GoDoc](https://godoc.org/github.com/MonaxGT/parsefields?status.png)](https://godoc.org/github.com/MonaxGT/parsefields)\n[![Build Status](https://travis-ci.com/MonaxGT/gomalshare.svg?branch=master)](https://travis-ci.com/MonaxGT/gomalshare)\n[![Go Report Card](https://goreportcard.com/badge/github.com/MonaxGT/parsefields)](https://goreportcard.com/report/github.com/MonaxGT/parsefields)\n[![Say Thanks!](https://img.shields.io/badge/Say%20Thanks-!-1EAEDB.svg)](https://saythanks.io/to/MonaxGT)\n\n# parsefields\n\nTool for parse JSON-like logs for collecting unique fields. Main purpose to collect JSON-data with typical events and fields it is useful when you want to create mapping schema for database and you want to reduce the risks of forgotten fields. \nBy default separator between to nested structs is \"-\u003e\", but you can change it with environment.\n\nAPI consist:\n\n```\n-  POST /v1/json/\n-  POST /v1/mjson/\n-  GET /v1/fileds/\n-  GET  /v1/events/\n-  GET /v1/events/:logname/:eventid\n-  DELETE /v1/events/:logname/:eventid\n-  DELETE /v1/fields/:field\n```\n\nP.S. additionally info about all new events/fields will be show in stdout.\n\n## Deploy\n\n```sh\ndocker build . -t parsefield\ndocker run -d -p 8000:8000 parsefield\n```\n\nor \n\n```sh\ndocker-compose -p 8000:8000 -d up \n```\n\n## Usage\n\n### Push new log for parse\n\nSingle message per request\n\n```sh\ncurl -X POST -d '{\"process_name\": \"calc.exe\", \"process_path\":\"C:\\\\windows\\\\system32\"}'  127.0.0.1:8000/v1/json/\n```\n\nMultiple message per request \n\n```sh\ncurl -X POST -d '[{\"process_name\": \"calc.exe\", \"process_path\":\"C:\\\\windows\\\\system32\"},{\"process_image\": \"calc.exe\", \"process_path\":\"C:\\\\windows\\\\system32\"},{\"pid\":\"1\"}]'  127.0.0.1:8000/v1/mjson/\n```\n\n### All unique fields\n\n```sh\ncurl 127.0.0.1:8000/v1/fields/\n```\n\n### All unique events\n\n```sh\ncurl 127.0.0.1:8000/v1/events/\n```\n\n### Show body of event\n\n```sh\ncurl 127.0.0.1:8000/v1/events/Sysmon/999 \n```\n\n### Delete events, fields\n\n```sh\ncurl -X DELETE 127.0.0.1:8000/v1/events/Sysmon/999 - delete events with logname Sysmon and eventId 999\ncurl -X DELETE 127.0.0.1:8000/v1/fields/key - delete field with name key\n```\n","funding_links":[],"categories":["Data Structures","Data Structures and Algorithms","数据结构与算法","Data Integration Frameworks","Generators","数据结构`go语言实现的数据结构与算法`","Uncategorized","数据结构"],"sub_categories":["Advanced Console UIs","Text Analysis","文本分析","标准 CLI"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMonaxGT%2Fparsefields","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FMonaxGT%2Fparsefields","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMonaxGT%2Fparsefields/lists"}