{"id":23752466,"url":"https://github.com/livelace/gosquito","last_synced_at":"2025-08-22T06:43:24.109Z","repository":{"id":41445097,"uuid":"307855215","full_name":"livelace/gosquito","owner":"livelace","description":"gosquito (\"go\" + \"mosquito\") is a pluggable tool for data gathering, data processing and data transmitting to various destinations.","archived":false,"fork":false,"pushed_at":"2024-07-07T21:50:54.000Z","size":3256,"stargazers_count":30,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-05T20:34:02.014Z","etag":null,"topics":["data-mining","jq","kafka","mattermost","minio","news","prometheus","regexp","rest","rss","s3","scrapper","slack","smtp","telegram","twitter","xpath"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/livelace.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-10-27T23:39:34.000Z","updated_at":"2025-02-07T14:31:35.000Z","dependencies_parsed_at":"2024-01-22T16:45:13.400Z","dependency_job_id":"fc55d87a-9bd3-4aa8-9c30-84f99af1e67c","html_url":"https://github.com/livelace/gosquito","commit_stats":null,"previous_names":[],"tags_count":45,"template":false,"template_full_name":null,"purl":"pkg:github/livelace/gosquito","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livelace%2Fgosquito","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livelace%2Fgosquito/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livelace%2Fgosquito/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livelace%2Fgosquito/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/livelace","download_url":"https://codeload.github.com/livelace/gosquito/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livelace%2Fgosquito/sbom","scorecard":{"id":594256,"data":{"date":"2025-08-11","repo":{"name":"github.com/livelace/gosquito","commit":"bfcb61970911a723feb6fa01235b59e16de69d1d"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.5,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v4.7.0 not signed: https://api.github.com/repos/livelace/gosquito/releases/163427157","Warn: release artifact v4.6.0 not signed: https://api.github.com/repos/livelace/gosquito/releases/161936676","Warn: release artifact v4.5.0 not signed: https://api.github.com/repos/livelace/gosquito/releases/160982225","Warn: release artifact v4.4.1 not signed: https://api.github.com/repos/livelace/gosquito/releases/101402129","Warn: release artifact v4.4.0 not signed: https://api.github.com/repos/livelace/gosquito/releases/72829214","Warn: release artifact v4.7.0 does not have provenance: https://api.github.com/repos/livelace/gosquito/releases/163427157","Warn: release artifact v4.6.0 does not have provenance: https://api.github.com/repos/livelace/gosquito/releases/161936676","Warn: release artifact v4.5.0 does not have provenance: https://api.github.com/repos/livelace/gosquito/releases/160982225","Warn: release artifact v4.4.1 does not have provenance: https://api.github.com/repos/livelace/gosquito/releases/101402129","Warn: release artifact v4.4.0 does not have provenance: https://api.github.com/repos/livelace/gosquito/releases/72829214"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":0,"reason":"12 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GO-2022-0635","Warn: Project is vulnerable to: GO-2022-0646","Warn: Project is vulnerable to: GO-2025-3372 / GHSA-6wxm-mpqj-6jpf","Warn: Project is vulnerable to: GO-2024-2948 / GHSA-xfhp-jf8p-mh5w","Warn: Project is vulnerable to: GO-2024-2947 / GHSA-v6v8-xj6m-xwqh","Warn: Project is vulnerable to: GO-2024-3321 / GHSA-v778-237x-gjrc","Warn: Project is vulnerable to: GO-2025-3487 / GHSA-hcg3-q754-cr77","Warn: Project is vulnerable to: GO-2024-3333","Warn: Project is vulnerable to: GO-2025-3503 / GHSA-qxp5-gwg8-xv66","Warn: Project is vulnerable to: GO-2025-3595 / GHSA-vvgc-356p-c3xw","Warn: Project is vulnerable to: GO-2025-3488 / GHSA-6v2p-p543-phr9","Warn: Project is vulnerable to: GO-2024-2631 / GHSA-c5q2-7r4c-mv6g"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-20T22:45:24.990Z","repository_id":41445097,"created_at":"2025-08-20T22:45:24.990Z","updated_at":"2025-08-20T22:45:24.990Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271599733,"owners_count":24787801,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-22T02:00:08.480Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-mining","jq","kafka","mattermost","minio","news","prometheus","regexp","rest","rss","s3","scrapper","slack","smtp","telegram","twitter","xpath"],"created_at":"2024-12-31T17:52:16.362Z","updated_at":"2025-08-22T06:43:24.062Z","avatar_url":"https://github.com/livelace.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gosquito\n\n**_gosquito_** (\"go\" + \"mosquito\") is a pluggable tool for data\ngathering, data processing and data transmitting to various destinations.\nMain goal is to replace various in-house automated tasks with a single tool and move those tasks at the edge.\nSee [docs](docs/README.md) and [examples](https://github.com/livelace/gosquito-public) for additional info.\n\n```mermaid\ngraph TD;\n    I1(io)--\u003eP1(process plugins);\n    I2(kafka)--\u003eP1;\n    I3(resty)--\u003eP1;\n    I4(rss)--\u003eP1;\n    I5(telegram)--\u003eP1;\n    P1--\u003eO1(kafka);\n    P1--\u003eO2(mattermost);\n    P1--\u003eO3(resty);\n    P1--\u003eO4(slack);\n    P1--\u003eO5(smtp);\n    P1--\u003eO6(telegram);\n```\n\n\u003cbr\u003e\n\n### Main features:\n\n- Pluggable architecture. Data processing organized as chains of plugins.\n- Flow approach. Flow consists of: input plugin (receive), process plugins (filter/transform), output\n  plugin (send).\n- Declarative YAML configurations with templates support.\n- Consider data as new by configurable signature (data timestamp by default). Force fetching is supported.\n- Dependencies between process plugins. Plugin \"B\" will process data only if plugin \"A\" derived some data.\n- Parallel executions limits.\n- Export metrics to [Prometheus](https://prometheus.io/).\n\n\u003cbr\u003e\n\n### Input plugins:\n\n| Plugin                                     | Description                                                                                    |\n|:-------------------------------------------|:-----------------------------------------------------------------------------------------------|\n| [io](docs/plugins/input/io.md)             | Use text and files as data source.                                                             |\n| [kafka](docs/plugins/input/kafka.md)       | [Kafka](https://kafka.apache.org/) topic as data source.                                       |\n| [resty](docs/plugins/input/resty.md)       | [REST](https://en.wikipedia.org/wiki/Representational_state_transfer) endpoint as data source. |\n| [rss](docs/plugins/input/rss.md)           | [RSS/Atom](https://en.wikipedia.org/wiki/RSS) feed as data source.                             |\n| [telegram](docs/plugins/input/telegram.md) | [Telegram](https://telegram.org/) chat as data source.                                         |\n\n### Process plugins:\n\n| Plugin                                                 | Description                                                                                            |\n|:-------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|\n| [dedup](docs/plugins/process/dedup.md)                 | Deduplicate datums by UUID.                                                                            |\n| [echo](docs/plugins/process/echo.md)                   | Echoing processing data.                                                                               |\n| [expandurl](docs/plugins/process/expandurl.md)         | Expand short URLs.                                                                                     |\n| [fetch](docs/plugins/process/fetch.md)                 | Fetch remote data.                                                                                     |\n| [iconv](docs/plugins/process/iconv.md)                 | Convert text encoding.                                                                                 |\n| [io](docs/plugins/process/io.md)                       | Read/write text and files.                                                                             |\n| [jq](docs/plugins/process/jq.md)                       | Extract JSON elements.                                                                                 |\n| [minio](docs/plugins/process/minio.md)                 | Get/put data from/to [S3](https://en.wikipedia.org/wiki/Amazon_S3) bucket.                             |\n| [regexpfind](docs/plugins/process/regexpfind.md)       | Find patters in data.                                                                                  |\n| [regexpmatch](docs/plugins/process/regexpmatch.md)     | Match data by patterns.                                                                                |\n| [regexpreplace](docs/plugins/process/regexpreplace.md) | Replace patterns in data.                                                                              |\n| [resty](docs/plugins/process/resty.md)                 | Perform REST queries.                                                                                  |\n| [same](docs/plugins/process/same.md)                   | Match data similarity.                                                                                 |\n| [split](docs/plugins/process/split.md)                 | Split single datum into multiple.                                                                      |\n| [unique](docs/plugins/process/unique.md)               | Get unique values from data.                                                                           |\n| [webchela](docs/plugins/process/webchela.md)           | Process web pages with [full browser support (Chrome, Firefox)](https://github.com/livelace/webchela). |\n| [xpath](docs/plugins/process/xpath.md)                 | Extract HTML/XML nodes.                                                                                |\n\n### Output plugins:\n\n| Plugin                                          | Description                                                                                  |\n|:------------------------------------------------|:---------------------------------------------------------------------------------------------|\n| [kafka](docs/plugins/output/kafka.md)           | Send data to [Kafka](https://kafka.apache.org/) topic.                                       |\n| [mattermost](docs/plugins/output/mattermost.md) | Send data to [Mattermost](https://mattermost.org/) channel/user.                             |\n| [resty](docs/plugins/output/resty.md)           | Send data to [REST](https://en.wikipedia.org/wiki/Representational_state_transfer) endpoint. |\n| [slack](docs/plugins/output/slack.md)           | Send data to [Slack](https://slack.com) channel/user.                                        |\n| [smtp](docs/plugins/output/smtp.md)             | Send data as email.                                                                          |\n| [telegram](docs/plugins/output/telegram.md)     | Send data to [Telegram](https://telegram.org) chat.                                          |","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flivelace%2Fgosquito","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flivelace%2Fgosquito","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flivelace%2Fgosquito/lists"}