{"id":34552668,"url":"https://github.com/usedatabrew/open_ai_benthos_processor","last_synced_at":"2026-04-11T23:40:14.628Z","repository":{"id":205279377,"uuid":"712950596","full_name":"usedatabrew/open_ai_benthos_processor","owner":"usedatabrew","description":"Open AI processor for Benthos","archived":false,"fork":false,"pushed_at":"2023-11-05T12:19:02.000Z","size":36,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-11T23:39:41.584Z","etag":null,"topics":["benthos","benthos-plugin","data-engineering","data-ops","etl","etl-pipeline","event-sourcing","golang-library","openai","stream-processor","streaming-data"],"latest_commit_sha":null,"homepage":"https://databrew.tech","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/usedatabrew.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-11-01T14:37:58.000Z","updated_at":"2025-10-16T05:57:58.000Z","dependencies_parsed_at":"2025-12-24T08:05:15.409Z","dependency_job_id":null,"html_url":"https://github.com/usedatabrew/open_ai_benthos_processor","commit_stats":null,"previous_names":["usedatabrew/open_ai_benthos_processor"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/usedatabrew/open_ai_benthos_processor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usedatabrew%2Fopen_ai_benthos_processor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usedatabrew%2Fopen_ai_benthos_processor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usedatabrew%2Fopen_ai_benthos_processor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usedatabrew%2Fopen_ai_benthos_processor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/usedatabrew","download_url":"https://codeload.github.com/usedatabrew/open_ai_benthos_processor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usedatabrew%2Fopen_ai_benthos_processor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31699438,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-11T21:17:31.016Z","status":"ssl_error","status_checked_at":"2026-04-11T21:17:24.556Z","response_time":54,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benthos","benthos-plugin","data-engineering","data-ops","etl","etl-pipeline","event-sourcing","golang-library","openai","stream-processor","streaming-data"],"created_at":"2025-12-24T08:02:57.554Z","updated_at":"2026-04-11T23:40:14.579Z","avatar_url":"https://github.com/usedatabrew.png","language":"Go","readme":"# Open AI Processor for Benthos\n\n\u003cimg src='https://github.com/usedatabrew/pglogicalstream/blob/main/images/databrew-logo.png' width=\"200px\" align=\"middle\" \u003e\n\nBenthos processors are functions applied to messages passing through a pipeline. The function signature allows a processor to mutate or drop messages depending on the content of the message\n\nWelcome to the Open AI Processor for Benthos! This processor allows you to enrich the context of your messages with requests to Open AI.\n\n## Getting Started\n\nTo get started you have to run benthos with custom processor. Since this processor is not adopted by benthos itself\nyou have to create a new benthos build with plugin registered\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"github.com/benthosdev/benthos/v4/public/service\"\n\n    _ \"github.com/benthosdev/benthos/v4/public/components/all\"\n    // import open_ai processor\n    _ \"github.com/usedatabrew/open_ai_benthos_processor\"\n)\n\nfunc main() {\n\t// here we initialize benthos\n\tservice.RunCLI(context.Background())\n}\n```\n\n### Create benthos configuration with processor\n\n```yaml\npipeline:\n  label: open_ai_processor\n  processors:\n    - open_ai:\n        source_field: \"comment\"\n        target_field: \"ai_processed_comment\"\n        prompt: \"Check the field for hateful speech\"\n        api_key: \"\"\n        api_endpoint: \"\" ## Optional for azure driver\n        driver: \"azure\" ## azure or openai\n        model: \"gpt-3.5-turbo-instruct\"\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fusedatabrew%2Fopen_ai_benthos_processor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fusedatabrew%2Fopen_ai_benthos_processor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fusedatabrew%2Fopen_ai_benthos_processor/lists"}