{"id":24518286,"url":"https://github.com/codefeedr/extra-plugins","last_synced_at":"2025-03-15T11:12:15.769Z","repository":{"id":132918789,"uuid":"176237497","full_name":"codefeedr/extra-plugins","owner":"codefeedr","description":"A set of not-so-important CodeFeedr plugins. ","archived":false,"fork":false,"pushed_at":"2019-03-18T09:10:47.000Z","size":50,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-01-22T01:41:38.206Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codefeedr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-18T08:32:20.000Z","updated_at":"2019-03-18T09:10:48.000Z","dependencies_parsed_at":"2024-06-29T13:47:04.756Z","dependency_job_id":null,"html_url":"https://github.com/codefeedr/extra-plugins","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codefeedr%2Fextra-plugins","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codefeedr%2Fextra-plugins/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codefeedr%2Fextra-plugins/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codefeedr%2Fextra-plugins/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codefeedr","download_url":"https://codeload.github.com/codefeedr/extra-plugins/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243719398,"owners_count":20336607,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-22T01:41:41.870Z","updated_at":"2025-03-15T11:12:15.751Z","avatar_url":"https://github.com/codefeedr.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# extra-plugins\n[![Build Status](https://travis-ci.org/codefeedr/extra-plugins.svg?branch=master)](https://travis-ci.org/codefeedr/extra-plugins)\n\nA set of not-so-important CodeFeedr plugins. \n\n## Plugins\n- [RSS](#rss)\n- [Travis](#travis)\n- [Weblogs](#weblogs)\n- [Twitter](#twitter)\n\n\n# RSS\nA plugin with a stage for reading from a basic RSS feed using polling.\n\n### Installation\n\n```scala\ndependencies += \"org.codefeedr\" %% \"codefeedr-rss\" % \"0.1.0\"\n```\n\n### Configuration\n\nAn RSS input stage requires the address of the feed, the date format used in the `pubDate`, and the number of\nmilliseconds between each poll.\n\n```scala\nval dateFormat = \"EEE, dd MMMM yyyy HH:mm:ss z\"\nval pollingInterval = 1000 // once per second\nval rss = new RSSInput(\"http://example.com/feed.xml\", dateFormat, pollingInterval)\n```\n\nThe result is a stream of `RSSItem` objects with title, date, link, category and guid.\n# Travis\nThe Travis plugin provides two transform stages to interact with the Travis API. It relies on the Github plugin because\nbecause it uses the output of the Github push event stage (see [Github](github)).\n\n## Installation\n\n```scala\ndependencies += \"org.codefeedr\" %% \"codefeedr-travis\" % \"0.1.0\"\n```\n\n## Stages\n\nThe plugin contains two stages: \n\n- The `TravisFilterActiveReposTransformStage` takes a push-event stream from the GitHubEventToPushEvent Stage (see\n[Github](github)) and filters it to only keep push events from repos that are active on Travis\n \n- The `TravisPushEventBuildInfoTransformStage` takes a push-event from active Travis repositories stream and requests\nthe build information from Travis. The constructor only has one parameter:\n  \n    - `capacity`: to specify the amount of builds that are simultaneously requested per Flink thread (`100` is the \n    default).\n\n\n### Configuration\n\n\n**NOTE**: This plugin needs API access to TravisCI and requires keys. It will request keys from a key manager\nwith the target `travis`.\n\n### Examples\n```scala\nnew PipelineBuilder()\n  .setBufferType(BufferType.Kafka)\n  .append(new GitHubEventsInput())\n  .append(new GitHubEventToPushEvent)\n  .append(new TravisFilterActiveReposTransformStage())\n  .append(new TravisPushEventBuildInfoTransformStage(capacity = 10))\n  .build()\n  .startLocal()\n```\nThis pipeline will create a real-time Travis build stream by:\n\n- Reading from a the `/events` endpoint (GitHubEventsInput stage)\n- Filter (\u0026 parse) the PushEvents (GitHubEventsToPushEvent stage)\n- Filter the push events from repositories that are active on Travis\n- Request build information of those push events\n\n**Note**: These samples do not show configuration of for instance key management. See the sections above to show the\nconfigurable options.\n\n# Weblogs\nPlugin that provides input stages for web server logs.\n\nCurrently supports default Apache httpd access logs. NginX support is welcome through a pull request.\n\n### Installation\n\n```scala\ndependencies += \"org.codefeedr\" %% \"codefeedr-weblogs\" % \"0.1.0\"\n```\n\n### Configuration\n\nThe Apache Httpd log input creates a stream of `HttpdLogItem` events from a file. The `HttpdLogItem` case class\ncontains every part of a log line parsed into Scala format.\n\n```scala\nval log = HttpdLogInput(\"/var/httpd/access.log\")\n```\n\n### Notes\n\nThis plugin currently does only read the file once and does not monitor for new changes. It is a simple proof of\nconcept of an input stage.\n# Twitter\nThis plugin provides stages related to the Twitter API. \nIt makes uses of the [twitter4s](https://github.com/DanielaSfregola/twitter4s) library, for detailed documentation see their GitHub page.\n\n## Installation\n\n```scala\ndependencies += \"org.codefeedr\" %% \"codefeedr-twitter\" % \"0.1.0\"\n```\n\n\n## Stages\nCurrently two stages are provided related to Twitter statuses. The `TwitterStatusInput` is an InputStage which\nstreams Twitter statuses based on some filters.  The Twitter plugin also provides the `TwitterTrendingStatusInput` to get the tweets regarding the current list of trending topics.\n\n### Configuration\nFor all the Twitter related stages you must provide the following case classes:\n\n```scala \ncase class ConsumerToken(key: String, secret: String)\ncase class AccessToken(key: String, secret: String)\n```\n\nThose tokens can be generated using a [Twitter app](https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens.html).\n\n#### TwitterStatusInput\nThe filters of this stage are explained [here](https://github.com/joskuijpers/bep_codefeedr/blob/fd14096544fe5a2390a356bd5cb8781a52e28db8/codefeedr-plugins/codefeedr-twitter/src/main/scala/org/codefeedr/plugin/twitter/stages/TwitterStatusInput.scala#L37) and should be passed through the constructor.\n\n**Note**: At least 1 filter should be given, see [here](https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter.html).\n\n#### TwitterTrendingStatusInput\nFor the TwitterTrendingStatusInput a `sleepTime` should be given, this is the amount of minutes to wait before dynamically refreshing the trending topics\nand restarting the stream. By default this is 15 minutes. \n\n**Note:** Retrieving the trending topics is a request that is cached for 5 minutes at the Twitter API. Therefore, setting the `sleepTime` lower than 5 minutes doesn't make any sense. \n\n## Notes\nTwitter also ratelimits its streams, [twitter4s](https://github.com/DanielaSfregola/twitter4s) takes care of that. To get more information\non this ratelimit see [here](https://developer.twitter.com/en/docs/basics/rate-limiting.html).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodefeedr%2Fextra-plugins","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodefeedr%2Fextra-plugins","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodefeedr%2Fextra-plugins/lists"}