{"id":21041386,"url":"https://github.com/spektom/spark-http-stream","last_synced_at":"2025-03-13T21:24:44.026Z","repository":{"id":138109875,"uuid":"98572560","full_name":"spektom/spark-http-stream","owner":"spektom","description":"Spark streaming HTTP direct stream implementation and usage example","archived":false,"fork":false,"pushed_at":"2017-07-27T19:29:04.000Z","size":250,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-20T16:53:27.040Z","etag":null,"topics":["http","spark","spark-streaming"],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spektom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-27T19:22:38.000Z","updated_at":"2017-08-07T07:49:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"2c31e681-8e58-4286-86a0-f6af122b34f0","html_url":"https://github.com/spektom/spark-http-stream","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spektom%2Fspark-http-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spektom%2Fspark-http-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spektom%2Fspark-http-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spektom%2Fspark-http-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spektom","download_url":"https://codeload.github.com/spektom/spark-http-stream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243483539,"owners_count":20298003,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["http","spark","spark-streaming"],"created_at":"2024-11-19T13:52:01.269Z","updated_at":"2025-03-13T21:24:44.007Z","avatar_url":"https://github.com/spektom.png","language":"Scala","readme":"spark-http-stream\n==================\n\nThe following example shows Spark streaming application, which reads\nfrom HTTP sources periodically, and processes downloaded content.\n\nThe input stream adapts automatically to the processing rate due to\nSpark streaming backpressure feature. The feature can be seen in action on this picture:\n\n![](bp.png)\n\n## Building\n\nTo build Spark application jar, run:\n\n    ~$ mvn package\n    \n## Running\n\nFirst, run some dummy multi-threaded Web server that listens on port 8000.\n\nThen, submit the application jar using the following command:\n \n    ~$ spark-submit --class com.github.spektom.spark.TestJob target/spark-http-stream_2.11-0.0.1.jar\n    \nTo view Spark streaming statistics, open this URL in your browser: [http://localhost:4040/streaming](http://localhost:4040/streaming)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspektom%2Fspark-http-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspektom%2Fspark-http-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspektom%2Fspark-http-stream/lists"}