{"id":13482072,"url":"https://github.com/Netflix/suro","last_synced_at":"2025-03-27T12:32:10.694Z","repository":{"id":7560760,"uuid":"8914350","full_name":"Netflix/suro","owner":"Netflix","description":"Netflix's distributed Data Pipeline","archived":false,"fork":false,"pushed_at":"2023-04-10T10:22:36.000Z","size":4383,"stargazers_count":792,"open_issues_count":31,"forks_count":170,"subscribers_count":513,"default_branch":"master","last_synced_at":"2024-08-01T17:31:29.376Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Netflix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2013-03-20T21:02:32.000Z","updated_at":"2024-06-05T08:34:00.000Z","dependencies_parsed_at":"2022-09-26T21:51:02.849Z","dependency_job_id":"98b8e77b-bb44-4d69-aa79-027e7b385238","html_url":"https://github.com/Netflix/suro","commit_stats":{"total_commits":413,"total_committers":21,"mean_commits":"19.666666666666668","dds":0.728813559322034,"last_synced_commit":"e73d1a2e5b0492e41feb5a57d9e2a2e741a38453"},"previous_names":[],"tags_count":38,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Fsuro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Fsuro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Fsuro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Fsuro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Netflix","download_url":"https://codeload.github.com/Netflix/suro/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222252078,"owners_count":16955962,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T17:00:58.748Z","updated_at":"2024-10-30T15:31:42.556Z","avatar_url":"https://github.com/Netflix.png","language":"Java","readme":"# Suro: Netflix's Data Pipeline\n\nSuro is a data pipeline service for collecting, aggregating, and dispatching large volume of application events including log data. It has the following features:\n\n- It is distributed and can be horizontally scaled.\n- It supports streaming data flow, large number of connections, and high throughput.\n- It allows dynamically dispatching events to different locations with flexible dispatching rules.\n- It has a simple and flexible architecture to allow users to add additional data destinations.\n- It fits well into NetflixOSS ecosystem\n- It is a best-effort data pipeline with support of flexible retries and store-and-forward to minimize message loss\n\nLearn more about Suro on the \u003ca href=\"https://github.com/Netflix/suro/wiki\"\u003eSuro Wiki\u003c/a\u003e and the \u003ca href=\"http://techblog.netflix.com/2013/12/announcing-suro-backbone-of-netflixs.html\"\u003eNetflix TechBlog post\u003c/a\u003e where Suro was introduced.\n\n## Master Build Status\n\n\u003ca href='https://netflixoss.ci.cloudbees.com/job/suro-master/'\u003e\u003cimg src='https://netflixoss.ci.cloudbees.com/job/suro-master/badge/icon'\u003e\u003c/a\u003e\n\n## Pull Request Build Status\n\n\u003ca href='https://netflixoss.ci.cloudbees.com/job/suro-pull-requests/'\u003e\u003cimg src='https://netflixoss.ci.cloudbees.com/job/suro-pull-requests/badge/icon'\u003e\u003c/a\u003e\u003c/img\u003e\u003c/a\u003e\u003c/img\u003e\u003c/a\u003e\n\nBuild\n-----\n\nNetflixGraph is built via Gradle (www.gradle.org). To build from the command line:\n\n    ./gradlew build\n\nSee the `build.gradle` file for other gradle targets, like `distTar`, `distZip`, `installApp`, and `runServer`.\n\nRunning the server\n------------------\n\nYou can run the server locally by just running `./gradlew runServer`.\n\nMore more advanced usage you may wish to run `./gradlew installApp` and then:\n\n\tcd suro-server\n\tjava -cp \"build/install/suro-server/lib/*\" com.netflix.suro.SuroServer -m conf/routingmap.json -s conf/sink.json -i conf/input.json\n\nTo enable basic logging you can downloaded `slf4j-simple-1.7.7.jar` and copy it into `suro-server` then run:\n\n\tcd suro-server\n\tjava -cp \"build/install/suro-server/lib/*:slf4j-simple-1.7.7.jar\" com.netflix.suro.SuroServer -m conf/routingmap.json -s conf/sink.json -i conf/input.json\n\nSupport\n-----\n\nWe will use the Google Group, Suro Users, to discuss issues: https://groups.google.com/forum/#!forum/suro-users\n","funding_links":[],"categories":["Table of Contents","Java","Data Ingestion","others","Data Ingestion and Integration","II. Databases, search engines, big data and machine learning","大数据"],"sub_categories":["Data Pipeline","6. Working with messy data"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNetflix%2Fsuro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNetflix%2Fsuro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNetflix%2Fsuro/lists"}