{"id":50350503,"url":"https://github.com/jboulon/Honu","last_synced_at":"2026-06-15T12:01:41.062Z","repository":{"id":149664346,"uuid":"568369","full_name":"jboulon/Honu","owner":"jboulon","description":"Honu is a large scale data collection and processing pipeline","archived":false,"fork":false,"pushed_at":"2011-02-04T03:47:11.000Z","size":5865,"stargazers_count":82,"open_issues_count":2,"forks_count":5,"subscribers_count":9,"default_branch":"master","last_synced_at":"2023-04-01T07:26:35.596Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://wiki.github.com/jboulon/Honu/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jboulon.png","metadata":{"files":{"readme":"README","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2010-03-18T15:27:10.000Z","updated_at":"2023-04-01T07:40:21.151Z","dependencies_parsed_at":"2023-04-01T07:47:39.811Z","dependency_job_id":null,"html_url":"https://github.com/jboulon/Honu","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/jboulon/Honu","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jboulon%2FHonu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jboulon%2FHonu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jboulon%2FHonu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jboulon%2FHonu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jboulon","download_url":"https://codeload.github.com/jboulon/Honu/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jboulon%2FHonu/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34361403,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-15T02:00:07.085Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-29T21:00:23.920Z","updated_at":"2026-06-15T12:01:41.054Z","avatar_url":"https://github.com/jboulon.png","language":"Java","funding_links":[],"categories":["大数据"],"sub_categories":["微服务框架"],"readme":"Honu\nHonu a large scale streaming data collection and processing pipeline build in the cloud for the cloud.\n\nNetflix moved a large portion of their infrastructure to the cloud in order to meet our reliability, scalability and availability requirements.\nAs we solved our reliability, scalability and availability requirements on the application side, we open a new range of challenging problem on the log analytics side.\nAs the number of instances running in the cloud increase, standard way of moving log files or loading log events to a database starts saturating the system. As the volume explode, the latency/thruput of the system becomes so slow that it was not usable for our operational/BI needs.\n\nHonu is the new streaming log collection and processing pipeline in the cloud for Netflix and leverage the computational power of Hadoop/Hive to solve our log analytics requirements.\n\nHonu solution includes three main components:\n- A client side SDK\n- A collector, server side component to receive all logs events\n- A Map/Reduce to process/parse those logs and save them in Hive format.\n\nThe client side SDK provides all the classes you need in order to generate/collect your unstructured and structured log events on your application.\nHonu collectors are a key component and are responsible for continuously saving log Events to HDFS.\nThe Map/Reduce is responsible for parsing log events and storing them in their corresponding Hive table format.\n\nThe presentation will explain how we are using those components in order to generate, collect and process all the unstructured and structured log events and make them available to our end-user through Hive.\n\nHonu will be released under the standard Apache license and will be hosted for now in gitHub.\nHonu has been running in production at Netflix for more than a year and have some specific integrations to leverage the Amazon EC2, S3 and EMR infrastructure.\nHonu is globally used at Netflix , handling more use cases everyday and collecting over 12 billions of messages a day.\n\nMore information can be found here: http://wiki.github.com/jboulon/Honu\nJerome Boulon – jboulon@apache.org\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjboulon%2FHonu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjboulon%2FHonu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjboulon%2FHonu/lists"}