{"id":13338945,"url":"https://github.com/positiveblue/heron_cpp_prototype","last_synced_at":"2026-03-19T09:30:18.474Z","repository":{"id":70775776,"uuid":"72693551","full_name":"positiveblue/heron_cpp_prototype","owner":"positiveblue","description":"A C++ prototype of Heron the stream processing engine from Twitter ","archived":false,"fork":false,"pushed_at":"2017-04-05T19:00:39.000Z","size":293,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-12-31T20:43:05.424Z","etag":null,"topics":["heron","prototype","real-time","streamming","twitter"],"latest_commit_sha":null,"homepage":"http://heronstreaming.io","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/positiveblue.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-11-03T00:34:43.000Z","updated_at":"2021-02-25T11:51:08.000Z","dependencies_parsed_at":null,"dependency_job_id":"a5b8029b-7d8e-43f9-8985-4e01e9b59f88","html_url":"https://github.com/positiveblue/heron_cpp_prototype","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/positiveblue%2Fheron_cpp_prototype","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/positiveblue%2Fheron_cpp_prototype/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/positiveblue%2Fheron_cpp_prototype/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/positiveblue%2Fheron_cpp_prototype/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/positiveblue","download_url":"https://codeload.github.com/positiveblue/heron_cpp_prototype/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239933067,"owners_count":19720728,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["heron","prototype","real-time","streamming","twitter"],"created_at":"2024-07-29T19:17:53.317Z","updated_at":"2026-03-19T09:30:16.390Z","avatar_url":"https://github.com/positiveblue.png","language":"C++","readme":"# Heron C++ prototype \n\n[Heron](http://twitter.github.io/heron/) is realtime analytics platform developed by Twitter. It is the direct successor of Apache Storm, built to be backwards compatible with Storm's topology API but with a wide array of architectural improvements.\n\nBack in september 2016, I attended to a talk given by [Karthik](https://twitter.com/karthikz), lead of the real time analytics at Twitter.\nAfter the talk I talked with Karthik to see if I could get involved in the project. Heron is mainly written in Java, Python and C++ but the only\nrobust API for the users is in Java. They had just started to writte a lite version for python and given that many companies have their infastructure in C++ having this platform in C++ would be great.\n\nThe main concept of Heron is *Tuple*. This tuples are implemented in Java as a map/array of objects. They are serialized, sent throw the network and deserialized using the features and flexibility that the JVM proporcinate. In C++ the arquitecture has to be a bit more complicated.\n\nIn this repository you can find a prototype of how a possible C++ implementation could look like. It is totally deattached of the Heron code but it is a good guide of what is necesary in a real-world implementation.\n\n## Architecture\nThe main features that I implemented are:\n\n- **Element:** A pure virtual class with two main methods: save and load. They are going to be used for the serialization/deserealization part. It uses a C++ library called [Cereal](http://uscilab.github.io/cereal/) for it.\n- **Basic types:** It is an Element wrapper for some basic types (String, Int, Double)\n- **Tuple:** A tuple is a collection (unordered_map) of *Elements*\n- **Serializer:** A pluggable serializer to define how tuples are serialize/deserialize (user defined)\n\n\n## Example\n\nHere is an example of how it looks like. I implemented two auxiliar clases *Tweet* and *User* to ilustrate it.\n\nFirst, we create the elements that we want to work with:\n```cpp\n    // Elements\n    std::shared_ptr\u003cElement\u003e eInt(new Int(15));\n    std::shared_ptr\u003cElement\u003e eDouble(new Double(3.14159));\n    std::shared_ptr\u003cElement\u003e eString(new String(\"Jordi\"));\n```\n\nWe Set them in a tuple:\n```cpp\n// Tuple\n    Tuple tuple;\n\n    tuple.Set(\"Worker\", eString);\n    tuple.Set(\"Salary\", eInt);\n    tuple.Set(\"Phi\", eDouble);\n```\n\nUsing dependency injection we serialize the tuple (the user would have to define the order a tuple is serialized/deserialized)\n```cpp\n    ////////////////////////\n    //* Sending a tuple *//\n    //////////////////////\n\n\n    // sstream: Will contain the serialization of a tuple\n    std::stringstream os;\n\n    // Serializer\n    IPluggableSerializer *CSerializer = new CerealSerializer();\n\n    tuple.serialize(CSerializer, os);\n    \n    writeToFile(\"serialize.out\", os);\n```\n\nDeserialize the tuple can be done casting the values:\n```cpp\n    //////////////////////////\n    //* Recieving a tuple *//\n    ////////////////////////\n\n    std::stringstream is;\n    readFromFile(\"serialize.out\", is);\n\n    Tuple new_tuple;\n    new_tuple.deserialize(CSerializer, is);\n\n    auto Salary = \n        std::static_pointer_cast\u003cInt\u003e(new_tuple.Get(\"Salary\"));\n    auto Phi = \n        std::static_pointer_cast\u003cDouble\u003e(new_tuple.Get(\"Phi\"));\n    auto Worker = \n        std::static_pointer_cast\u003cString\u003e(new_tuple.Get(\"Worker\"));\n\n    std::cout \u003c\u003c \"Salary: \" \u003c\u003c Salary-\u003egetValue() \u003c\u003c std::endl;\n    std::cout \u003c\u003c \"Phi: \" \u003c\u003c Phi-\u003egetValue() \u003c\u003c std::endl;\n    std::cout \u003c\u003c \"Worker: \" \u003c\u003c Worker-\u003egetValue() \u003c\u003c std::endl;\n```\n\nWell, this is more a proof of concept than a real-world implementation. If someone finds it interesting enough to integrate it in Heron I will be glad to help but I am busy right now with other projects so I could not do it alone.\n\n## License\n\nThis software is licensed under the same license as Heron. Learn more about it [here](https://github.com/jomsdev/heron_cpp_prototype/blob/master/LICENSE)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpositiveblue%2Fheron_cpp_prototype","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpositiveblue%2Fheron_cpp_prototype","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpositiveblue%2Fheron_cpp_prototype/lists"}