{"id":19643767,"url":"https://github.com/logicalclocks/epipe","last_synced_at":"2026-03-11T03:31:43.253Z","repository":{"id":37864060,"uuid":"50195241","full_name":"logicalclocks/ePipe","owner":"logicalclocks","description":"ePipe is a metadata system for HopsFS that provides replicated-metadata-as-a-service. ","archived":false,"fork":false,"pushed_at":"2024-08-15T08:50:43.000Z","size":9197,"stargazers_count":3,"open_issues_count":2,"forks_count":10,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-04-28T13:33:50.970Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/logicalclocks.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-01-22T17:05:45.000Z","updated_at":"2024-06-04T11:14:53.000Z","dependencies_parsed_at":"2024-11-11T14:24:14.433Z","dependency_job_id":"9f30e0fd-335b-4814-9140-3a18958689b3","html_url":"https://github.com/logicalclocks/ePipe","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/logicalclocks/ePipe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2FePipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2FePipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2FePipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2FePipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/logicalclocks","download_url":"https://codeload.github.com/logicalclocks/ePipe/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2FePipe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30369379,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T21:41:54.280Z","status":"online","status_checked_at":"2026-03-11T02:00:07.027Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T14:23:58.550Z","updated_at":"2026-03-11T03:31:43.221Z","avatar_url":"https://github.com/logicalclocks.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ePipe\n\nePipe is a metadata system for HopsFS that provides replicated-metadata-as-a-service. The key component of ePipe is a databus that both creates a consistent, correctly-ordered change stream from HopsFS and eventually delivers the stream with low latency (sub-second) to external stores and downstream clients. \n\n\nHow To build\n============\n\nUsing Docker\n------------\n\nIn this setup, we compile ePipe againt different platforms (ubuntu, centos) using docker containers. Therefore, you will need to have docker installed on your machine.\n\nClone the ePipe repo, and then build it using docker.\n```\ngit clone https://github.com/logicalclocks/ePipe.git\ncd ePipe\n./docker-build.sh\n```\n\nThe docker build script will create two docker images _epipe_build_ubuntu_ and _epipe_build_centos_ with their tag set to be the current ePipe version. These images are created based on the corresponding docker files located in _docker/ubuntu_ and _docker/centos_.\nThen, it will run the containers to build the ePipe code for both platforms. The resultant binaries will be found under _builds_ directory.\n\n**Interactive mode**\n\nYou can use the docker interactive mode, to build and test ePipe againt ubuntu or centos depending on the image you use.\nAn example of running a docker container with the ubuntu image (_epipe_build_ubuntu_) where we mount the ePipe repo to the container. The script _build.sh_ can then be used to build ePipe. \n\n```\ncd ePipe\ndocker run --rm -v \"$PWD\":/usr/epipe:z -w /usr/epipe -it epipe_build_ubuntu:0.18.0 \nepipe@24d10f5aacc3:/usr/epipe$ ./build.sh\n```\n\nManual Setup\n------------\n**Software Required**\n\nFor compiling ePipe you will need the following software.\n\n* CMake 3.5.0 or higher (3.15.0 is recommended)\n* GCC 7.0 or higher\n* Boost 1.70 or higher \n* [RapidJson](http://rapidjson.org/) 1.1.0\n* [MySQL Cluster NDB](https://dev.mysql.com/downloads/cluster/)\n\nUntar the MySQL Cluster binaries in /usr/local/\n```\ncd /usr/local\nwget https://dev.mysql.com/get/Downloads/MySQL-Cluster-7.6/mysql-cluster-gpl-7.6.10-linux-glibc2.12-x86_64.tar.gz\ntar xzvf mysql-cluster-gpl-7.6.10-linux-glibc2.12-x86_64.tar.gz\nln -s mysql-cluster-gpl-7.6.10-linux-glibc2.12-x86_64 mysql\n```\n\nClone the ePipe repo, and then build it.\n```\ngit clone https://github.com/logicalclocks/ePipe.git\ncd ePipe\nmkdir build\ncd build\ncmake ..\nmake\n```\n\nHow To Run\n============\n\nTo run ePipe, you need to create a config.ini file, you can copy the config.ini.template and then update the configuration accordingly, then run ePipe.\n\n```\ncp config.ini.template config.ini\n./ePipe -c config.ini\n```\n\nA description of the allowed configuration parameters can be found in the config.ini.template file as well as when running the description (-d | --desc) switch on ePipe.\n```\n./ePipe -d\n```\n\nePipe is installed as part of the Hopsworks platform as specified by the [epipe-chef](https://github.com/logicalclocks/epipe-chef) cookbook\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flogicalclocks%2Fepipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flogicalclocks%2Fepipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flogicalclocks%2Fepipe/lists"}