{"id":13561901,"url":"https://github.com/paypal/junodb","last_synced_at":"2025-05-15T03:04:32.978Z","repository":{"id":163637507,"uuid":"606125384","full_name":"paypal/junodb","owner":"paypal","description":"JunoDB is PayPal's home-grown secure, consistent and highly available key-value store providing low, single digit millisecond, latency at any scale.","archived":false,"fork":false,"pushed_at":"2024-06-21T17:23:30.000Z","size":19692,"stargazers_count":2575,"open_issues_count":38,"forks_count":173,"subscribers_count":29,"default_branch":"dev","last_synced_at":"2025-04-03T17:08:49.347Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paypal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"docs/code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.txt","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-24T16:52:02.000Z","updated_at":"2025-03-15T08:33:26.000Z","dependencies_parsed_at":"2024-08-01T13:17:40.110Z","dependency_job_id":"c0c89b5d-809b-409d-ab1b-bab9661003b7","html_url":"https://github.com/paypal/junodb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paypal%2Fjunodb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paypal%2Fjunodb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paypal%2Fjunodb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paypal%2Fjunodb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paypal","download_url":"https://codeload.github.com/paypal/junodb/tar.gz/refs/heads/dev","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248368493,"owners_count":21092370,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:01:02.387Z","updated_at":"2025-04-11T09:35:15.612Z","avatar_url":"https://github.com/paypal.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# JunoDB - A secure, consistent and highly available key-value store\n\n[![License](http://img.shields.io/:license-Apache%202-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0.txt)\n[![Build](https://github.com/paypal/junoDB/actions/workflows/juno_server_bin_build.yml/badge.svg?branch=main)](https://github.com/paypal/junoDB/actions/workflows/juno_server_bin_build.yml)\n[![Docker](https://github.com/paypal/junoDB/actions/workflows/juno_server_docker_build.yml/badge.svg?branch=main)](https://github.com/paypal/junoDB/actions/workflows/juno_server_docker_build.yml)\n\n\n## What is JunoDB\nJunoDB is PayPal's home-grown secure, consistent and highly available key-value store providing low, single digit millisecond, latency at any scale. \n\n  * [High Level Architecture](docs/junodb_arch_overview.md)\n  * [Wire protocol](docs/wireprotocol.md)\n  * [Contributing](docs/contributing.md)\n\n\n## Getting Started with the JunoDB Server\n\n### Clone the repository from [github](https://github.com/paypal/junodb)\n\n```bash\ngit clone https://github.com/paypal/junodb.git\n```\n\n### Set BUILDTOP variable\n\n```bash\nexport BUILDTOP=\u003cpath_to_junodb_folder\u003e/junodb\ncd $BUILDTOP\n```\n\nContinue building JunoDB server with \n1. [Docker build](#docker-build) or \n2. [Manual build](#manual-build) \n\u003cbr\u003e\n\n## Docker Build\n\n* [Docker](#docker-build)    \n    * [Install Dependencies](#docker_install_dependencies)\n    * [Build JunoDB](#docker_build_junodb)\n    * [Run JunoDB](#docker_run_junodb)\n    * [Generate Secrets for Dev](#docker_secrets)\n    * [Validate JunoDB](#docker_validate_junodb)\n\n\n\u003c!-- toc --\u003e\n\n**Note** :\nDocker build supported platforms\n- Linux (Ubuntu)\n- OS X (macOS)\n### \u003ch3 id=\"docker_install_dependencies\"\u003eInstall Dependencies\u003c/h3\u003e\n[Install Docker Engine version 20.10.0+](https://docs.docker.com/engine/install/ubuntu/)\nCheck for existing docker version\n```bash\ndocker version\n```\n\nInstall Docker if not installed or version is older than 20.10.0\n```bash\ndocker/setup.sh\n#If you are not added to the docker group, you will have to logout and login in the machine after running docker/setup.sh\n```\n\nIf the user is not added to `docker` group, you may add manually. Logout and re-login after this step.\n```bash\nsudo usermod -a -G docker $USER\n```\n\nVerify that your username is added to docker group\n```bash\ngroups\n\n#or\ncat /etc/group | grep docker\n```\n\n\n### \u003ch3 id=\"docker_build_junodb\"\u003eBuild JunoDB\u003c/h3\u003e\n```bash\n#Login to docker hub account \ndocker login\n\n# Build junodb docker images\n#etcd\n#clustercfg\n#storageserv\n#proxy \n#junoclient\n\n\ndocker/build.sh \n```\n\n### \u003ch3 id=\"docker_run_junodb\"\u003eRun JunoDB\u003c/h3\u003e\n```bash\n# Setup junodb network and start junodb services\n#etcd\n#clustercfg\n#storageserv\n#proxy \n#junoclient\n\n\n# JunoDB proxy service listens on port \n# :5080 TLS and :8080 TCP\ndocker/start.sh \n\n```\n\n### Shutdown JunoDB services\n```bash\n# This will shutdown junodb services\n#etcd\n#clustercfg\n#storageserv\n#proxy \n#junoclient\n\ndocker/shutdown.sh \n\n```\n\n### Manually Run JunoDB services\n```bash\n#This can be done instead of ./start.sh to start up the docker services\n\ncd $BUILDTOP/docker/manifest\n\n# To run junodb services in --detach mode (recommended)\ndocker compose up -d\n\n# Juno proxy service listens on port \n# :5080 TLS and :8080 TCP\n\n#To view the running containers \ndocker ps\n\n# To stop junodb services\ndocker compose down\n```\n### \u003ch3 id=\"docker_secrets\"\u003eGenerate Secrets for Dev\u003c/h3\u003e\n\n\u003cbr\u003e\n\n\u003e **_NOTE:_**  secrets for TLS and Encryption can be generated for dev/testing.\n```bash \nsh $BUILDTOP/docker/manifest/config/secrets/gensecrets.sh\n\n## generated secrets\n# server.crt/server.pem - certificate/key for junodb proxy for TLS \n# ca.crt - CA cert\n# keystore.toml - sample keystore file\n```\n\n\n### \u003ch3 id=\"docker_validate_junodb\"\u003eValidate JunoDB\u003c/h3\u003e\n\nLogin to docker client and check connection with proxy\n```bash \ndocker exec -it junoclient bash -c 'nc -vz proxy 5080'\n```\n\n\n\u003cbr\u003e\n\n### You can also test the junodb server by running junocli and junoload\n\u003cbr\u003e\n\n### JunoCLI\u003cbr\u003e\nThe following commands log in to the docker client and run the ./junocli command directly. The proxy ip is aliased as \"proxy\"\n\n1. CREATE\n```bash \ndocker exec -it junoclient bash -c '/opt/juno/junocli create -s proxy:8080 -c config.toml -ns test_ns test_key test_value'\n```\n\n2. GET\n```bash \ndocker exec -it junoclient bash -c '/opt/juno/junocli get -s proxy:8080 -c config.toml -ns test_ns test_key'\n```\n\n3. UPDATE\n```bash\ndocker exec -it junoclient bash -c '/opt/juno/junocli update -s proxy:8080 -c config.toml -ns test_ns test_key test_value_updated'\n```\n\n4. DESTROY\n```bash\ndocker exec -it junoclient bash -c '/opt/juno/junocli destroy -s proxy:8080 -c config.toml -ns test_ns test_key'\n```\n\nMore about junocli [here](docs/junocli.md) \u003cbr\u003e\n\n\u003cbr\u003e\n\n### Junoload\u003cbr\u003e\n\nThe following command logs in to the docker client and runs the ./junoload command directly. Junoload can be used for benchmarking juno server. \nThe proxy ip is aliased as \"proxy\"\n\n```bash \ndocker exec -it junoclient bash -c '/opt/juno/junoload -s proxy:5080 -ssl -c config.toml -o 1'\n```\nMore about junoload [here](docs/junoload.md) \n\n\u003cbr\u003e\n\n\n\u003cbr\u003e\n\n## Manual Build\n\n**Note** : \nManual build supported platforms\n- Linux (Ubuntu 20.04)\n\nThe following sections explain the process for manually building the JunoDB server without Docker. These instructions are based on an Ubuntu 20.04.5 system\n* [Manual](#manual-build)    \n    * [Install Dependencies](#manual_install_dependencies)\n    * [Build JunoDB](#manual_build_junodb)\n    * [Run JunoDB](#manual_run_junodb)\n    * [Validate](#manual_validate_junodb)\n    * [Shutdown JunoDB](#manual_shutdown_junodb)\n\n\n### \u003ch3 id=\"manual_install_dependencies\"\u003eInstall Dependencies\u003c/h3\u003e\n\nInstall [OpenSSL 1.0.2g+](https://www.openssl.org/source/)\n```bash\nsudo apt install openssl\n```\n\n\n\nInstall [multilog](https://manpages.ubuntu.com/manpages/bionic/man8/multilog.8.html)\n\n```bash\nsudo apt install daemontools\n```\n\nInstall dependencies for rocksdb\n\n```bash\nsudo apt-get install build-essential libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev -y\n```\n\n\u003cbr\u003e\nInstall Python \n\n```bash\n#install python\nsudo apt-get install python3.8\n#set soft link\ncd /usr/bin\nsudo ln -s python3.8 python\n```\n\n\n\n\n### \u003ch3 id=\"manual_build_junodb\"\u003eBuild JunoDB\u003c/h3\u003e\n```bash\nbinary_build/build.sh\n```\n\u003cbr\u003e\n\n\n### \u003ch3 id=\"manual_run_junodb\"\u003eRun JunoDB\u003c/h3\u003e\n```bash\nexport JUNO_BUILD_DIR=$BUILDTOP/release-binary/code-build\nscript/deploy.sh\n```\n\u003cbr\u003e\n\n\n### \u003ch3 id=\"manual_validate_junodb\"\u003eValidate JunoDB\u003c/h3\u003e\n```bash\n#Validate if deploy was successful by checking if the proxy (junoserv), storage (junostorageserv), and etcd (junoclusterserv) processes are running\nps -eaf | grep juno\n#There should be 41 processes running\n#5 for junoclusterserv (3 logs, 1 etcdsvr.py, 1 etcdsvr_exe)\n#20 for junostorageserv (6 logs, 1 manager, 12 workers, 1 monitor)\n#16 for junoserv (6 logs, 1 manager, 8 workers, 1 monitor)\n```\n\u003cbr\u003e\n\n### Test out the server using junocli and junoload command\nSee instructions for junocli [here](docs/junocli.md) \u003cbr\u003e\nSee instructions for junoload [here](docs/junoload.md) \n\n### Run functional tests\n```bash\n#Assuming user is in $BUILDTOP folder\ntest/functest/configsetup.sh\ncd test/functest\n$BUILDTOP/release-binary/tool/go/bin/go test -v -config=config.toml\n```\n\u003cbr\u003e\n\n\n### Run unit tests\n```bash\n#Assuming user is in $BUILDTOP folder\ncd test/unittest\n$BUILDTOP/release-binary/tool/go/bin/go test -v\n```\n\n### \u003ch3 id=\"manual_shutdown_junodb\"\u003eShutdown JunoDB Services\u003c/h3\u003e\n```bash\n#Assuming user is in $BUILDTOP folder\nscript/deploy.sh stop\n```\n\u003cbr\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaypal%2Fjunodb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaypal%2Fjunodb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaypal%2Fjunodb/lists"}