{"id":20252617,"url":"https://github.com/streamdal/njst","last_synced_at":"2025-03-03T16:43:00.690Z","repository":{"id":39831159,"uuid":"474538364","full_name":"streamdal/njst","owner":"streamdal","description":"Distributed benchmark tool for NATS Jetstream","archived":false,"fork":false,"pushed_at":"2022-05-25T20:07:38.000Z","size":1797,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-01-14T03:10:53.700Z","etag":null,"topics":["benchmark","kubernetes","load-testing","nats"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/streamdal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-27T04:54:48.000Z","updated_at":"2023-02-14T17:25:56.000Z","dependencies_parsed_at":"2022-09-23T00:11:25.381Z","dependency_job_id":null,"html_url":"https://github.com/streamdal/njst","commit_stats":null,"previous_names":["batchcorp/njst"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdal%2Fnjst","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdal%2Fnjst/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdal%2Fnjst/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamdal%2Fnjst/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/streamdal","download_url":"https://codeload.github.com/streamdal/njst/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241703577,"owners_count":20006208,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","kubernetes","load-testing","nats"],"created_at":"2024-11-14T10:17:38.051Z","updated_at":"2025-03-03T16:43:00.650Z","avatar_url":"https://github.com/streamdal.png","language":"Go","readme":"# njst\n\n`njst` is a [NATS JetStream](https://docs.nats.io/nats-concepts/jetstream) \n_distributed_ benchmark and testing tool.\n\n[Batch.sh](https://batch.sh) makes significant use of NATS JetStream and we use\nthis tool to run periodic benchmarks on our internal NATS JS clusters.\n\nWhile `njst` supports read and write testing, it is primarily geared towards\ntesting reads with durable consumers.\n\n**NOTE: While benchmarks are dumb, this will at least give you a _general_ idea \nabout the performance capabilities of your NATS cluster.**\n\n**NOTE 2: You might see some discrepancy in performance between `njst` and `nats bench`.\nWhile slight deviation might be there due to `njst` missing some optimizations,\nthe results should be close. If they are not, ensure that you are passing `--pull`\nto `nats bench` so that it uses durable pull consumers instead of ordered push\nconsumers (which are much faster but do not require ACKs).**\n\n## Features\n\n* Built *specifically* for testing JetStream\n* Distributed by default\n* No leader, no followers\n* Cloud native - works best in k8s\n* Simple [HTTP REST'ish API](./docs/api.md) for job control\n* Ability to perform *massively parallel* tests to (attempt to) simulate real-world stress\n* Uses _only_ durable pull consumers for reads\n* Multi-consumer, multi-worker workloads with support for `FilterSubject`\n\n## Usage\n\n`njst` uses NATS internally to keep track of `njst` nodes and results. For this\nreason, you will need to bring up a NATS instance in your env and point `njst`\nnode(s) to it via env vars or flags.\n\n### Local\n\n1. Bring up a NATS instance via docker-compose\n   1. `docker-compose up -d`\n2. Run 2 instances of `njst`\n   1. Open terminal one: `go run main.go --debug --http-address=:5000`\n   2. Open terminal two: `go run main.go --debug --http-address=:5001`\n3. Verify that both `njst` nodes are in the cluster: \n   ```bash\n      ❯ curl --request GET --url http://localhost:5000/bench/cluster\n      {\"nodes\":[\"489e8fd7\",\"6ebcb9bb\"],\"count\":2}\n   ```\n4. Perform a test\n   ```bash\n   curl -X POST -L \\\n    --url http://localhost:5000/bench/ \\\n    --header 'Content-Type: application/json' \\\n    --data '{\n       \"description\": \"128\",\n       \"nats\": {\n         \"address\": \"127.0.0.1\",\n         \"shared_connection\": false\n       },\n       \"write\": {\n         \"num_nodes\": 1,\n         \"num_streams\": 1,\n         \"num_messages_per_stream\": 1000,\n         \"num_workers_per_stream\": 1,\n         \"msg_size_bytes\": 24,\n         \"keep_streams\": false,\n         \"batch_size\": 100,\n         \"storage\": \"memory\",\n         \"subjects\": [\"1\", \"2\"]\n       }\n    }'\n   {\"id\":\"srOqCKmq\",\"message\":\"benchmark created successfully; created 1 jobs\"}\n   ```\n5. Grab the returned id and check its status\n    ```bash\n    ❯ curl -s http://localhost:5000/bench/srOqCKmq | jq\n    {\n      \"status\": {\n        \"status\": \"completed\",\n        \"message\": \"benchmark completed; final\",\n        \"job_id\": \"srOqCKmq\",\n        \"elapsed_seconds\": 0.05,\n        \"avg_msg_per_sec_per_node\": 18142.89,\n        \"total_msg_per_sec_all_nodes\": 18142.89,\n        \"total_processed\": 1000,\n        \"total_errors\": 0,\n        \"started_at\": \"2022-05-25T04:24:23.498141Z\",\n        \"ended_at\": \"2022-05-25T04:24:23.553259Z\"\n      },\n      \"settings\": {\n        \"description\": \"128\",\n        \"nats\": {\n          \"address\": \"127.0.0.1\",\n          \"shared_connection\": false\n        },\n        \"write\": {\n          \"num_streams\": 1,\n          \"num_nodes\": 1,\n          \"num_messages_per_stream\": 1000,\n          \"num_workers_per_stream\": 1,\n          \"subjects\": [\n            \"1\",\n            \"2\"\n          ],\n          \"num_replicas\": 0,\n          \"batch_size\": 100,\n          \"msg_size_bytes\": 24,\n          \"keep_streams\": false,\n          \"storage\": \"memory\"\n        },\n        \"id\": \"srOqCKmq\"\n      }\n    }\n    ```\n\n### Production\n\n1. Modify and use the following [k8s deploy config](./deploy.dev.yaml) to deploy\n   3+ `nsjt` instance(s) to your kubernetes cluster.\n    1. Update NATS env vars to point to your NATS cluster\n    2. `kubectl apply -f deploy.dev.yaml`\n2. Talk to any of the `njst` nodes via the [HTTP API](docs/api.md) to manage jobs\n\nNOTE: To do anything useful with `njst`, you'll want to be able to talk to its API.\nSince there's no service or ingres defined, you'll probably want to do a port-forward\nto one of the pods via `kubectl`:\n   * `kubectl port-forward njst-deployment-78d7b584cd-5sf9c 5000:5000`\n\n## Why?\n\nWhen evaluating a new technology such as a message bus/queue, in addition to\nsingle node read/write performance, you'll probably want to also perform \n\"real-world\"-like tests to see how your cluster behaves.\n\nNATS Jetstream is _amazing_ but aside from using `nats bench` to perform basic\nread/write tests, we wanted to get insight into how our cluster will perform\nunder more realistic load (without updating all applications).\n\nWe created `njst` to do this. Maybe you'll find it useful too.\n\n## You Should Know\n\n`njst` is \"dumb-distributed\". Meaning, there are no concepts of leaders or\nfollowers, there is no \"election\" and there is no consensus. \n\nIn other words:\n\n* When you submit a new job, the job will only be ran by the cluster members\nthat are currently connected to the cluster \n  * If new members join the cluster, they will have no idea about the pre-existing\n  jobs.\n\n* There is no auth - we have no need for it. If you want it, feel free to add it.\n\n## Sample Jobs \u0026 Results\n\n### Heavy Write Test\n\n* This will cause 16 streams to be divided across 3 nodes\n* Each stream will have 4 concurrent writers launched\n* Each concurrent writer will write 25,000 messages to a stream\n* Each message will contain 4096 random bytes (generated once at test creation time)\n* `keep_streams\" tells the test to NOT delete the generated streams and their contents\n  * **We want this so that we can use the generated content for a future read test**\n* Result: you will write 1,600,000 messages in total (16 streams x 100,000 messages)\n\n```json\n{\n\t\"description\": \"heavy\",\n    \"nats\": {\n        \"address\": \"127.0.0.1\",\n        \"connection_per_stream\": false\n    },\n\t\"write\": {\n\t\t\"num_nodes\": 3,\n\t\t\"num_streams\": 16,\n\t\t\"num_messages_per_stream\": 100000,\n\t\t\"num_workers_per_stream\": 4,\n\t\t\"msg_size_bytes\": 4096,\n\t\t\"keep_streams\": true\n\t}\n}\n```\n\n### Heavy Read Test\n\n* Spread reading from 16 streams across 3 nodes\n  * 2 nodes will handle 5 streams, 1 node will handle 6 streams\n* Each stream will be read by 4 concurrent workers\n* Batching is *highly* important: each worker will read 1,000 messages at a time\n\n```json\n{\n\t\"description\": \"heavy read test\",\n    \"nats\": {\n        \"address\": \"127.0.0.1\",\n        \"connection_per_stream\": false\n    },\n\t\"read\": {\n\t\t\"write_id\": \"i1mSzcm8\", \u003c--- write id from previous write test\n\t\t\"num_nodes\": 3,\n\t\t\"num_streams\": 16,\n\t\t\"num_messages_per_stream\": 100000,\n\t\t\"num_workers_per_stream\": 4,\n\t\t\"batch_size\": 1000\n\t}\n}\n```\n\n### Results\n\n**NOTE**: These are dummy output values. For our own benchmark results, see [docs/benchmarks.md](./docs/benchmarks.md).\n\n```json\n{\n\t\"status\": {\n\t\t\"status\": \"completed\",\n\t\t\"message\": \"benchmark completed; final\",\n\t\t\"job_id\": \"dyczUnmQ\",\n\t\t\"node_id\": \"1efe113e\",\n\t\t\"elapsed_seconds\": 245,\n\t\t\"avg_msg_per_sec_per_node\": 2176.67,\n\t\t\"avg_msg_per_sec_all_nodes\": 6530.61,\n\t\t\"total_processed\": 1600000,\n\t\t\"total_errors\": 0,\n\t\t\"started_at\": \"2022-04-13T14:18:22.483853-07:00\",\n\t\t\"ended_at\": \"0001-01-01T00:00:00Z\"\n\t},\n\t\"settings\": {\n\t\t\"description\": \"medium read test with workermap\",\n        \"nats\": {\n            \"address\": \"127.0.0.1\",\n            \"connection_per_stream\": false\n         },\n\t\t\"read\": {\n\t\t\t\"write_id\": \"i1mSzcm8\",\n\t\t\t\"num_streams\": 16,\n\t\t\t\"num_nodes\": 3,\n\t\t\t\"num_messages_per_stream\": 100000,\n\t\t\t\"num_workers_per_stream\": 4,\n\t\t\t\"batch_size\": 1000\n\t\t},\n\t\t\"id\": \"dyczUnmQ\"\n\t}\n}\n```\n\n\n## Internal Flow\n\nThe general job flow can visualized via this confusing mermaid diagram:\n\n```mermaid\ngraph TD;\n    A(Create new test job)--1. HTTP-API--\u003eB(njst cluster);\n    B--2. Emit new job message--\u003eNATS\n    N1(njst nodes)--3. Get new job message --\u003eNATS\n    N1--4. Start job--\u003eResults\n    Results--5. Send results back--\u003eNATS   \n    NATS--6. Here are the results--\u003eB\n```\n \n1. Client talks to any `njst` node via HTTP API to create a new test job\n2. `njst` node emits a new job message to the cluster (via NATS)\n3. All `njst` nodes get the new job message\n4. All `njst` nodes start the job\n5. All `njst` nodes send their results back via NATS KV\n6. All `njst` nodes listen for result completions in result KV and analyze the result\n7. Any `njst` node can now respond to a \"status\" HTTP call for a specific job\n\n## Our Benchmarks\n\nYou can read our benchmark results [here](./docs/benchmarks.md).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamdal%2Fnjst","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstreamdal%2Fnjst","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamdal%2Fnjst/lists"}