{"id":24010654,"url":"https://github.com/sonicfromnewyoke/solana-rpc","last_synced_at":"2025-05-05T16:26:33.371Z","repository":{"id":271130846,"uuid":"912480914","full_name":"sonicfromnewyoke/solana-rpc","owner":"sonicfromnewyoke","description":"Configure a slightly more performant Solana RPC than a regular one","archived":false,"fork":false,"pushed_at":"2025-04-05T15:14:44.000Z","size":38,"stargazers_count":48,"open_issues_count":0,"forks_count":9,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-26T15:08:07.401Z","etag":null,"topics":["rpc","solana"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sonicfromnewyoke.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-05T17:33:48.000Z","updated_at":"2025-04-26T05:26:26.000Z","dependencies_parsed_at":"2025-01-28T14:41:40.165Z","dependency_job_id":null,"html_url":"https://github.com/sonicfromnewyoke/solana-rpc","commit_stats":null,"previous_names":["sonicfromnewyoke/solana-rpc"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonicfromnewyoke%2Fsolana-rpc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonicfromnewyoke%2Fsolana-rpc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonicfromnewyoke%2Fsolana-rpc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonicfromnewyoke%2Fsolana-rpc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sonicfromnewyoke","download_url":"https://codeload.github.com/sonicfromnewyoke/solana-rpc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252532950,"owners_count":21763500,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rpc","solana"],"created_at":"2025-01-08T04:15:50.512Z","updated_at":"2025-05-05T16:26:33.360Z","avatar_url":"https://github.com/sonicfromnewyoke.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003eSolana RPC\u003c/h1\u003e\n\n\u003e [!NOTE]\n\u003e This repo contains steps to configure a **slightly** more performant RPC than a **regular** one\n\n## 🤖 Server\n\n### Requirements\n\n[Solana Hardware Compatibility List](https://solanahcl.org/) can be found here\n\n- **CPU**\n  - `2.8GHz` base clock speed, or faster\n  - `16` cores / `32` threads, or more\n  - SHA extensions instruction support\n  - AMD Gen 3 or newer\n  - Intel Ice Lake or newer\n  - Higher clock speed is preferable over more cores\n  - AVX2 instruction support (to use official release binaries, self-compile otherwise)\n  - Support for AVX512f is helpful\n- **RAM** \n  - `256GB` or more (for account indexes)\n  - Error Correction Code (ECC) memory is suggested\n  - Motherboard with 512GB capacity suggested\n- **Disk**\n  - **Accounts:** `500GB`, or larger. High TBW (Total Bytes Written)\n  - **Ledger:** `1TB` or larger. High TBW suggested\n  - **Snapshots:** `250GB` or larger. High TBW suggested\n  - **OS:** (Optional) `500GB`, or larger. SATA OK\n\nConsider a larger ledger disk if longer transaction history is required\n\n\u003e [!CAUTION]\n\u003e Accounts and ledger **should not be** stored on the same disk\n\n### Sol User\n\nCreate a new Ubuntu user, named `sol`, for running the validator:\n\n```bash\nsudo adduser sol\n```\n\nIt is a best practice to always run your validator as a non-root user, like the `sol` user we just created.\n\n### Ports Opening\n\nNote: RPC port remains closed, only SSH and gossip ports are opened.\n\nFor new machines with UFW disabled:\n\n1. Add OpenSSH first to prevent lockout if you don't have password access\n2. Open required ports\n3. Close RPC port\n\n```bash\nsudo ufw enable\n\nsudo ufw allow 22/tcp\nsudo ufw allow 8000:8020/tcp\nsudo ufw allow 8000:8020/udp\n\nsudo ufw deny 8899/tcp\nsudo ufw deny 8899/udp\n\nsudo ufw reload\n```\n\n### Hard Drive Setup\n\nOn your Ubuntu computer make sure that you have at least `2TB` of disk space mounted. You can check disk space using the `df` command:\n\n```bash\ndf -h\n```\n\nTo see the hard disk devices that you have available, use the list block devices command:\n\n```bash\nlsblk -f\n```\n\n#### Drive Formatting: Ledger\n\nAssuming you have an nvme drive that is not formatted, you will have to format the drive and then mount it.\n\nFor example, if your computer has a device located at `/dev/nvme0n1`, then you can format the drive with the command:\n\n```bash\nsudo mkfs -t xfs /dev/nvme0n1\n```\nFor your computer, the device name and location may be different.\n\nNext, check that you now have a UUID for that device:\n\n```bash\nlsblk -f\n```\n\nIn the fourth column, next to your device name, you should see a string of letters and numbers that look like this: `6abd1aa5-8422-4b18-8058-11f821fd3967`. That is the UUID for the device\n\n#### Mounting Your Drive: Ledger\n\nSo far we have created a formatted drive, but you do not have access to it until you mount it. Make a directory for mounting your drive:\n\n```bash\nsudo mkdir -p /mnt/ledger\n```\n\nNext, change the ownership of the directory to your sol user:\n\n```bash\nsudo chown -R sol:sol /mnt/ledger\n```\n\nNow you can mount the drive:\n\n```bash\nsudo mount -o noatime /dev/nvme0n1 /mnt/ledger\n```\n\n#### Formatting And Mounting Drive: AccountsDB\n\nYou will also want to mount the accounts db on a separate hard drive. The process will be similar to the ledger example above.\n\nAssuming you have device at `/dev/nvme1n1`, format the device and verify it exists:\n\n```bash\nsudo mkfs -t xfs /dev/nvme1n1\n```\n\nThen verify the UUID for the device exists:\n\n```bash\nlsblk -f\n```\n\nCreate a directory for mounting:\n\n```bash\nsudo mkdir -p /mnt/accounts\n```\n\nChange the ownership of that directory:\n\n```bash\nsudo chown -R sol:sol /mnt/accounts\n```\n\nAnd lastly, mount the drive:\n\n```bash\nsudo mount -o noatime  /dev/nvme1n1 /mnt/accounts\n```\n\n\n#### Modify fstab\n\n```bash\nsudo vi /etc/fstab\n```\n\nshould contains something similar\n\n```\n# /etc/fstab: static file system information.\n#\n# Use 'blkid' to print the universally unique identifier for a\n# device; this may be used with UUID= as a more robust way to name devices\n# that works even if disks are added and removed. See fstab(5).\n#\n# \u003cfile system\u003e \u003cmount point\u003e   \u003ctype\u003e  \u003coptions\u003e       \u003cdump\u003e  \u003cpass\u003e\n\n...\n\nUUID=\"03477038-6fa7-4de3-9ca6-4b0aef52bf42\" /mnt/ledger xfs defaults,noatime 0 2\nUUID=\"68ff3738-f9f7-4423-a24c-68d989a2e496\" /mnt/accounts xfs defaults,noatime 0 2\n```\n\n## 🌵 Agave Client\n\n### Install rustc, cargo and rustfmt\n\n```bash\ncurl https://sh.rustup.rs -sSf | sh\nsource $HOME/.cargo/env\nrustup component add rustfmt\n```\n\nWhen building the master branch, please make sure you are using the latest stable rust version by running:\n\n```bash\nrustup update\nrustup default nightly\nrustup override set nightly\n```\n\nyou may need to install libssl-dev, pkg-config, zlib1g-dev, protobuf etc.\n\n```bash\nsudo apt-get update\nsudo apt-get install libssl-dev libudev-dev pkg-config zlib1g-dev llvm clang cmake make libprotobuf-dev protobuf-compiler lld\n```\n\n### Download the source code\n\n```bash\ngit clone https://github.com/anza-xyz/agave.git\ncd agave\n\n# let's asume we would like to build v2.18 of validator\nexport TAG=\"v2.1.18\" \ngit switch tags/$TAG --detach\n```\n\n### Add `profile.release-with-lto`\n\n```bash\nvi Cargo.toml\n```\n\n```diff\n+[profile.release-with-lto]\n+inherits = \"release\"\n+lto = \"fat\"\n+codegen-units = 1\n```\n\n```bash\nvi ./scripts/cargo-install-all.sh\n```\n\n```diff\nbuildProfileArg='--profile release-with-debug'\n      buildProfile='release-with-debug'\n      shift\n+elif [[ $1 = --release-with-lto ]]; then\n+      buildProfileArg='--profile release-with-lto'\n+      buildProfile='release-with-lto'\n+      shift\nelif [[ $1 = --validator-only ]]; then\n      validatorOnly=true\n      shift\n```\n\n### Build\n\n```bash\nexport RUSTFLAGS=\"-Clink-arg=-fuse-ld=lld -Ctarget-cpu=native\"\n./scripts/cargo-install-all.sh  --release-with-lto --validator-only .\nexport PATH=$PWD/bin:$PATH\n```\n\n## 🦾 System Tuning\n\n### Set CPU Governor to `Performance`\n\n```bash\necho \"performance\" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor\n```\n\n### Disable THP (Transparent Huge Pages)\n\nLinux transparent huge page (THP) support allows the kernel to automatically promote regular memory pages into huge pages.\nHuge pages reduces TLB pressure, but THP support introduces latency spikes when pages are promoted into huge pages and when memory compaction is triggered.\n\n```bash\necho never \u003e /sys/kernel/mm/transparent_hugepage/enabled\n```\n\n### Disable KSM (Kernel Samepage Merging)\n\nLinux kernel samepage merging (KSM) is a feature that de-duplicates memory pages that contains identical data.\nThe merging process needs to lock the page tables and issue TLB shootdowns, leading to unpredictable memory access latencies.\nKSM only operates on memory pages that has been opted in to samepage merging using `madvise(...MADV_MERGEABLE)`.\nIf needed KSM can be disabled system wide by running the following command:\n\n```bash\necho 0 \u003e /sys/kernel/mm/ksm/run\n```\n\n### Disable automatic NUMA memory balancing\n\nLinux supports automatic page fault based NUMA memory balancing and manual page migration of memory between NUMA nodes.\nMigration of memory pages between NUMA nodes will cause TLB shootdowns and page faults for applications using the affected memory\n\n```bash\necho 0 \u003e /proc/sys/kernel/numa_balancing\n```\n\n### Huge and Gigantic Pages\n\n```bash\nsudo mkdir -p /mnt/hugepages\nsudo mount -t hugetlbfs -o pagesize=2097152,min_size=228589568 none /mnt/hugepages\n\nsudo mkdir -p /mnt/gigantic\nsudo mount -t hugetlbfs -o pagesize=1073741824,min_size=27917287424 none /mnt/gigantic\n\ncat /proc/mounts | grep hugetlbfs\n# none /mnt/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M,min_size=95124124 0 0\n# none /mnt/gigantic hugetlbfs rw,seclabel,relatime,pagesize=1024M,min_size=540092137472 0 0\n```\n\n### Sysctl Tuning\n\n```bash\nsudo bash -c \"cat \u003e/etc/sysctl.d/21-agave-validator.conf \u003c\u003cEOF\n# TCP Buffer Sizes (10k min, 87.38k default, 12M max)\nnet.ipv4.tcp_rmem=10240 87380 12582912\nnet.ipv4.tcp_wmem=10240 87380 12582912\n\n# Increase UDP buffer sizes\nnet.core.rmem_default = 134217728\nnet.core.rmem_max = 134217728\nnet.core.wmem_default = 134217728\nnet.core.wmem_max = 134217728\n\n# TCP Optimization\nnet.ipv4.tcp_congestion_control=westwood\nnet.ipv4.tcp_fastopen=3\nnet.ipv4.tcp_timestamps=0\nnet.ipv4.tcp_sack=1\nnet.ipv4.tcp_low_latency=1\nnet.ipv4.tcp_tw_reuse=1\nnet.ipv4.tcp_no_metrics_save=1\nnet.ipv4.tcp_moderate_rcvbuf=1\n\n# Kernel Optimization\nkernel.timer_migration=0\nkernel.hung_task_timeout_secs=30\nkernel.pid_max=49152\n\n# Virtual Memory Tuning\nvm.swappiness=0\nvm.max_map_count = 2000000\nvm.stat_interval=10\nvm.dirty_ratio=40\nvm.dirty_background_ratio=10\nvm.min_free_kbytes=3000000\nvm.dirty_expire_centisecs=36000\nvm.dirty_writeback_centisecs=3000\nvm.dirtytime_expire_seconds=43200\n\n# Increase number of allowed open file descriptors\nfs.nr_open = 2000000\n\n# Increase Sync interval (default is 3000)\nfs.xfs.xfssyncd_centisecs = 10000\nEOF\"\n```\n\n```bash\nsudo sysctl -p /etc/sysctl.d/21-agave-validator.conf\n```\n\n### Increase systemd and session file limits\n\nAdd\n\n```ini\nLimitNOFILE=2000000\n```\nto the `[Service]` section of your systemd service file, if you use one, otherwise add\n\n```ini\nDefaultLimitNOFILE=2000000\n```\n\nto the `[Manager]` section of `/etc/systemd/system.conf`.\n\n```bash\nsudo systemctl daemon-reload\n```\n\n```bash\nsudo bash -c \"cat \u003e/etc/security/limits.d/90-solana-nofiles.conf \u003c\u003cEOF\n# Increase process file descriptor count limit\n* - nofile 2000000\nEOF\"\n```\n\n### Isolate one core for PoH\n\n\u003e [!NOTE]\n\u003e Many thanks for that guide to [ax | 1000x.sh](https://discord.com/channels/428295358100013066/811317327609856081/1257995317190852651)\n\nfind out the nearest available core. in most cases, it's core 2 (cores 0 and 1 are often used by the kernel). if you have more cores, you can choose another available nearest core.\n\n#### Know your topology\n\n```bash\nlstopo\n```\n\ncheck your cores and hyperthreads\nlook at the \"cores\" table to find your core and its hyperthread. for example, if you choose core 2, its hyperthread might be 26 (in my case)\n\n```bash\nlscpu --all -e\n```\n\nthe easiest way to find the hyperthread for eg core 2:\n\n```bash\ncat /sys/devices/system/cpu/cpu2/topology/thread_siblings_list\n```\n\n#### Isolate the core and its hyperthread\n\nin my case the hyperthread for core 2 is 26\n`/etc/default/grub` (dont forget to run update-grub and reboot afterwards) \n\n```\nGRUB_CMDLINE_LINUX_DEFAULT=\"quiet amd_pstate=passive nvme_core.default_ps_max_latency_us=0 nohz_full=2,26 isolcpus=domain,managed_irq,2,26 irqaffinity=0-1,3-25,27-47\"\n```\n\n```sh\nupdate-grub\n```\n\n\u003e [!NOTE]\n\u003e - `nohz_full=2,26` enables full dynamic ticks for core 2 and its hyperthread 26 to reducing overhead and latency.\n\u003e - `isolcpus=domain,managed_irq,2,26` isolates core 2 and hyperthread 26 from the general scheduler\n\u003e - `irqaffinity=0-1,3-25,27-47` directs interrupts away from core 2 and hyperthread 26 \n\n#### Set the poh thread to core 2\n\nadd the cli to your validator\n\n```\n...\n--experimental-poh-pinned-cpu-core 2 \\ \n...\n```\n\nthere is a bug with core_affinity if you isolate your cores: [link](https://github.com/anza-xyz/agave/issues/1968)\n\nyou can take the next bash script to identify the pid of solpohtickprod and set it to eg. core 2\n\n```bash\n#!/bin/bash\n\n# main pid of agave-validator\nagave_pid=$(pgrep -f \"^agave-validator --identity\")\nif [ -z \"$agave_pid\" ]; then\n    logger \"set_affinity: agave_validator_404\"\n    exit 1\nfi\n\n# find thread id\nthread_pid=$(ps -T -p $agave_pid -o spid,comm | grep 'solPohTickProd' | awk '{print $1}')\nif [ -z \"$thread_pid\" ]; then\n    logger \"set_affinity: solPohTickProd_404\"\n    exit 1\nfi\n\ncurrent_affinity=$(taskset -cp $thread_pid 2\u003e\u00261 | awk '{print $NF}')\nif [ \"$current_affinity\" == \"2\" ]; then\n    logger \"set_affinity: solPohTickProd_already_set\"\n    exit 1\nelse\n    # set poh to cpu2\n    sudo taskset -cp 2 $thread_pid\n    logger \"set_affinity: set_done\"\n     # $thread_pid\nfi\n```\n\n## 🚀 Validator Startup\n\n### Options\n\n```bash\n\n```\n\n### Let's use the common one\n\n```bash\nmkdir -p /home/sol/bin\n```\n\nNext, open the [`validator.sh`](./bin/validator.sh) file for editing:\n\n```bash\nvi /home/sol/bin/validator.sh\n```\nThen\n\n```bash\nchmod +x /home/sol/bin/validator.sh\n```\n\n### Create a System Service\n\nYou can use the [`sol.service`](./system/sol.service) from this repo or `sudo vi /etc/systemd/system/sol.service` and paste\n\n```ini\n[Unit]\nDescription=Solana Validator\nAfter=network.target\nStartLimitIntervalSec=0\n\n[Service]\nType=simple\nRestart=always\nRestartSec=1\nLimitNOFILE=2000000\nLogRateLimitIntervalSec=0\nUser=sol\nEnvironment=\"PATH=/home/sol/agave/bin\"\nEnvironment=SOLANA_METRICS_CONFIG=host=https://metrics.solana.com:8086,db=mainnet-beta,u=mainnet-beta_write,p=password\nExecStart=/home/sol/bin/validator.sh\n\n[Install]\nWantedBy=multi-user.target\n```\n\n### Setup log rotation\n\nYou can use the [`logrotate.sol`](./logrotate.sol) from this repo run the next commands\n\n```bash\ncat \u003e logrotate.sol \u003c\u003cEOF\n/home/sol/agave-validator.log {\n  rotate 7\n  daily\n  missingok\n  postrotate\n    systemctl kill -s USR1 sol.service\n  endscript\n}\nEOF\n\nsudo cp logrotate.sol /etc/logrotate.d/sol\nsystemctl restart logrotate.service\n```\n\nThe validator log file, as specified by `--log ~/agave-validator.log`, can get very large over time and it's recommended that log rotation be configured.\n\nThe validator will re-open its log file when it receives the `USR1` signal, which is the basic primitive that enables log rotation.\n\nIf the validator is being started by a wrapper shell script, it is important to launch the process with `exec` (`exec agave-validator ...`) when using logrotate. This will prevent the `USR1` signal from being sent to the script's process instead of the validator's, which will kill them both.\n\n### Enable and start System Service\n\n```bash\nsudo systemctl enable --now sol\n```\n\n```bash\nsudo systemctl status sol.service\n```\n\n### Setup Shredstream Proxy\n\nJito’s ShredStream service delivers the lowest latency shreds from leaders on the Solana network, optimizing performance for high-frequency trading, validation, and RPC operations.\nShredStream ensures minimal latency for receiving shreds, which can save hundreds of milliseconds during trading on Solana—a critical advantage in high-frequency trading environments.\nAdditionally, it provides a redundant shred path for servers in remote locations, enhancing reliability and performance for users operating in less connected regions.\nThis makes ShredStream particularly valuable for traders, validators, and node operators who require the fastest and most reliable data to maintain a competitive edge.\n\n#### Download the source code\n\n```bash\ngit clone https://github.com/jito-labs/shredstream-proxy.git --recurse-submodules\ncd shredstream-proxy\n```\n\n#### Build \n\n```bash\nexport RUSTFLAGS=\"-Clink-arg=-fuse-ld=lld -Ctarget-cpu=native\"\ncargo build --release --bin jito-shredstream-proxy\n\nmkdir ./bin\ncp target/release/jito-shredstream-proxy ./bin/jito-shredstream-proxy\nexport PATH=$PWD/bin:$PATH\n```\n\n#### Preparation\n\n1. Get your Solana [public key approved](https://web.miniextensions.com/WV3gZjFwqNqITsMufIEp)\n2. Ensure your firewall is open.\n  - Default port for incoming shreds is **20000/udp.**\n  - NAT connections currently not supported.\n  - If you use a firewall, see the firewall [configuration section](https://docs.jito.wtf/lowlatencytxnfeed/#firewall-configuration)\n3. Find your TVU port\n\n```bash\nexport LEDGER_DIR=/mnt/ledger; bash -c \"$(curl -fsSL https://raw.githubusercontent.com/jito-labs/shredstream-proxy/master/scripts/get_tvu_port.sh)\"\n```\n\n\nNext, open the [`shredstream.sh`](./bin/shredstream.sh) file for editing:\n\n```bash\nvi /home/sol/bin/shredstream.sh\n```\nThen\n\n```bash\nchmod +x /home/sol/bin/shredstream.sh\n```\n\n#### Create a System Service\n\nYou can use the [`shredstream.service`](./system/shredstream.service) from this repo or `sudo vi /etc/systemd/system/shredstream.service` and paste\n\n```ini\n[Unit]\nDescription=Shredstream Proxy\nAfter=network.target\nStartLimitIntervalSec=0\n\n[Service]\nType=simple\nRestart=always\nRestartSec=1\nLogRateLimitIntervalSec=0\nUser=sol\nEnvironment=\"PATH=/home/sol/shredstream-proxy/bin\"\nEnvironment=\"RUST_LOG=info\"\nExecStart=/home/sol/bin/shredstream.sh\n\n[Install]\nWantedBy=multi-user.target\n```\n\n#### Enable and start System Service\n\n```bash\nsudo systemctl enable --now shredstream\n```\n\n```bash\nsudo systemctl status shredstream.service\n```\n\n## 📝 Cheat Sheet\n\n| Command                          | Description                         |\n| -------------------------------- | ----------------------------------- |\n| `tail -f ~/agave-validator.log`  | follow the logs of the Agave node   |\n| `agave-validator monitor`        | monitor the validator               |\n| `solana validators`              | check list of validators            |\n| `solana catchup --our-localhost` | check how far it is from chain head |\n| `btop`                           | monitor server resources            |\n| `iostat -mdx`                    | check NVMe utilization              |\n\n## 🎉 Credits\n\n- [Anza](https://docs.anza.xyz/operations/setup-an-rpc-node)\n- [1000x.sh](https://1000x.sh)\n- [StakeWare](https://www.stakeware.xyz)\n- [Jito](https://docs.jito.wtf/lowlatencytxnfeed/)\n\n## 🔗 Links\n\n- [XFS Performance](https://wiki.archlinux.org/title/XFS#Performance)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsonicfromnewyoke%2Fsolana-rpc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsonicfromnewyoke%2Fsolana-rpc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsonicfromnewyoke%2Fsolana-rpc/lists"}