{"id":13706030,"url":"https://github.com/manifold-inc/targon","last_synced_at":"2025-05-05T19:34:11.120Z","repository":{"id":198547814,"uuid":"697482210","full_name":"manifold-inc/targon","owner":"manifold-inc","description":"A library for building subnets with the manifold reward stack","archived":false,"fork":false,"pushed_at":"2025-04-28T20:27:34.000Z","size":53837,"stargazers_count":44,"open_issues_count":0,"forks_count":43,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-28T21:39:05.681Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/manifold-inc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-09-27T20:23:15.000Z","updated_at":"2025-04-28T20:27:38.000Z","dependencies_parsed_at":"2023-12-28T00:02:57.975Z","dependency_job_id":"0a33b4e7-baec-44c3-8d18-c31553a5197c","html_url":"https://github.com/manifold-inc/targon","commit_stats":null,"previous_names":["manifold-inc/targon"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manifold-inc%2Ftargon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manifold-inc%2Ftargon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manifold-inc%2Ftargon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manifold-inc%2Ftargon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/manifold-inc","download_url":"https://codeload.github.com/manifold-inc/targon/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252563150,"owners_count":21768411,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T22:00:51.514Z","updated_at":"2025-05-05T19:34:06.111Z","avatar_url":"https://github.com/manifold-inc.png","language":"Go","funding_links":[],"categories":["Registered Subnets"],"sub_categories":[],"readme":"# Targon: A Deterministic Verification of Large Language Models\n\nTargon (Bittensor Subnet 4) is a deterministic verification mechanism that is\nused to incentivize miners to run openai compliant endpoints and serve synthetic\nand organic queries.\n\nNOTICE: Using this software, you must agree to the Terms and Agreements provided\nin the terms and conditions document. By downloading and running this software,\nyou implicitly agree to these terms and conditions.\n\n# Table of Contents\n\n1. [Compute Requirements](#recommended-compute-requirements)\n1. [Installation](#installation)\n   - [Install PM2](#install-pm2)\n   - [Install Targon](#install-targon-on-your-machine)\n1. [How to Run Targon](#how-to-run-targon)\n   - [Running VLLM](#vllm)\n   - [Running a Miner](#running-a-miner)\n   - [Running a Validator](#running-a-validator)\n1. [What is a Deterministic Verification Network?](#what-is-a-deterministic-verification-network)\n   - [Role of a Miner](#role-of-a-miner)\n   - [Role of a Validator](#role-of-a-validator)\n1. [Features of Targon](#features-of-targon)\n   - [Full OpenAI Compliance](#full-openai-compliance)\n   - [Targon-Hub](#targon-hub)\n1. [How to Contribute](#how-to-contribute)\n\n# Recommended Compute Requirements\n\nFor validators we recommend a 8xA100, although a 1xA100 could also be used. We\nplan on focusing on bringing these costs down in the coming updates.\n\nFor miners, A100 or H100s are common choices. Benchmarking is up to the miner to\ndetermine what GPU works best for their optimizations.\n\n#### Minimum Viable Compute Recommendations\n\n- **VRAM:** 80 GB\n- **Storage:** 200 GB\n- **RAM:** 16 GB\n- **CPU**: 4 core\n\n# Installation\n\n## Overview\n\nIn order to run Targon, you will need to install PM2 and the Targon package. The\nfollowing instructions apply only to Ubuntu OSes. For your specific OS, please\nrefer to the official documentation.\n\n### Install PM2 on your machine\n\n#### Download NVM\n\nTo install or update nvm, you should run the install script. To do that, you may\neither download and run the script manually, or use the following cURL or Wget\ncommand:\n\n```bash\ncurl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash\n```\n\n#### Add NVM to bash profile\n\nRunning either of the above commands downloads a script and runs it. The script\nclones the nvm repository to ~/.nvm, and attempts to add the source lines from\nthe snippet below to the correct profile file (~/.bash_profile, ~/.zshrc,\n~/.profile, or ~/.bashrc).\n\n```bash\nexport NVM_DIR=\"$([ -z \"${XDG_CONFIG_HOME-}\" ] \u0026\u0026 printf %s \"${HOME}/.nvm\" || printf %s \"${XDG_CONFIG_HOME}/nvm\")\"\n[ -s \"$NVM_DIR/nvm.sh\" ] \u0026\u0026 \\. \"$NVM_DIR/nvm.sh\" # This loads nvm\n```\n\n#### Install Node\n\n```bash\nnvm install node\n```\n\n#### Install PM2\n\n```bash\nnpm install pm2@latest -g\n```\n\nYou have now installed PM2.\n\n### Install Targon on your machine\n\n#### Clone the repository\n\n```bash\ngit clone https://github.com/manifold-inc/targon.git\ncd targon\n```\n\n#### Install dependencies\n\n```bash\npython3 -m pip install -e .\n```\n\nYou have now installed Targon. You can now run a validator or a miner.\n\n# How to Run Targon\n\n## Running a Miner\n\nBefore starting or registering your miner in Targon, first you will want to run\nVLLM serving different images validators are requesting. You can find a list at\nhttps://stats.sybil.com/stats/validator under the live tab. The more models you\nrun, the higher your incentive.\n\nVLLM is the recommended engine, however it is not required. If you are using\nVLLM, make sure yo include the `--return-tokens-as-token-ids` flag, or else your\nresponses will fail.\n\nOnce you have one (or multiple) models running, modify the default miner code to\nproxy to the proper VLLM instance on each request. Verifiers will include the\n`X-Targon-Model` header so that the miner node does not need to parse the actual\nbody.\n\nIn the `miner.py` script you will find a function called `list_models`. To serve multiple models you must:\n1. Fill this out to respond to validators with any model you currently have available (below is an example):\n```\n    async def list_models(self):\n        return [\n            \"ExampleName/Meta-Llama-3.1-8B-Instruct\",\n            \"ExampleName/mythomax-l2-13b\",\n            \"ExampleName/Hermes-3-Llama-3.1-8B\",\n            \"ExampleName/Nxcode-CQ-7B-orpo\",\n            \"ExampleName/deepseek-coder-33b-instruct\",\n            \"ExampleName/Llama-3.1-Nemotron-70B-Instruct-HF\",\n        ]\n```\n2. Update the `create_chat_completion` and `create_completion` methods in neurons/miner.py to route to the appropriate vllm upstream server based on the model (which is either in the headers or from the request payload's model param)\n\nHere is a hint / incomplete code snippet to get you started:\n```\nmodel_port_map = {\n    'ExampleName/mythomax-l2-13b': 1001,\n    'ExampleName/Hermes-3-Llama-3.1-8B': 1002,\n    'ExampleName/Nxcode-CQ-7B-orpo': 1003,\n    'ExampleName/deepseek-coder-33b-instruct': 1004,\n    'ExampleName/Llama-3.1-Nemotron-70B-Instruct-HF': 1005\n}\nfull_url = f\"http://127.0.0.1:{model_port_map.get(body.get('model'), 1000)}{path}\"\n```\n\nOnce this is complete, you are ready to continue starting your miner node.\n\n### PM2\n\nRunning a miner through PM2 will require the vLLM instance to be running.\n\n```bash\npm2 start neurons/miner.py --name miner --interpreter  python3 -- --wallet.name [WALLET_NAME] --netuid 4 --wallet.hotkey [WALLET_HOTKEY] --subtensor.network finney --model-endpoint [MODEL_ENDPOINT] --api_key [API_KEY] --axon.port [AXON PORT] --logging.trace\n```\n\n\u003e Please replace the following with your specific configuration:\n\u003e\n\u003e - \\[WALLET_NAME\\]\n\u003e - \\[WALLET_HOTKEY\\]\n\u003e - \\[MODEL_ENDPOINT\\]\n\u003e - \\[API_KEY\\]\n\u003e - \\[AXON_PORT\\]\n\nNOTE: Trace logging is very verbose. You can use `--logging.info` instead for\nless log bloat.\n\nAdditionally:\n\n```bash\n--no-force-validator-permit [TRUE/FALSE]\n\n```\n\nis defaulted to false to force incoming requests to have a permit. Set this to\ntrue if you are having trouble getting requests from validators on the 'test'\nnetwork.\n\n## Running a Validator\n\n### PM2\n\nValidators are simply run through pm2, enabling auto restarts and auto updates.\nA validator should be run on atleast an A100, but the larger the better, as\nlarger clusters can handle more models. The machine should have\n[nvidia-smi / cuda](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu)\ninstalled along with [docker](https://docs.docker.com/engine/install/ubuntu/).\n\n**No vllm instance needed**\n\nValidator Instance:\n\n```bash\npm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME]\n\n```\n\n\u003e Please replace the following with your specific configuration:\n\u003e\n\u003e - \\[WALLET_NAME\\]\n\n## Explanation of Args\n\n### Shared Args\n\n1. **--netuid** ==\u003e Subnet Netuid. *Defaults to 4*\n1. **--epoch-length** ==\u003e Default epoch length (how often we set weights,\n   measured in 12 second blocks). *Defaults to 360*\n1. **--mock** ==\u003e Mock neuron and all network components. *Defaults to False*\n\n### Miner Args\n\n1. **--neuron.name** ==\u003e Trials for this neuron go in neuron.root/ (wallet-cold\n   \\- wallet-hot) / neuron.name. *Defaults to miner*\n1. **--force_validator.permit** ==\u003e If set, forces incoming requests to have a\n   permit. *Defaults to True*\n1. **--model-endpoint** ==\u003e Endpoint to use for the OpenAi CompatibleClient.\n   *Defaults to \"http://127.0.0.1:8000/v1\"*\n1. **--api-key** ==\u003e API key for OpenAi Compatible API. *Defaults to \"12345\"*\n\n### Validator Args\n\n1. **--neuron.name** ==\u003e Trials for this neuron go in neuron.root/ (wallet-cold\n   \\- wallet-hot) / neuron.name. *Defaults to validator*\n1. **--timeout** ==\u003e The timeout for each forward call in seconds. *Defaults to\n   8*\n1. **--vpermit-tao-limit** ==\u003e The maximum number of TAO allowed to query a\n   validator with a permit. *Defaults to 4096*\n1. **--cache-file** ==\u003e Pickle file to save score cache to. *Defaults to\n   cache.pickle*\n1. **--database.url** ==\u003e Database URL to save Miner Data to Targon Hub.\n1. **--autoupdate-off** ==\u003e Disable automatic updates to Targon on latest\n   version on Main if set. *Defaults to True*\n1. **--models.mode** ==\u003e Mode to use for determining what models to run. Can be\n   one of:`default`, or `config`.\n   - `endpoint`: defaults to `https://targon.sybil.com/api/models`. This will\n     mimic the manifold validator\n   - `default`: only run NousResearch/Meta-Llama-3.1-8B-Instruct\n   - `config`: parse a text file named `models.txt` with a list of models\n     separated by newlines\n1. **--models.endpoint** ==\u003e Only used when models.mode is `endpoint`. Sets the\n   api endpoint to ping for list of models. Defaults to targon hub.\n\n\u003e Example model config file `models.txt`\n\u003e\n\u003e ```\n\u003e NousResearch/Meta-Llama-3.1-8B-Instruct\n\u003e NousResearch/Meta-Llama-3.1-70B-Instruct\n\u003e NousResearch/Meta-Llama-3.1-405B-Instruct\n\u003e ```\n\n## Autoupdate\n\nAutoupdate is implemented in targon/utils.py. This is to ensure that your\ncodebase matches the latest version on Main of the Targon Github Repository.\n\n### Validator Autoupdate\n\nValidator Autoupdate is implemented and defaulted to run once weights have been\nset. To **disable**, please add the flag to your command line build:\n\n```bash\npm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME] --autoupdate-off\n```\n\n### Miner Autoupdate\n\nMiner Autoupdate is **not** implemented. Miners will need to check the Targon\nrepository and update themselves as new versions are released. If interested in\nutilizing the autoupdate feature that Validators use, please follow the steps\nbelow:\n\n*NOTE*: This will not be maintained by the Manifold Labs Team.\n\n1. Import the autoupdate function into your miner script (neurons/miner.py) at\n   the top of the file.\n\n```python\nfrom targon.updater import autoupdate\n```\n\n3. Call the function at a place of your choosing.\n\n```python\n    if self.config.autoupdate:\n        autoupdate(branch=\"main\")\n\n```\n\n4. Relaunch your miner with the changes.\n\n### Targon Hub (WIP)\n\nThe goal of the hub is to give validators a simple way to directly generate\nrevenue off of their bittensor bandwidth. This is designed as a template for\nvalidators to take and create their own branded hubs with, however pull requests\nare still encouraged.\n\nIf you are interested in running your own instance of Targon Hub, you will need\nto add an additional flag to save the records of miners' responses to a\nPostgreSQL DB.\n\n**NOTE**: No flag means no database!\n\n```bash\n--database.url [DB_CONNECTION_STRING]\n\n```\n\n\u003e Please replace the following with your specific connection URL:\n\u003e\n\u003e - \\[DB_CONNECTION_STRING\\]\n\nBelow are steps to create a Supabase connection string to utilze this feature:\n\n1. Either create an account or log in to\n   [Supabase](https://supabase.com/dashboard/sign-in%5D)\n1. You might be asked to create an organization. In which case, choose the\n   options best suited for your use case.\n1. Once completed, create a new project with a secure password and location of\n   your choosing. Save your password, you will need it later. Your project will\n   then take a few minutes to be provisioned.\n1. Once the project has been created, click on the green `Connect` button near\n   the top right of the screen\n1. A modal should open up. Click on connection string, URI, and change the mode\n   from `transaction` to `session` in the dropdown\n1. Copy the connection string shown and insert your password\n1. Clone [Targon Hub](https://github.com/manifold-inc/targon-hub) and follow its\n   setup instructions\n1. Launch the validator with new `--database.url` flag and connection string\n\nPlease reach out to the SN4 team for help setting up targon hub in sn4 chat or\n[our discord](https://discord.gg/manifold)\n\n```bash\npm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME] --database.url [DB_CONNECTION_STRING]\n```\n\nAs your validator runs, you will start seeing records being added into your\nSupabase database. This will be directly what your Targon Hub will query.\n\n# What is A Deterministic Verification Network\n\nTargon uses a novel, in-house proof-of-work value to verify model responses.\nThese are generated just like log-probs, and can be used to verify with high\naccuracy that a response was generated with the model requested. We use this in\ncombination with a fast and slow version of log-prob verification. Verified\nresponses are scored by speed and consistency of verification.\n\n## Role of a Miner\n\nA miner is a node that is responsible for generating a output from a query, both\norganic and synthetic.\n\n## Role of a Validator\n\nA validator is a node that is responsible for verifying a miner's output. The\nvalidator will send an openai compliant request to a miner with. The miner will\nthen send back a response with the output. The validator will then use the proof\nof work values of the response to verify that each miners response is accurate.\nValidators will keep score of each miners response time and use their averages\nto assign scores each epoch.\n\n# Features of Targon\n\n## Full OpenAI Compliance\n\nValidators can query miners directly using any openai package, and Epistula\nheaders. Below is boilerplate for querying a miner in python.\n\n```py\nminer = openai.AsyncOpenAI(\n    base_url=f\"http://{axon.ip}:{axon.port}/v1\",\n    api_key=\"sn4\",\n    max_retries=0,\n    timeout=Timeout(12, connect=5, read=5),\n    http_client=openai.DefaultAsyncHttpxClient(\n        event_hooks={\n            \"request\": [\n                # This injects Epistula headers right before the request is sent.\n                # wallet.hotkey is the public / private keypair\n                #\n                # You can find this function in the `epistula.py` file in \n                # the targon repo\n                create_header_hook(wallet.hotkey, axon.hotkey_ss58)\n            ]\n        }\n    ),\n)\n```\n\n# How to Contribute\n\n## Code Review\n\nProject maintainers reserve the right to weigh the opinions of peer reviewers\nusing common sense judgement and may also weigh based on merit. Reviewers that\nhave demonstrated a deeper commitment and understanding of the project over time\nor who have clear domain expertise may naturally have more weight, as one would\nexpect in all walks of life. Where a patch set affects consensus-critical code,\nthe bar will be much higher in terms of discussion and peer review requirements,\nkeeping in mind that mistakes could be very costly to the wider community. This\nincludes refactoring of consensus-critical code. Where a patch set proposes to\nchange the Targon subnet, it must have been discussed extensively on the discord\nserver and other channels, be accompanied by a widely discussed BIP and have a\ngenerally widely perceived technical consensus of being a worthwhile change\nbased on the judgement of the maintainers. That being said, Manifold welcomes\nall PR's for the betterment of the subnet and Bittensor as a whole. We are\nstriving for improvement at every interval and believe through open\ncommunication and sharing of ideas will success be attainable.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanifold-inc%2Ftargon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmanifold-inc%2Ftargon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanifold-inc%2Ftargon/lists"}