{"id":27505491,"url":"https://github.com/Desearch-ai/subnet-22","last_synced_at":"2025-04-17T19:02:31.462Z","repository":{"id":214405137,"uuid":"732052524","full_name":"Desearch-ai/subnet-22","owner":"Desearch-ai","description":"Subnet 22","archived":false,"fork":false,"pushed_at":"2025-04-16T13:36:33.000Z","size":3068,"stargazers_count":28,"open_issues_count":3,"forks_count":29,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-16T20:17:45.412Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://desearch.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Desearch-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-15T14:20:06.000Z","updated_at":"2025-04-16T05:17:19.000Z","dependencies_parsed_at":"2024-03-18T19:49:32.945Z","dependency_job_id":"81a7d7b9-455d-47ca-8572-0ce141c5a760","html_url":"https://github.com/Desearch-ai/subnet-22","commit_stats":null,"previous_names":["surcyf123/smart-scrape","datura-ai/smart-scrape","datura-ai/desearch","desearch-ai/subnet-22"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Desearch-ai%2Fsubnet-22","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Desearch-ai%2Fsubnet-22/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Desearch-ai%2Fsubnet-22/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Desearch-ai%2Fsubnet-22/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Desearch-ai","download_url":"https://codeload.github.com/Desearch-ai/subnet-22/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249368843,"owners_count":21258901,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-17T19:01:41.875Z","updated_at":"2025-04-17T19:02:31.452Z","avatar_url":"https://github.com/Desearch-ai.png","language":"Python","funding_links":[],"categories":["Registered Subnets"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# **Desearch (Subnet 22) on Bittensor**\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) \n\n\u003c/div\u003e\n\n## Introduction\n\n**Bittensor Desearch (Subnet 22):** \n\nWelcome to Desearch, the AI-powered search engine built on Bittensor. Designed for the Bittensor community and general internet users, Desearch delivers an unbiased and verifiable search experience. Through our API, developers and AI builders are empowered to integrate AI search capabilities into their products, with access to metadata from platforms like X, Reddit, Arxiv and general web search.\n\n\n### Key Features\n\n- **AI-powered Analysis:** Utilizes decentralized AI models to deliver relevant, contextual, and unfiltered search results.\n- **Real-time Access to Diverse Data Sources:** Access metadata from platforms like X, Reddit, Arxiv, and broader web data.\n- **Sentiment and Metadata Analysis:** Determines the emotional tone of social posts while analyzing key metadata to provide a comprehensive understanding of public sentiment.\n- **Time-efficient:** Minimizes manual data sorting, saving valuable research time.\n- **User-friendly Design:** Suitable for both beginners and experts.\n\n### Advantages\n\n- **Decentralized Platform:** Built on the Bittensor network, ensures unbiased and highly relevant search results through decentralization.\n- **Customizability:** Tailors data analysis to meet specific user requirements.\n- **Versatility:** Applicable for diverse research fields, from market analysis to academic studies.\n- **Community-driven Innovation:** Built and optimized by a decentralized network of Bittensor miners, validators, and users for continuous search result enhancement.\n\n---\n\n## Installation\n\n**Requirements:** Python 3.10 or higher\n\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/Datura-ai/desearch.git\n   ```\n2. Install the requirements:\n   ```bash\n   cd desearch\n   python -m pip install -r requirements.txt\n   python -m pip install -e .\n   ```\n\n---\n\n## Preparing Your Environment\n\nBefore running a miner or validator, ensure to:\n\n- [Create a wallet](https://github.com/opentensor/docs/blob/main/reference/btcli.md).\n- [Register the wallet to a netuid](https://github.com/opentensor/docs/blob/main/subnetworks/registration.md).\n\n\n### Environment Variables Configuration\n\nFor setting up the necessary environment variables for your miner or validator, please refer to the [Environment Variables Guide](./docs/env_variables.md).\n\n# Running the Miner\n\n  ```bash\n  python -m neurons/miners/miner.py \n      --netuid 22\n      --subtensor.network finney\n      --wallet.name \u003cyour miner wallet\u003e\n      --wallet.hotkey \u003cyour validator hotkey\u003e\n      --axon.port 14000\n  ```\n\n# Running the Validator API with Automatic Updates\n\nThese validators are designed to run and update themselves automatically. To run a validator, follow these steps:\n\n1. Install this repository, you can do so by following the steps outlined in [the installation section](#installation).\n2. Install [Weights and Biases](https://docs.wandb.ai/quickstart) and run `wandb login` within this repository. This will initialize Weights and Biases, enabling you to view KPIs and Metrics on your validator. (Strongly recommended to help the network improve from data sharing)\n3. Install [Redis](https://redis.io/docs/latest/operate/oss_and_stack/install/install-redis/).\n4. Install [PM2](https://pm2.io/docs/runtime/guide/installation/) and the [`jq` package](https://jqlang.github.io/jq/) on your system.\n   **On Linux**:\n   ```bash\n   sudo apt update \u0026\u0026 sudo apt install jq \u0026\u0026 sudo apt install npm \u0026\u0026 sudo npm install pm2 -g \u0026\u0026 pm2 update\n   ``` \n   **On Mac OS**\n   ```bash\n   brew update \u0026\u0026 brew install jq \u0026\u0026 brew install npm \u0026\u0026 sudo npm install pm2 -g \u0026\u0026 pm2 update\n   ```\n5. Run the `run.sh` script which will handle running your validator and pulling the latest updates as they are issued. \n   ```bash\n   pm2 start run.sh --name smart_scrape_validators_autoupdate -- --wallet.name \u003cyour-wallet-name\u003e --wallet.hotkey \u003cyour-wallet-hot-key\u003e\n   ```\n\nThis will run **two** PM2 process: one for the validator which is called `smart_scrape_validators_main_process` by default (you can change this in `run.sh`), and one for the run.sh script (in step 4, we named it `smart_scrape_validators_autoupdate`). The script will check for updates every 30 minutes, if there is an update then it will pull it, install it, restart `smart_scrape_validators_main_process` and then restart itself.\n\n### Detailed Setup Instructions\n\nFor step-by-step guidance on setting up and running a miner, validator, or operating on the testnet or mainnet, refer to the following guides:\n- [Miner Setup](./docs/running_a_miner.md)\n- [Validator Setup](./docs/running_a_validator.md)\n- [Testnet Operations](./docs/running_on_testnet.md)\n- [Mainnet Operations](./docs/running_on_mainnet.md)\n- [Setting Up and Running the Web Application with Validator Integration](./ui/README.md)\n\n---\n\n\n## Real-time Monitoring with wandb Integration\n\nThe text prompting validator sends data to wandb, allowing real-time monitoring with key metrics like:\n- Gating model loss\n- Hardware usage\n- Forward pass time\n- Block duration\n\nData is publicly available at [this link](https://wandb.ai/smart-scrape/smart-scrape-1.0). Note that [data from anonymous users is deleted after 7 days](https://docs.wandb.ai/guides/app/features/anon).\n\n\u003c/div\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDesearch-ai%2Fsubnet-22","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FDesearch-ai%2Fsubnet-22","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDesearch-ai%2Fsubnet-22/lists"}