{"id":34682447,"url":"https://github.com/izeigerman/mindshard","last_synced_at":"2025-12-24T21:14:59.751Z","repository":{"id":320110017,"uuid":"1077659547","full_name":"izeigerman/mindshard","owner":"izeigerman","description":"A Rust-based HTTP proxy that captures and indexes HTTP traffic using vector embeddings.","archived":false,"fork":false,"pushed_at":"2025-11-14T21:57:31.000Z","size":22794,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-14T23:40:07.236Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/izeigerman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-16T14:56:55.000Z","updated_at":"2025-11-14T21:57:35.000Z","dependencies_parsed_at":"2025-10-22T02:40:31.080Z","dependency_job_id":"e466d40d-9ddb-43c9-b67e-5cb54961641f","html_url":"https://github.com/izeigerman/mindshard","commit_stats":null,"previous_names":["izeigerman/mindshard"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/izeigerman/mindshard","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izeigerman%2Fmindshard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izeigerman%2Fmindshard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izeigerman%2Fmindshard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izeigerman%2Fmindshard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/izeigerman","download_url":"https://codeload.github.com/izeigerman/mindshard/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izeigerman%2Fmindshard/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28008635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-24T02:00:07.193Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-24T21:14:59.424Z","updated_at":"2025-12-24T21:14:59.746Z","avatar_url":"https://github.com/izeigerman.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🧠 MindShard\n\nMindShard is a shared memory of everything you browse. Built in Rust, it captures and indexes HTTP traffic with vector embeddings, enabling semantic search across your browsing history.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/mindshard_demo.gif\" alt=\"MindShard\"/\u003e\n\u003c/p\u003e\n\nMindShard respects your privacy. All processing happens locally, and no third-party services are involved beyond the initial download of the embedding model from Hugging Face.\n\n**This project is experimental and should be used for educational purposes only.** MindShard intercepts and stores HTTP traffic, which may include sensitive information. Use only in controlled environments. The author is not responsible for any misuse or data exposure resulting from the use of this software.\n\n- [Why?](#why)\n- [Prerequisites](#prerequisites)\n- [Build](#build)\n- [Run](#run)\n  - [Local](#local)\n  - [Docker](#docker)\n- [Configuration](#configuration)\n- [Installing and Trusting the Root Certificate](#installing-and-trusting-the-root-certificate)\n  - [MacOS](#macos)\n  - [Windows](#windows)\n  - [Linux (Ubuntu/Debian)](#linux-ubuntudebian)\n- [Configuring Your System Proxy](#configuring-your-system-proxy)\n  - [MacOS](#macos-1)\n  - [Windows](#windows-1)\n  - [Linux (Ubuntu/Debian)](#linux-ubuntudebian-1)\n  - [Browser-Specific Configuration](#browser-specific-configuration)\n\n## Why?\n\nHave you ever read an interesting article, blog post, or research paper, only to struggle weeks later to recall the source of a quote or idea you remembered? You search through your browser history, try various keyword combinations, but can't locate the original source.\n\nMindShard solves this problem by automatically capturing and indexing everything you read online using vector embeddings. It provides semantic search capabilities that let you find content based on meaning and context, not just exact keyword matches. Simply describe the idea or concept you remember, and MindShard will surface the relevant passages along with links to the original sources.\n\nThink of it as a photographic memory for your web browsing - every article, blog post, and paper you read is preserved and instantly searchable by its actual content and meaning.\n\n## Prerequisites\n\n- Rust 1.88 or later\n- OpenSSL\n\nGenerate a self-signed certificate and private key:\n\n```bash\n./create_cer.sh\n```\n\nThis creates `mindshard.key` and `mindshard.cer` required for the proxy.\n\n## Build\n\n```bash\ncargo build --release\n```\n\n## Run\n\n### Local\n\n```bash\ncargo run --release\n```\n\nMake sure you have the required certificate files (`mindshard.key` and `mindshard.cer`) in your working directory.\n\n### Docker\n\n#### Quick Start with Docker Compose\n\nThe easiest way to run MindShard with Docker:\n\n```bash\ndocker-compose up -d\n```\n\nThis will:\n- Build the Docker image\n- Mount your certificates from the current directory\n- Create a persistent volume for the database\n- Start the application with ports 8080 (proxy) and 3000 (web) exposed\n\n#### Building the image\n\n```bash\ndocker build -t mindshard:latest .\n```\n\n#### Running with Docker\n\n**Important:** The Docker container requires mounting the private key and certificate files.\n\nBasic usage:\n```bash\ndocker run -d \\\n  --name mindshard \\\n  -p 8080:8080 \\\n  -p 3000:3000 \\\n  -v $(pwd)/mindshard.key:/app/certs/mindshard.key:ro \\\n  -v $(pwd)/mindshard.cer:/app/certs/mindshard.cer:ro \\\n  -v mindshard-data:/app/data \\\n  mindshard:latest\n```\n\n#### Volume Mounts\n\n**Required:**\n- **Private Key**: `-v /path/to/mindshard.key:/app/certs/mindshard.key:ro`\n- **Certificate**: `-v /path/to/mindshard.cer:/app/certs/mindshard.cer:ro`\n\n**Recommended:**\n- **Database** (for persistence): `-v mindshard-data:/app/data` or `-v $(pwd)/data:/app/data`\n\n## Configuration\n\nThe application is configured via environment variables:\n\n| Variable                     | Default         | Description                                                                 |\n|------------------------------|-----------------|-----------------------------------------------------------------------------|\n| `MINDSHARD_PROXY_PORT`       | `8080`          | Port for the HTTP proxy server                                              |\n| `MINDSHARD_WEB_PORT`         | `3000`          | Port for the web server                                                     |\n| `MINDSHARD_PRIVATE_KEY_PATH` | `mindshard.key` | Path to the private key file                                                |\n| `MINDSHARD_CA_CERT_PATH`     | `mindshard.cer` | Path to the CA certificate file                                             |\n| `MINDSHARD_DB_PATH`          | `mindshard.db`  | Path to the LibSQL database file                                            |\n| `MINDSHARD_BROWSER_ONLY`     | `true`          | Only process requests from web browsers (filters out API calls, bots, etc.) |\n\n## Installing and Trusting the Root Certificate\n\nTo avoid browser security warnings when using the proxy, you need to install and trust the generated `mindshard.cer` certificate as a trusted root certificate authority on your system.\n\n### MacOS\n\n1. Double-click `mindshard.cer` to open Keychain Access\n2. Select \"System\" keychain and click \"Add\"\n3. Find the certificate in Keychain Access, double-click it\n4. Expand \"Trust\" section and set \"When using this certificate\" to \"Always Trust\"\n5. Close the window and enter your password to confirm\n\n[Official guide](https://support.apple.com/guide/keychain-access/add-certificates-to-a-keychain-kyca2431/mac)\n\n### Windows\n\n1. Double-click `mindshard.cer`\n2. Click \"Install Certificate\"\n3. Select \"Local Machine\" and click \"Next\"\n4. Select \"Place all certificates in the following store\"\n5. Click \"Browse\" and select \"Trusted Root Certification Authorities\"\n6. Click \"Next\" and then \"Finish\"\n\n[Official guide](https://learn.microsoft.com/en-us/skype-sdk/sdn/articles/installing-the-trusted-root-certificate)\n\n### Linux (Ubuntu/Debian)\n\n```bash\nsudo cp mindshard.cer /usr/local/share/ca-certificates/mindshard.crt\nsudo update-ca-certificates\n```\n\nFor Firefox on Linux, you need to import the certificate separately in Firefox settings under Privacy \u0026 Security \u003e Certificates \u003e View Certificates \u003e Authorities \u003e Import.\n\n[Official guide](https://ubuntu.com/server/docs/install-a-root-ca-certificate-in-the-trust-store)\n\n## Configuring Your System Proxy\n\nTo route traffic through MindShard, configure your system's HTTP(S) proxy settings to point to `localhost:8080` (or your configured `MINDSHARD_PROXY_PORT`).\n\n### MacOS\n\n1. Open **System Settings** \u003e **Network**\n2. Select your active network connection (Wi-Fi or Ethernet)\n3. Click **Details** \u003e **Proxies**\n4. Check **Web Proxy (HTTP)** and **Secure Web Proxy (HTTPS)**\n5. For both, enter:\n   - **Server**: `localhost` or `127.0.0.1`\n   - **Port**: `8080`\n6. Click **OK** and **Apply**\n\n### Windows\n\n1. Open **Settings** \u003e **Network \u0026 Internet** \u003e **Proxy**\n2. Under **Manual proxy setup**, toggle **Use a proxy server** to **On**\n3. Enter:\n   - **Address**: `localhost` or `127.0.0.1`\n   - **Port**: `8080`\n4. Click **Save**\n\nAlternatively, via Command Prompt (as Administrator):\n```cmd\nnetsh winhttp set proxy localhost:8080\n```\n\nTo disable:\n```cmd\nnetsh winhttp reset proxy\n```\n\n### Linux (Ubuntu/Debian)\n\n**GNOME Desktop:**\n1. Open **Settings** \u003e **Network** \u003e **Network Proxy**\n2. Select **Manual**\n3. For **HTTP Proxy** and **HTTPS Proxy**, enter:\n   - **Host**: `localhost` or `127.0.0.1`\n   - **Port**: `8080`\n4. Click **Apply**\n\n**Command Line (for current session):**\n```bash\nexport http_proxy=http://localhost:8080\nexport https_proxy=http://localhost:8080\n```\n\n**Persistent (add to `~/.bashrc` or `~/.zshrc`):**\n```bash\necho 'export http_proxy=http://localhost:8080' \u003e\u003e ~/.bashrc\necho 'export https_proxy=http://localhost:8080' \u003e\u003e ~/.bashrc\nsource ~/.bashrc\n```\n\n### Browser-Specific Configuration\n\nSome browsers (like Firefox) use their own proxy settings independent of system settings:\n\n**Firefox:**\n1. Open **Settings** \u003e **General** \u003e **Network Settings**\n2. Click **Settings**\n3. Select **Manual proxy configuration**\n4. For **HTTP Proxy** and **HTTPS Proxy**, enter:\n   - **Host**: `localhost`\n   - **Port**: `8080`\n5. Check **Also use this proxy for HTTPS**\n6. Click **OK**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fizeigerman%2Fmindshard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fizeigerman%2Fmindshard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fizeigerman%2Fmindshard/lists"}