{"id":43938194,"url":"https://github.com/knap-ai/knapsack","last_synced_at":"2026-02-07T01:26:29.766Z","repository":{"id":237568264,"uuid":"787166916","full_name":"knap-ai/knapsack","owner":"knap-ai","description":"Fast, private data connectors for AI ⚡️🤖","archived":false,"fork":false,"pushed_at":"2024-05-20T21:15:58.000Z","size":287,"stargazers_count":6,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-05-20T22:59:54.547Z","etag":null,"topics":["artificial-intelligence","data-engineering","ml-engineering"],"latest_commit_sha":null,"homepage":"https://knap.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/knap-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-16T02:31:48.000Z","updated_at":"2024-05-20T20:00:07.000Z","dependencies_parsed_at":"2024-05-20T21:23:54.287Z","dependency_job_id":"597c7719-2099-4444-a0f0-c4ba1c27233a","html_url":"https://github.com/knap-ai/knapsack","commit_stats":null,"previous_names":["knap-ai/knapsack"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/knap-ai/knapsack","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knap-ai%2Fknapsack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knap-ai%2Fknapsack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knap-ai%2Fknapsack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knap-ai%2Fknapsack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/knap-ai","download_url":"https://codeload.github.com/knap-ai/knapsack/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knap-ai%2Fknapsack/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29183824,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T00:44:15.062Z","status":"ssl_error","status_checked_at":"2026-02-07T00:35:01.758Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","data-engineering","ml-engineering"],"created_at":"2026-02-07T01:26:29.176Z","updated_at":"2026-02-07T01:26:29.754Z","avatar_url":"https://github.com/knap-ai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"assets/knapsack-logo.svg\" width=\"100\" height=\"100\"\u003e\n\nKnapsack 🎒 - Data connectors for fast, private AI.\n---\n\n## Title and Description 📝\nKnapsack 🎒 is a open-source service that hosts and runs fast, private connectors for data to AI projects. Much like Glean or Perplexity, Knapsack 🎒 powers intelligent search and next-gen AI applications, but with an emphasis on community, privacy, and security.  \n\n## Installation and Setup ⚙️\nKnapsack connectors fetch data, transform, and load that data into a VectorDB backend. Efficient, secure, and easy data handling is our bread and butter. To this end, Knapsack 🎒 provides a simple, easy-to-use API for data connectors and the service can be launched via Docker.\n\nTo get started with Knapsack 🎒, ensure you have Docker installed on your machine. You can launch the service using Docker Compose:\n\n1. Clone the repository to your local machine:\n   ```bash\n   git clone https://github.com/your-repo/knapsack.git\n   cd knapsack\n\n\n2. Run it as a FastAPI server on your local machine:\n```bash\npython -m knapsack.cli deploy --port 8888\n```\n\n3. Or run it as a FastAPI server in a Docker container \n```bash\ndocker-compose up\n```\n\n4. Or utilize it directly as a library:\n```python\nfrom knapsack import Knapsack\nks = Knapsack()\nks.run()\n```\n\nPlease note that the main_url property in the knapsack.toml database configuration must be set to the database image name (e.g., qdrant if using Docker Compose) or to localhost if running in the local environment\n\n## Roadmap 🔨\n\n- [x] ArXiv, Base connector\n- [x] Qdrant integration\n- [x] Caching of certain APIs\n- [x] Smart upsert to vector DB (hashed values, only upsert on change)\n- [x] Scheduling\n- [ ] GSuite\n- [ ] BioArXiv\n- [ ] PubMed\n\n### VectorDB Integrations\n- [x] Qdrant\n- [ ] Milvus\n- [ ] Weaviate\n- [ ] Chroma\n\n\n## How to Contribute 🤝\nWe welcome contributions from the community! Currently, we are particularly interested in adding more connectors. If you have developed a connector that could be useful to others, please consider submitting a pull request.\n\nFor those interested in public data, Knapsack 🎒 hosts publicly-accessible datasets, such as data derived from ArXiv, available for search and GPT chat via the Knapsack Desktop application. If you want to contribute to Knapsack 🎒 could, please reach out via our GitHub issues or file a pull request. Knap will host any new connectors that connect public data so that all users can take benefit from the abilities of LLM chat and search.\n\n## License Information 📄\nKnapsack 🎒 is released under the GNU General Public License v3.0. For more information, please refer to the LICENSE file in the repository.\n\nFeel free to explore, modify, and distribute any part of Knapsack's 🎒 codebase. If you use Knapsack 🎒 in your research or projects, please consider citing it.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fknap-ai%2Fknapsack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fknap-ai%2Fknapsack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fknap-ai%2Fknapsack/lists"}