{"id":7275834,"url":"https://github.com/superlinked/superlinked","last_synced_at":"2026-01-16T11:11:18.899Z","repository":{"id":227678636,"uuid":"715536935","full_name":"superlinked/superlinked","owner":"superlinked","description":"Superlinked is a Python framework for AI Engineers building high-performance search \u0026 recommendation applications that combine structured and unstructured data.","archived":false,"fork":false,"pushed_at":"2025-12-10T15:35:42.000Z","size":146477,"stargazers_count":1452,"open_issues_count":17,"forks_count":110,"subscribers_count":28,"default_branch":"main","last_synced_at":"2025-12-11T00:54:46.741Z","etag":null,"topics":["data-pipeline","deep-learning","embeddings","etl","information-retrieval","llm","ml","mlops","natural-language-processing","nlp","python","retrieval","retrieval-augmented-generation","semantic-search","vector-database","vector-search","vectorization"],"latest_commit_sha":null,"homepage":"https://superlinked.com","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/superlinked.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-11-07T10:45:53.000Z","updated_at":"2025-12-10T15:35:46.000Z","dependencies_parsed_at":"2024-03-16T19:08:17.969Z","dependency_job_id":"e00ff500-aadb-4f36-a165-625e3fd00146","html_url":"https://github.com/superlinked/superlinked","commit_stats":null,"previous_names":["superlinked/superlinked"],"tags_count":804,"template":false,"template_full_name":null,"purl":"pkg:github/superlinked/superlinked","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superlinked%2Fsuperlinked","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superlinked%2Fsuperlinked/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superlinked%2Fsuperlinked/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superlinked%2Fsuperlinked/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/superlinked","download_url":"https://codeload.github.com/superlinked/superlinked/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superlinked%2Fsuperlinked/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478135,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T06:30:42.265Z","status":"ssl_error","status_checked_at":"2026-01-16T06:30:16.248Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-pipeline","deep-learning","embeddings","etl","information-retrieval","llm","ml","mlops","natural-language-processing","nlp","python","retrieval","retrieval-augmented-generation","semantic-search","vector-database","vector-search","vectorization"],"created_at":"2024-04-03T06:15:57.820Z","updated_at":"2026-01-16T11:11:18.892Z","avatar_url":"https://github.com/superlinked.png","language":"Jupyter Notebook","readme":"\u003c!-- ── HERO block ── --\u003e\n\u003cdiv align=\"center\"\u003e\n\n\u003c!-- Logo (auto dark/light) --\u003e\n\u003cpicture\u003e\n  \u003csource srcset=\"https://cdn.prod.website-files.com/65dce6831bf9f730421e2915/66ef0317ed8616151ee1d451_superlinked_logo_white.png\"\n          media=\"(prefers-color-scheme: dark)\"\u003e\n  \u003cimg width=\"320\"\n       src=\"https://cdn.prod.website-files.com/65dce6831bf9f730421e2915/65dce6831bf9f730421e2929_superlinked_logo.svg\"\n       alt=\"Superlinked logo\"\u003e\n\u003c/picture\u003e\n\n\u003c!-- Primary CTA --\u003e\n\u003cp\u003e\n  \u003ca href=\"https://links.superlinked.com/deep_wiki_repo\" style=\"text-decoration:none;display:inline-block\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/chat%20with%20our%20AI%20docs-%E2%86%92-72A1FF?style=for-the-badge\u0026logo=readthedocs\u0026logoColor=white\"\n         alt=\"chat with our AI docs\" width=\"200\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003c!-- Secondary badges --\u003e\n\u003cp style=\"margin:6px 0;text-decoration:none\"\u003e\n  \u003ca href=\"https://docs.superlinked.com/\"                     style=\"text-decoration:none;display:inline-block\"\u003e\u003cimg src=\"https://img.shields.io/badge/Docs-orange?logo=readthedocs\"                                        alt=\"Docs\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://deepwiki.com/superlinked/superlinked\"      style=\"text-decoration:none;display:inline-block\"\u003e\u003cimg src=\"https://deepwiki.com/badge.svg\"                                                       alt=\"DeepWiki\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/superlinked/\"             style=\"text-decoration:none;display:inline-block\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/superlinked\"                                             alt=\"PyPI\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/superlinked/superlinked/stargazers\" style=\"text-decoration:none;display:inline-block\"\u003e\u003cimg src=\"https://img.shields.io/github/stars/superlinked/superlinked?style=social\" alt=\"Stars\"\u003e\u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/github/last-commit/superlinked/superlinked\" style=\"display:inline-block\" alt=\"Last commit\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/license/superlinked/superlinked\"     style=\"display:inline-block\" alt=\"License\"\u003e\n\u003c/p\u003e\n\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cem\u003eBuild high-performance \u003cb\u003e AI search \u003c/b\u003e applications for both humans and agents. \u003c/em\u003e\n\u003c/p\u003e\n\n## Quickstart\n```python\n%pip install superlinked\n```\n```python\nfrom superlinked import framework as sl  \n  \n# Define schema for movie reviews  \nclass Review(sl.Schema):  \n    id: sl.IdField  \n    text: sl.String  \n  \nreview = Review()  \nspace = sl.TextSimilaritySpace(text=review.text, model=\"all-MiniLM-L6-v2\")  \nindex = sl.Index(space)  \nquery = sl.Query(index).find(review).similar(space, sl.Param(\"search\")).select_all()  \n  \n# Setup and run  \nsource = sl.InMemorySource(review)  \napp = sl.InMemoryExecutor(sources=[source], indices=[index]).run()  \n  \n# Add data and search  \nsource.put([  \n    {\"id\": \"1\", \"text\": \"Amazing acting and great story\"},  \n    {\"id\": \"2\", \"text\": \"Boring plot with bad acting\"}  \n])  \n  \nresult = app.query(query, search=\"excellent performance\")  \nprint(sl.PandasConverter.to_pandas(result))\n\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eTable\u0026nbsp;of\u0026nbsp;Contents\u003c/strong\u003e\u003c/summary\u003e\n\n- [Table of Contents](#table-of-contents)\n- [Quickstart](#quickstart)\n- [Overview](#overview)\n- [Hands on tutorials](#hands-on-tutorials)\n- [Use-cases](#use-cases)\n- [Experiment in a notebook](#experiment-in-a-notebook)\n    - [Install the superlinked library](#install-the-superlinked-library)\n    - [Run the example:](#run-the-example)\n- [Run in production](#run-in-production)\n  - [Supported Vector Databases](#supported-vector-databases)\n- [Logging](#logging)\n- [Resources](#resources)\n- [Support](#support)\n\n\u003c/details\u003e\n\n## Overview\n\n- **WHY**: Improve your vector search relevance by encoding metadata together with your unstructured data into vectors.\n- **WHAT**: A framework and a self-hostable REST API server that connects your data, vector database and backend services.\n- **HOW**: Construct custom data \u0026 query embedding models from pre-trained encoders from `sentence-transformers`, `open-clip` and custom encoders for numbers, timestamps and categorical data. See the [tutorials](#hands-on-tutorials) and [use-case](#use-cases) notebooks below for examples.\n\nIf you like what we do, give us a star! ⭐\n\n![](./docs/images/sl_diagram.png)\n\n\n## Hands-on Tutorials\n\n| Level | What you’ll build \u0026 learn | Try it now |\n|-------|---------------------------|------------|\n| **Start here** | Embed text · images · numbers · categories · time · events. | Text embedding\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/text_embedding.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eImage embedding\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/image_embedding.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eNumeric (min-max)\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/number_embedding_minmax.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eNumeric (similar)\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/number_embedding_similar.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eCategorical\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/categorical_embedding.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eRecency embedding\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/recency_embedding.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eEvent effects\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/event_effects.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e |\n| **Build \u0026 extend** | Combine spaces or add custom / optional schemas. | Combine embeddings\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/combine_multiple_embeddings.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eCustom space\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/custom_space.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eOptional fields\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/optional_schema_fields.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e |\n| **Optimise relevance** | Real-time updates \u0026 query-time personalisation. | Dynamic parameters\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/dynamic_parameters.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eQuery-time weights\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/query_time_weights.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eQuery result\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/query_result.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e |\n| **Search \u0026 filter** | NL search + hard filters. | NL query\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/natural_language_querying.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eHard filtering\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/hard_filtering.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eQuery options\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/querying_options.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e\u003cbr\u003eVector parts\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/accessing_stored_vector_parts.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e |\n| **Analyse \u0026 export** | Sample embeddings offline. | Vector sampler\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/vector_sampler.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e |\n| **Go multi-modal** | Unified text + image space. | Multimodal search\u0026nbsp;\u003ca href=\"https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/feature/image_embedding.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"\u003e\u003c/a\u003e |\n\n\n\u003e 💡 **Want even more?** Browse the complete list of features \u0026 concepts in our docs →  \n\u003e \u003chttps://docs.superlinked.com/concepts/overview\u003e\n\n\n## Experiment in a notebook\n\nLet's build an e-commerce product search that understands product descriptions and ratings:\n\n#### Run the notebook example:\n\n\u003eFirst run will take a minute to download the embedding model.  \n\n```python\n%pip install superlinked\n```\n```python\nimport json\nimport os\n\nfrom superlinked import framework as sl\n\n\nclass Product(sl.Schema):\n    id: sl.IdField\n    description: sl.String\n    rating: sl.Integer\n\n\nproduct = Product()\n\ndescription_space = sl.TextSimilaritySpace(\n    text=product.description, model=\"Alibaba-NLP/gte-large-en-v1.5\"\n)\nrating_space = sl.NumberSpace(\n    number=product.rating, min_value=1, max_value=5, mode=sl.Mode.MAXIMUM\n)\nindex = sl.Index([description_space, rating_space], fields=[product.rating])\n\n\n# Define your query and parameters to set them directly at query-time\n# or let an LLM fill them in for you using the `natural_language_query` param.\n# Don't forget to set your OpenAI API key to unlock this feature.\nquery = (\n    sl.Query(\n        index,\n        weights={\n            description_space: sl.Param(\"description_weight\"),\n            rating_space: sl.Param(\"rating_weight\"),\n        },\n    )\n    .find(product)\n    .similar(\n        description_space,\n        sl.Param(\n            \"description_query\",\n            description=\"The text in the user's query that refers to product descriptions.\",\n        ),\n    )\n    .limit(sl.Param(\"limit\"))\n    .with_natural_query(\n        sl.Param(\"natural_language_query\"),\n        sl.OpenAIClientConfig(api_key=os.environ[\"OPEN_AI_API_KEY\"], model=\"gpt-4o\")\n    )\n)\n\n# Run the app in-memory (server \u0026 Apache Spark executors available too!).\nsource = sl.InMemorySource(product)\nexecutor = sl.InMemoryExecutor(sources=[source], indices=[index])\napp = executor.run()\n\n\n# Ingest data into the system - index updates and other processing happens automatically.\nsource.put([\n    {\n        \"id\": 1,\n        \"description\": \"Budget toothbrush in black color. Just what you need.\",\n        \"rating\": 1,\n    },\n    {\n        \"id\": 2,\n        \"description\": \"High-end toothbrush created with no compromises.\",\n        \"rating\": 5,\n    },\n    {\n        \"id\": 3,\n        \"description\": \"A toothbrush created for the smart 21st century man.\",\n        \"rating\": 3,\n    },\n])\n\nresult = app.query(query, natural_query=\"best toothbrushes\", limit=1)\n\n# Examine the extracted parameters from your query\nprint(json.dumps(result.metadata, indent=2))\n\n# The result is the 5-star rated product.\nsl.PandasConverter.to_pandas(result)\n```\n\n## Use-cases\n\nDive deeper with our notebooks into how each use-case benefits from the Superlinked framework.\n\n- **RAG**: [HR Knowledgebase](https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/rag_hr_knowledgebase.ipynb)\n- **Semantic Search**: [Movies](https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/semantic_search_netflix_titles.ipynb), [Business News](https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/semantic_search_news.ipynb), [Product Images \u0026 Descriptions](https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/image_search_e_commerce.ipynb)\n- **Recommendation Systems**: [E-commerce](https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/recommendations_e_commerce.ipynb)\n- **Analytics**: [User Acquisition](https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/analytics_user_acquisition.ipynb), [Keyword expansion](https://colab.research.google.com/github/superlinked/superlinked/blob/main/notebook/analytics_keyword_expansion_ads.ipynb)\n\nYou can check a full list of examples [here](https://github.com/superlinked/superlinked/tree/main/notebook).\n\n\n## Run in production\n\nWith a single command you can run Superlinked as a REST API Server locally or in your cloud with [Superlinked Server](https://pypi.org/project/superlinked-server). Get data ingestion and query APIs, embedding model inference and deep vector database integrations for free!\n\nUnify your evaluation, ingestion and serving stacks with a single declarative python codebase. Superlinked enables this by letting you define your data schema, vector indexes and the compute DAG that links them all at once and then choose the right executor for the task - in-memory or server.\n\nIf you are interested in learning more about running at scale, [Book a demo](https://links.superlinked.com/sl-repo-readme-form) for early access to our managed cloud.\n\n### Supported Vector Databases\n\nSuperlinked stores *your vectors* in *your vector database*, with deep integrations for:\n\n- [Redis](https://docs.superlinked.com/run-in-production/index-1/redis)\n- [MongoDB](https://docs.superlinked.com/run-in-production/index-1/mongodb)\n- [Qdrant](https://docs.superlinked.com/run-in-production/index-1/qdrant)\n- [TopK](https://docs.superlinked.com/run-in-production/index-1/topk)\n- [Which one should we support next?](https://github.com/superlinked/superlinked/discussions/41)\n\nCurious about vector database pros \u0026 cons in general? Our community [compared 44 Vector Databases on 30+ features](https://superlinked.com/vector-db-comparison/).\n\n## Logging\n\nThe Superlinked framework logs include contextual information, such as the process ID and package scope. Personally Identifiable Information (PII) is filtered out by default but can be exposed with the `SUPERLINKED_EXPOSE_PII` environment variable to `true`.\n\n## Resources\n\n- [Vector DB Comparison](https://superlinked.com/vector-db-comparison/): Open-source collaborative comparison of vector databases by Superlinked.\n- [VectorHub](https://superlinked.com/vectorhub/): VectorHub is a free and open-sourced learning hub for people interested in adding vector retrieval to their ML stack\n\n\n## Support\n\nNeed help? We're here to support you:\n- Report a bug by creating an [issue](https://github.com/superlinked/superlinked/issues/new?assignees=kembala\u0026labels=bug\u0026projects=\u0026template=bug_report.md\u0026title=)\n- Request a new feature [here](https://github.com/superlinked/superlinked/issues/new?assignees=kembala\u0026labels=enhancement\u0026projects=\u0026template=feature_request.md\u0026title=)\n- Start a [discussion](https://github.com/superlinked/superlinked/discussions/new/choose) about your ideas or questions\n\nPlease create separate issues/discussions for each topic to help us better address your feedback. Thank you for contributing!\n","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperlinked%2Fsuperlinked","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsuperlinked%2Fsuperlinked","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperlinked%2Fsuperlinked/lists"}