{"id":18493794,"url":"https://github.com/neurocode-io/alexis","last_synced_at":"2025-04-08T22:30:55.069Z","repository":{"id":49037194,"uuid":"358703588","full_name":"neurocode-io/alexis","owner":"neurocode-io","description":"Alexis is an app that uses RedisAI and RediSearch to retrieve information from a corpus (knowledge source) in response to a query.","archived":false,"fork":false,"pushed_at":"2023-09-10T08:23:40.000Z","size":35749,"stargazers_count":2,"open_issues_count":2,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-23T18:52:18.613Z","etag":null,"topics":["ai","onnx","react","redis","redisai","redisjson","redissearch","redisstreams","typescript","valtio"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/neurocode-io.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-04-16T19:45:20.000Z","updated_at":"2025-02-25T17:27:40.000Z","dependencies_parsed_at":"2022-09-20T08:00:36.767Z","dependency_job_id":null,"html_url":"https://github.com/neurocode-io/alexis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neurocode-io%2Falexis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neurocode-io%2Falexis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neurocode-io%2Falexis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neurocode-io%2Falexis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/neurocode-io","download_url":"https://codeload.github.com/neurocode-io/alexis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247939915,"owners_count":21021870,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","onnx","react","redis","redisai","redisjson","redissearch","redisstreams","typescript","valtio"],"created_at":"2024-11-06T13:16:04.286Z","updated_at":"2025-04-08T22:30:50.061Z","avatar_url":"https://github.com/neurocode-io.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Alexis\n\nAlexis is an app that uses RedisAI and RediSearch to retrieve information from a corpus (indexed library) in response to a query. Essentially, it's a text ranker that uses natural language processing combined with the BM25 ranking function to retrieve and answer questions in response to user queries.\n\nWe all are familiar with keyword search, also called keyword querying, where a user types a few query terms into a search box and gets back results containing representations of the indexed sources.\n\nAlexis extends this basic keyword search by providing a more natural way of searching indexed sources. Furthermore, it aims to identify a span of text that directly answers the query instead of returning a list of documents (or extracts of documents).\n\n## Screenshots\n\n![Login](https://github.com/neurocode-io/alexis/raw/main/docs/login.png)\n![How it works](https://github.com/neurocode-io/alexis/raw/main/docs/answer.png)\n\n## Architecture\n\nWe break the challenge of finding the correct answer to a user query into two steps:\n\n1. First we select the text that is likely to contain answers. We use RediSearch with the BM25 ranking function for this step.\n2. We use a Transformer AI model loaded into RedisAI to identify the answer spans in the text.\n\nUsing RediSearch in the first step, we drastically reduce the search space, thus making the app's overall experience faster.\n\n\nWe use NodeJS with typescript in the backend and React with typescript in the frontend. Besides RedisAI and RediSearch we also use RedisJSON for our user model and an asynchronous worker implemented with Redis Streams.\n\nWe expose a web server with the express framework that has the following endpoints:\n\n```\nPOST /v1/users\nPOST /v1/login\nPOST /v1/logout\nGET /v1/me\n\nPOST /pdf  (pdfUpload)\nPOST /v1/query\n```\n\nOnce a user registers and logs into the app. He can start adding documents to his indexed library.\nA PDF upload writes an event to redis stream. A consumer from the consumer group picks up the event for async processing. The consumer processes the PDF, applies some cleaning and stores the PDF in a Redis hash that is indexed with RediSearch.\n\nAfterward the user can send natural queries to the server and is not confined to basic keyword search such as \"kubernetes deployments\", \"DDD root aggregate\" etc.. but can query with more relevance such as \"how do kubernetes deployments get updated?\", \"What is the role of a root aggregate in DDD\"\n\n\n## Flowchart\n\nA general overview of what Alexis does can be represented as follows:\n\n![general overview](https://github.com/neurocode-io/alexis/raw/main/docs/general-overview.png)\n\nNow let's break down how the *Upload PDFs \u0026 Index PDF Content* and the *Answer Query* parts of the flowchart operate! \n\n### Upload PDFs \u0026 Index PDF Content\n\n![upload and index](https://github.com/neurocode-io/alexis/raw/main/docs/upload-and-index.png)\n\n### Answer Query\n\n![answer query](https://github.com/neurocode-io/alexis/raw/main/docs/answer-query.png)\n\n\n\n# How it works?\n\n## 1. How the data is stored\n\n1. The user data is stored in a RedisJSON:\n\n   ```\n   {\n    firstName: string\n    lastName: string\n    email: string\n    password: string\n    pdfs: Array\u003c{id: string, fileName: string}\u003e\n   }\n   ```\n\n2. A RediSearch index is created for each user with:\n   \n   ```\n   FT.CREATE ax:idx:\u003cuserId\u003e on HASH PREFIX 1 ax:pdfs:\u003cuserId\u003e SCHEMA content TEXT PHONETIC dm:en\n   ```\n\n3. Once a user uploads a PDF we update his pdfs array with RedisJSON:\n   ```\n   JSON.ARRAPPEND ax:users:\u003cuserId\u003e .pdfs {id: pdfId, fileName: \u003cuploadedPdf\u003e}\n   ```\n\n\n4. The file upload also triggers an event being written to the **ax:stream:pdf-processing** stream. The payload of the stream is\n   \n   ```\n   {\n     id: string,\n     fileName: string\n   }\n   ```\n\n4. A consumer within a consumer group picks this event off the stream and processess the file and writes the content in a hash:\n\n   ```\n   HSET ax:pdfs:\u003cuserId\u003e.\u003cparagraph\u003e content \u003ccleanedParagraphBlock\u003e fileName \u003cpdfFileName\u003e\n   ```\n\n## 2. How the data is accessed\n\n1. As mentioned earlier. We have an RediSearch index for each user that indexes this hash and provides lookup capabilities to find relevant content given a user query. We look up content with:\n\n   ```\n   FT.SEARCH ax:idx:\u003cuserId\u003e '@content:\u003cuserQuery\u003e' SCORER BM25 WITHSCORES LIMIT 0 4\n   ```\n\n2. The returned content from RediSearch we feed into the AI model that is being surfed with RedisAI!\n\n\n\n## How to run it locally?\n\n### Prerequisites\n- Node - v12.x.x\n- NPM - v6.x.x\n- Docker and docker-compose\n\n### Commands\n\n1. npm install\n2. npm run bootstrap \n3. npm start\n\n\nThe app: http://localhost:3000\n\nRedis Insight: http://localhost:8001\n\n## Test\n\n```\nnpm test\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneurocode-io%2Falexis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneurocode-io%2Falexis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneurocode-io%2Falexis/lists"}