{"id":15697962,"url":"https://github.com/abdbarho/rerecall","last_synced_at":"2025-04-30T14:41:26.094Z","repository":{"id":241530106,"uuid":"806175979","full_name":"AbdBarho/ReRecall","owner":"AbdBarho","description":"ReRecall is a PoC for an open-source implementation of Microsoft's Windows Recall","archived":false,"fork":false,"pushed_at":"2024-05-26T16:57:57.000Z","size":394,"stargazers_count":24,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-30T17:04:46.263Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AbdBarho.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-26T15:31:44.000Z","updated_at":"2025-02-24T21:27:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"21b56b3c-b31f-4f32-b3ce-84947dc316f6","html_url":"https://github.com/AbdBarho/ReRecall","commit_stats":null,"previous_names":["abdbarho/rerecall"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbdBarho%2FReRecall","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbdBarho%2FReRecall/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbdBarho%2FReRecall/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbdBarho%2FReRecall/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AbdBarho","download_url":"https://codeload.github.com/AbdBarho/ReRecall/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251722807,"owners_count":21633019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T19:22:24.871Z","updated_at":"2025-04-30T14:41:26.072Z","avatar_url":"https://github.com/AbdBarho.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ReRecall\n\nReRecall is a proof of concept for an open source implementation of Microsoft's Recall feature. It periodically takes screenshots of the user's desktop and stores them in a database. The user can then search through these screenshots using a text query.\n\n**THIS IS A PROOF OF CONCEPT, EXPECT A LOT OF PROBLEMS.**\n\n\n### Requirements\n\nYou need a modern python version and a docker installation with GPU support, the `docker-compose.yml` file is configured for nvidia gpus. I have ran this on a Windows 10 with a GTX 1060. No part of the code is os-specific so it should work fine on other systems. Feel free to play around.\n\n### Setup\n\n\nClone the repository\n\n```bash\ngit clone https://github.com/AbdBarho/ReRecall.git\ncd ReRecall\n```\n\nStart Ollama\n```bash\ndocker compose up\n```\n\nOpen another terminal, and setup a virtual environment and install the requirements:\n\n```bash\npython -m venv .venv\nsource .venv/bin/activate\n# or .\\.venv\\Scripts\\Activate.ps1 on Windows Powershell\npip install -r requirements.txt\n```\n\nNow you can start the recording\n```bash\npython rerecall/record.py\n```\nBy default it will capture screenshots of all monitors every 60 seconds, see `rerecall/config.py` for all options you can change.\n\n\n### Search\n\nAfter recording some data, you can search through it using the search script. Ollama should be running in the background.\n\n```bash\npython rerecall/query.py -n 1 my search query\n```\nwhere `-n 1` is the number of results you want to see, and `my search query` is the text you want to search for.\n\n\n### Notes\n\n\nThe current implementation is very primitive, it takes screenshots, uses `ollama/llava-phi3:3.8b-mini-q4_0` to generate descriptions of the screenshots, and uses `ollama/mxbai-embed-large` to generate embedding for vector search, and stores them in a chroma db database.\n\nFor the search part, the embedding is done on the query text and matches against existing embeddings in the database.\n\nYou can change the models used for the description and embedding in `rerecall/config.py`.\n\nThere is definitely room for improvement here:\n* There are plenty of hallucinations in the generated descriptions of screenshots, this could be a combination of the size the MLLM used to generate the descriptions (I use a very small model), or because the screenshots are very high in resolution.\n* The search is very basic, it just matches the embeddings of the query text with the embeddings of the screenshots, a potential improvement could be to use the model to enrich the user query with more information before embedding it for search.\n* I am fairly certain that Microsoft does not rely solely on screenshots as I do, but also captures of individual app windows, and also extracts meta information like window title \u0026 others, these could definitely improve the results.\n\n### Results\n\nCherry picked, ignore the vertical monitor.\n\n![image](./example.jpg)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabdbarho%2Frerecall","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabdbarho%2Frerecall","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabdbarho%2Frerecall/lists"}