{"id":21455507,"url":"https://github.com/hookdeck/index-all-the-things","last_synced_at":"2026-04-12T20:36:04.795Z","repository":{"id":255545130,"uuid":"852341518","full_name":"hookdeck/index-all-the-things","owner":"hookdeck","description":"A Python web application built on Flask that allows an asset with a URL to be analyzed and a textual and embedding representation stored in MongoDB Atlas. Uses Replicate and Hookdeck.","archived":false,"fork":false,"pushed_at":"2024-11-22T10:55:57.000Z","size":9839,"stargazers_count":1,"open_issues_count":6,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-17T03:25:08.876Z","etag":null,"topics":["ai","hookdeck","llm","mongodb","mongodb-atlas","replicate","webhooks"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hookdeck.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-04T16:33:27.000Z","updated_at":"2024-11-22T10:56:02.000Z","dependencies_parsed_at":"2024-11-19T13:33:15.494Z","dependency_job_id":null,"html_url":"https://github.com/hookdeck/index-all-the-things","commit_stats":null,"previous_names":["hookdeck/index-all-the-things"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hookdeck/index-all-the-things","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hookdeck%2Findex-all-the-things","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hookdeck%2Findex-all-the-things/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hookdeck%2Findex-all-the-things/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hookdeck%2Findex-all-the-things/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hookdeck","download_url":"https://codeload.github.com/hookdeck/index-all-the-things/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hookdeck%2Findex-all-the-things/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265844936,"owners_count":23837682,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","hookdeck","llm","mongodb","mongodb-atlas","replicate","webhooks"],"created_at":"2024-11-23T05:12:26.890Z","updated_at":"2026-04-12T20:35:59.737Z","avatar_url":"https://github.com/hookdeck.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Index All The Things!\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"docs/att.png\" height=\"200\" /\u003e\n\u003c/p\u003e\n\nA Python web application built on Flask that allows an asset with a URL to be analyzed and a textual and embedding representation stored in [MongoDB Atlas](https://www.mongodb.com/atlas).\n\nA vector search can then be performed on the embeddings.\n\nThe application uses [Replicate](https://replicate.com) to run AI models and [Hookdeck](https://hookdeck.com?ref=github-iatt) to reliability receive asynchronous results from Replicate.\n\nAt present the application supports analyzing audio assets and getting the transcribed contents. However, there is a framework in place to support other asset types such as text, HTML, images, and video\n\n## How it works\n\nThe following diagram shows the sequence of how assets are submitted within the Flask application and processed by Replicate, and the results sent via webhooks through Hookdeck back to the Flask application and stored in MongoDB.\n\n![Index All The Things Sequence Diagram](docs/sequence-diagram.png)\n\n## Prerequisites\n\n- A free [Hookdeck account](https://dashboard.hookdeck.com/signup?ref=github-iatt)\n- The [Hookdeck CLI installed](https://hookdeck.com/docs/cli?ref=github-iatt)\n- A trial [MongoDB Atlas account](https://www.mongodb.com/cloud/atlas/register)\n- A [Replicate account](https://replicate.com/signin)\n- [Python 3](https://www.python.org/downloads/)\n- [Poetry](https://python-poetry.org/docs/#installation) for package management\n\n## Development setup\n\n### Dependencies\n\nActivate the virtual environment:\n\n```sh\npoetry shell\n```\n\nInstall dependencies:\n\n```sh\npoetry install\n```\n\n### Configuration\n\nCreate a `.env` file with the following configuration, replacing with values as indicated:\n\n```\n# A secret used for signing session cookies\n# https://flask.palletsprojects.com/en/2.3.x/config/#SECRET_KEY\nSECRET_KEY=\"\"\n\n# MongoDB Atlas connection string\nMONGODB_CONNECTION_URI=\"\"\n\n# Hookdeck Project API Key\n# Hookdeck Dashboard -\u003e Settings -\u003e Secrets\nHOOKDECK_PROJECT_API_KEY=\"\"\n\n# Replicate API Token\nREPLICATE_API_TOKEN=\"\"\n\n# Hookdeck Source URLs\n# These will be automatically populated for you in the next step\nAUDIO_WEBHOOK_URL=\"\"\nEMBEDDINGS_WEBHOOK_URL=\"\"\n```\n\nRun the following to create Hookdeck connections to receive webhooks from Replicate:\n\n```sh\npoetry run python create-hookdeck-connections.py\n```\n\nRun the following to create a search indexes within MongoDB:\n\n\u003e [!WARNING]\n\u003e You may need some data within MongoDB before you can create the indexes.\n\n```sh\npoetry run python create-indexes.py\n```\n\n### Run the app\n\nRun the app:\n\n```sh\npoetry run python -m flask --app app --debug run\n```\n\nCreate localtunnels to receive webhooks from the two Hookdeck Connections:\n\n```sh\nhookdeck listen 5000 '*'\n```\n\nNavigate to `localhost:5000` within your web browser.\n\n![Index All The Things home page running in a web browser](docs/iatt-home.png)\n\n![Index All The Things search page with search result running in a web browser](docs/iatt-search.png)\n\n## Learn more\n\n- [Hookdeck docs](https://hookdeck.com/docs?ref=github-iatt)\n- [MongoDB Atlas docs](https://www.mongodb.com/docs/atlas/)\n- [Replicate docs](https://replicate.com/docs/)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhookdeck%2Findex-all-the-things","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhookdeck%2Findex-all-the-things","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhookdeck%2Findex-all-the-things/lists"}