{"id":13593526,"url":"https://github.com/ahmedbesbes/media-agent","last_synced_at":"2025-07-14T18:31:16.498Z","repository":{"id":176921091,"uuid":"652154549","full_name":"ahmedbesbes/media-agent","owner":"ahmedbesbes","description":"Scrape data from social media and chat with it using Langchain","archived":false,"fork":false,"pushed_at":"2023-07-23T08:29:15.000Z","size":601,"stargazers_count":107,"open_issues_count":7,"forks_count":16,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-05-18T19:26:13.879Z","etag":null,"topics":["langchain","large-language-models","llms","nlp","nlproc","python","tweepy"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ahmedbesbes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-11T09:02:35.000Z","updated_at":"2024-08-01T16:38:23.350Z","dependencies_parsed_at":null,"dependency_job_id":"2c1e221a-335f-44e6-a200-e8217d10b77d","html_url":"https://github.com/ahmedbesbes/media-agent","commit_stats":null,"previous_names":["ahmedbesbes/twitter-agent"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fmedia-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fmedia-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fmedia-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fmedia-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ahmedbesbes","download_url":"https://codeload.github.com/ahmedbesbes/media-agent/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225629758,"owners_count":17499295,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["langchain","large-language-models","llms","nlp","nlproc","python","tweepy"],"created_at":"2024-08-01T16:01:21.208Z","updated_at":"2024-11-23T02:16:41.653Z","avatar_url":"https://github.com/ahmedbesbes.png","language":"Python","funding_links":[],"categories":["开源项目"],"sub_categories":["其他聊天机器人"],"readme":"# Media Agent \n\nMedia Agent scrapes Twitter and Reddit submissions, summarizes them, and chats with them in an interactive terminal.\n\n---\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./assets/media_agent.png\" width=\"90%\"\u003e\n\u003c/p\u003e\n\n### Demo\n\u003cp align=\"center\"\u003e\n  https://www.loom.com/share/f4954e7d34ef4b7b8491e2bf910e8521\n\u003c/p\u003e\n\n### Functionalities\n\n- Scrapes tweets/submissions on your behalf either from a list of \nuser accounts or a list of keywords.\n- Embeds the tweets/submissions using OpenAI \n- Indexes the embeddings (i.e. *vectors*) in ChromaDB\n- Enriches the index with additional metadata\n- Creates a summary of the tweets/submissions and provides potential questions to answer\n- Opens a chat session on top of the tweets\n- Saves the conversation with its metadata\n- A rich terminal UI and logging features\n\n\n### Tools and libraries used\n\n* **Langchain** 🦜 to build and compose LLMs\n* **ChromaDB** to store vectors (a.k.a embeddings) and query them to build conversational bots\n* **Tweepy** to connect to your the Twitter API and extract Tweets and metadata\n* **Praw** to connect to Reddit API\n* **Rich** to build a cool terminal UX/UI\n* **Poetry** to manage dependencies\n\n### Third party services\n* OpenAI (🔑 needed)\n* Twitter (🔑 needed)\n* Reddit (🔑 needed)\n\n### Run the app locally\n\n* Install dependencies with poetry\n\n```bash\npoetry install --with dev\n```\n\n* Add API credentials\n\nCreate .env file at the root of the project with the following keys:\n\n```bash\nOPENAI_API_KEY=\u003cOPENAI KEY\u003e\nTWITTER_BEARER_TOKEN=\u003cTWITTER BEARER TOKEN\u003e\nREDDIT_API_CLIENT_ID=\u003cREDDIT_API_CLIENT_ID\u003e\nREDDIT_API_SECRET=\u003cREDDIT_API_SECRET\u003e\nREDDIT_USER_AGENT=\u003cREDDIT_USER_AGENT\u003e\n```\n\nMore info on these credentials [here](https://openai.com/), [here](https://developer.twitter.com/en/docs/apps/overview) and [here](https://www.geeksforgeeks.org/how-to-get-client_id-and-client_secret-for-python-reddit-api-registration/)\n\n* Run the app\n\n```bash\nmake run-media-agent\n```\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=ahmedbesbes/media-agent\u0026type=Timeline)](https://star-history.com/#ahmedbesbes/media-agent\u0026Timeline)\n\n\n### Future features:\n\nThis is an on-going project, so feel free to contribute:\n\nHere's what I plan in the near future:\n\n* Add more data sources: substack, press, LinkedIN\n* support open-source LLMs (Falcon?)\n* support Pinecone in addition to ChromaDB\n* add instructions to deployment on a cloud infrastructure\n* improve the prompts to have a more engaging/enriching conversations\n* add actions to open URLs and fetch content from it\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedbesbes%2Fmedia-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fahmedbesbes%2Fmedia-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedbesbes%2Fmedia-agent/lists"}