{"id":16294440,"url":"https://github.com/scriptsmith/socialreaper","last_synced_at":"2025-04-04T11:09:26.019Z","repository":{"id":44931067,"uuid":"80514836","full_name":"ScriptSmith/socialreaper","owner":"ScriptSmith","description":"Social media scraping / data collection library for Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs","archived":false,"fork":false,"pushed_at":"2020-06-25T20:23:18.000Z","size":2667,"stargazers_count":548,"open_issues_count":1,"forks_count":94,"subscribers_count":31,"default_branch":"master","last_synced_at":"2024-10-11T20:15:44.281Z","etag":null,"topics":["api","facebook","pinterest","python","reddit","scraping","social-media","tumblr","twitter","youtube"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ScriptSmith.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-31T11:34:05.000Z","updated_at":"2024-10-04T19:45:29.000Z","dependencies_parsed_at":"2022-08-04T02:00:12.285Z","dependency_job_id":null,"html_url":"https://github.com/ScriptSmith/socialreaper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScriptSmith%2Fsocialreaper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScriptSmith%2Fsocialreaper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScriptSmith%2Fsocialreaper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScriptSmith%2Fsocialreaper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ScriptSmith","download_url":"https://codeload.github.com/ScriptSmith/socialreaper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247166168,"owners_count":20894654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","facebook","pinterest","python","reddit","scraping","social-media","tumblr","twitter","youtube"],"created_at":"2024-10-10T20:15:21.920Z","updated_at":"2025-04-04T11:09:26.002Z","avatar_url":"https://github.com/ScriptSmith.png","language":"Python","readme":"# socialreaper\r\n[![](https://readthedocs.org/projects/socialreaper/badge/?version=latest)](https://socialreaper.readthedocs.io)\r\n[![Downloads](http://pepy.tech/badge/socialreaper)](http://pepy.tech/count/socialreaper)\r\n[![Gitter](https://img.shields.io/gitter/room/socialreaper/socialreaper.svg)](https://gitter.im/socialreaper)\r\n\r\n`socialreaper` is a Python 3.6+ library that scrapes Facebook, Twitter, Reddit, Youtube, Pinterest, and Tumblr. \r\n\r\n[Documentation](https://socialreaper.readthedocs.io)\r\n\r\nNot a programmer? [Try the GUI](https://github.com/scriptsmith/reaper)\r\n\r\n# Install\r\n```\r\npip3 install socialreaper\r\n```\r\n\r\n# Examples\r\nFor version 0.3.0 only\r\n\r\n```\r\npip3 install socialreaper==0.3.0\r\n```\r\n\r\n## Facebook\r\nGet the comments from McDonalds' 1000 most recent posts\r\n```python\r\nfrom socialreaper import Facebook\r\n\r\nfbk = Facebook(\"api_key\")\r\n\r\ncomments = fbk.page_posts_comments(\"mcdonalds\", post_count=1000, \r\n    comment_count=100000)\r\n\r\nfor comment in comments:\r\n    print(comment['message'])\r\n```\r\n\r\n## Twitter\r\nSave the 500 most recent tweets from the user `@realDonaldTrump` to a csv file\r\n```python\r\nfrom socialreaper import Twitter\r\nfrom socialreaper.tools import to_csv\r\n\r\ntwt = Twitter(app_key=\"xxx\", app_secret=\"xxx\", oauth_token=\"xxx\", \r\n    oauth_token_secret=\"xxx\")\r\n    \r\ntweets = twt.user(\"realDonaldTrump\", count=500, exclude_replies=True, \r\n    include_retweets=False)\r\n    \r\nto_csv(list(tweets), filename='trump.csv')\r\n\r\n```\r\n\r\n## Reddit\r\nGet the top 10 comments from the top 50 threads of all time on reddit\r\n```python\r\nfrom socialreaper import Reddit\r\nfrom socialreaper.tools import flatten\r\n\r\nrdt = Reddit(\"xxx\", \"xxx\")\r\n \r\ncomments = rdt.subreddit_thread_comments(\"all\", thread_count=50, \r\n    comment_count=500, thread_order=\"top\", comment_order=\"top\", \r\n    search_time_period=\"all\")\r\n    \r\n# Convert nested dictionary into flat dictionary\r\ncomments = [flatten(comment) for comment in comments]\r\n\r\n# Sort by comment score\r\ncomments = sorted(comments, key=lambda k: k['data.score'], reverse=True)\r\n\r\n# Print the top 10\r\nfor comment in comments[:9]:\r\n    print(\"###\\nUser: {}\\nScore: {}\\nComment: {}\\n\".format(comment['data.author'], comment['data.score'], comment['data.body']))\r\n```\r\n\r\n## Youtube\r\nGet the comments containing the strings `prize`, `giveaway` from \r\nyoutube channel `mkbhd`'s videos\r\n```python\r\nfrom socialreaper import Youtube\r\n\r\nytb = Youtube(\"api_key\")\r\n\r\nchannel_id = ytb.api.guess_channel_id(\"mkbhd\")[0]['id']\r\n\r\ncomments = ytb.channel_video_comments(channel_id, video_count=500, \r\n    comment_count=100000, comment_text=[\"prize\", \"giveaway\"], \r\n    comment_format=\"plainText\")\r\n    \r\nfor comment in comments:\r\n    print(comment)\r\n```\r\n\r\n# CSV export\r\nYou can export a list of dictionaries using socialreaper's `CSV` class\r\n\r\n```python\r\nfrom socialreaper import Facebook\r\nfrom socialreaper.tools import CSV\r\n\r\nfbk = Facebook(\"api_key\")\r\nposts = list(fbk.page_posts(\"mcdonalds\"))\r\nCSV(posts, file_name='mcdonalds.csv')\r\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscriptsmith%2Fsocialreaper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscriptsmith%2Fsocialreaper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscriptsmith%2Fsocialreaper/lists"}