{"id":14978893,"url":"https://github.com/shaikhsajid1111/twitter-scraper-selenium","last_synced_at":"2025-04-07T05:07:26.900Z","repository":{"id":37980087,"uuid":"400134442","full_name":"shaikhsajid1111/twitter-scraper-selenium","owner":"shaikhsajid1111","description":"Python's package to scrap Twitter's front-end easily ","archived":false,"fork":false,"pushed_at":"2024-09-07T14:54:39.000Z","size":130,"stargazers_count":338,"open_issues_count":35,"forks_count":54,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-31T04:04:11.791Z","etag":null,"topics":["automation","contribution-welcome","csv","hacktoberfest","json","open-source","pypi","python","python3","selenium","social-media","tweets","twitter","twitter-api","twitter-bot","twitter-hashtag","twitter-profile","twitter-profiles","twitter-scraper","web-scraping"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/twitter-scraper-selenium","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shaikhsajid1111.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-26T10:49:23.000Z","updated_at":"2025-03-11T02:59:45.000Z","dependencies_parsed_at":"2024-09-28T02:05:05.647Z","dependency_job_id":"86dd80e3-612c-4ed5-9b46-a345e08ea6b6","html_url":"https://github.com/shaikhsajid1111/twitter-scraper-selenium","commit_stats":{"total_commits":118,"total_committers":6,"mean_commits":"19.666666666666668","dds":"0.14406779661016944","last_synced_commit":"05010d432d82103259109c2fad0dc0678ef995d5"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shaikhsajid1111%2Ftwitter-scraper-selenium","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shaikhsajid1111%2Ftwitter-scraper-selenium/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shaikhsajid1111%2Ftwitter-scraper-selenium/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shaikhsajid1111%2Ftwitter-scraper-selenium/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shaikhsajid1111","download_url":"https://codeload.github.com/shaikhsajid1111/twitter-scraper-selenium/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247595334,"owners_count":20963943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","contribution-welcome","csv","hacktoberfest","json","open-source","pypi","python","python3","selenium","social-media","tweets","twitter","twitter-api","twitter-bot","twitter-hashtag","twitter-profile","twitter-profiles","twitter-scraper","web-scraping"],"created_at":"2024-09-24T13:58:35.602Z","updated_at":"2025-04-07T05:07:26.865Z","avatar_url":"https://github.com/shaikhsajid1111.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003ch1\u003e Twitter scraper selenium \u003c/h1\u003e\n\u003cp\u003e Python's package to scrape Twitter's front-end easily with selenium.  \u003c/p\u003e\n\n\n[![PyPI license](https://img.shields.io/pypi/l/ansicolortags.svg)](https://opensource.org/licenses/MIT) [![Python \u003e=3.8](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/release/python-360/)\n[![Maintenance](https://img.shields.io/badge/Maintained-Yes-green.svg)](https://github.com/shaikhsajid1111/facebook_page_scraper/graphs/commit-activity)\n\n\u003c!--TABLE of contents--\u003e\n\u003ch2\u003e Table of Contents \u003c/h2\u003e\n\u003cdetails open=\"open\"\u003e\n  \u003csummary\u003eTable of Contents\u003c/summary\u003e\n  \u003col\u003e\n    \u003cli\u003e\n      \u003ca href=\"#getting-started\"\u003eGetting Started\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#Prerequisites\"\u003ePrerequisites\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#Installation\"\u003eInstallation\u003c/a\u003e\n        \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#sourceInstallation\"\u003eInstalling from source\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#pypiInstallation\"\u003eInstalling with PyPI\u003c/a\u003e\u003c/li\u003e\n        \u003c/ul\u003e\n        \u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#Usage\"\u003eUsage\u003c/a\u003e\n    \u003cul\u003e\u003cli\u003e\u003ca href=\"#availableFunction\"\u003eAvailable Functions in this package- Summary\u003c/a\u003e\u003c/li\u003e\u003c/ul\u003e\n    \u003cul\u003e\u003cli\u003e\u003ca href=\"#profileDetail\"\u003eScraping profile's details\u003c/a\u003e\n    \u003cul\u003e\n    \u003cli\u003e\u003ca href=\"#profileDetailExample\"\u003eIn JSON Format - Example\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#profileDetailArgument\"\u003eFunction Argument\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#profileDetailKeys\"\u003eKeys of the output\u003c/a\u003e\u003c/li\u003e\n    \u003c/ul\u003e\n    \u003c/li\u003e\u003c/ul\u003e\n    \u003c!----\u003e\n    \u003cul\u003e\n    \u003cli\u003e\u003ca href=\"#profile\"\u003eScraping profile's tweets\u003c/a\u003e\n    \u003cul\u003e\n    \u003cli\u003e\u003ca href=\"#profileJson\"\u003eIn JSON format - Example\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#profileCSV\"\u003eIn CSV format - Example\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#profileArgument\"\u003eFunction Arguments\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#profileOutput\"\u003eKeys of the output data\u003c/a\u003e\u003c/li\u003e\n    \u003c/ul\u003e\n    \u003cli\u003e\u003ca href='#to-scrape-user-tweets-with-api'\u003eScraping user's tweet using API\u003c/a\u003e\u003c/li\u003e\n    \u003cul\u003e\n    \u003cli\u003e\u003ca href='#to-scrape-user-tweets-with-api'\u003eIn JSON format - Example\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href='#users_api_parameter'\u003eFunction Arguments\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href='#scrape_user_with_api_args_keys'\u003eKeys of the output\u003c/a\u003e\u003c/li\u003e\n    \u003c/ul\u003e\n    \u003cli\u003e\u003ca href=\"#proxy\"\u003eUsing scraper with proxy\u003c/a\u003e\n    \u003cul\u003e\n    \u003cli\u003e\u003ca href=\"#unauthenticatedProxy\"\u003eUnauthenticated Proxy\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#authenticatedProxy\"\u003eAuthenticated Proxy\u003c/a\u003e\u003c/li\u003e\n    \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003c/li\u003e\n    \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#privacy\"\u003ePrivacy\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#license\"\u003eLicense\u003c/a\u003e\u003c/li\u003e\n  \u003c/ol\u003e\n\u003c/details\u003e\n\n\u003c!--TABLE of contents //--\u003e\n\u003cbr\u003e\n\u003chr\u003e\n\u003ch2 id=\"Prerequisites\"\u003ePrerequisites \u003c/h2\u003e\n\u003cli\u003e Internet Connection \u003c/li\u003e\n\u003cli\u003e Python 3.6+ \u003c/li\u003e\n\u003cli\u003e Chrome or Firefox browser installed on your machine \u003c/li\u003e\n\u003chr\u003e\n\u003ch2 id=\"Installation\"\u003e Installation \u003c/h2\u003e\n\u003ch3 id=\"sourceInstallation\"\u003eInstalling from the source\u003c/h3\u003e\n\u003cp\u003eDownload the source code or clone it with:\u003cp\u003e\n\n```\ngit clone https://github.com/shaikhsajid1111/twitter-scraper-selenium\n```\n\n\u003cp\u003eOpen terminal inside the downloaded folder:\u003c/p\u003e\n\n\u003cbr\u003e\n\n```\n python3 setup.py install\n```\n\n\u003ch3 id=\"pypiInstallation\"\u003e\nInstalling with \u003ca href=\"https://pypi.org\"\u003ePyPI\u003c/a\u003e\n\u003c/h3\u003e\n\n```\npip3 install twitter-scraper-selenium\n```\n\n\u003chr\u003e\n\u003ch2 id=\"Usage\"\u003e\nUsage\u003c/h2\u003e\n\u003ch3 id=\"availableFunction\"\u003eAvailable Function In this Package - Summary\u003c/h3\u003e\n\u003cdiv\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003ctd\u003eFunction Name\u003c/td\u003e\n\u003ctd\u003eFunction Description\u003c/td\u003e\n\u003ctd\u003eScraping Method\u003c/td\u003e\n\u003ctd\u003eScraping Speed\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003escrape_profile()\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eScrape's Twitter user's profile tweets\u003c/td\u003e\n\u003ctd\u003eBrowser Automation\u003c/td\u003e\n\u003ctd\u003eSlow\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003eget_profile_details()\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eScrape's Twitter user details.\u003c/td\u003e\n\u003ctd\u003eHTTP Request\u003c/td\u003e\n\u003ctd\u003eFast\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003escrape_profile_with_api()\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eScrape's Twitter tweets by twitter profile username. It expects the username of the profile\u003c/td\u003e\n\u003ctd\u003eBrowser Automation \u0026 HTTP Request\u003c/td\u003e\n\u003ctd\u003eFast\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\u003cp\u003e\nNote: HTTP Request Method sends the request to Twitter's API directly for scraping data, and Browser Automation visits that page, scroll while collecting the data.\u003c/p\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\u003chr\u003e\n\u003ch3 id=\"profileDetail\"\u003eTo scrape twitter profile details:\u003c/h3\u003e\n\u003cdiv id=\"profileDetailExample\"\u003e\n\n```python\nfrom twitter_scraper_selenium import get_profile_details\n\ntwitter_username = \"TwitterAPI\"\nfilename = \"twitter_api_data\"\nbrowser = \"firefox\"\nheadless = True\nget_profile_details(twitter_username=twitter_username, filename=filename, browser=browser, headless=headless)\n\n```\nOutput:\n```js\n{\n\t\"id\": 6253282,\n\t\"id_str\": \"6253282\",\n\t\"name\": \"Twitter API\",\n\t\"screen_name\": \"TwitterAPI\",\n\t\"location\": \"San Francisco, CA\",\n\t\"profile_location\": null,\n\t\"description\": \"The Real Twitter API. Tweets about API changes, service issues and our Developer Platform. Don't get an answer? It's on my website.\",\n\t\"url\": \"https:\\/\\/t.co\\/8IkCzCDr19\",\n\t\"entities\": {\n\t\t\"url\": {\n\t\t\t\"urls\": [{\n\t\t\t\t\"url\": \"https:\\/\\/t.co\\/8IkCzCDr19\",\n\t\t\t\t\"expanded_url\": \"https:\\/\\/developer.twitter.com\",\n\t\t\t\t\"display_url\": \"developer.twitter.com\",\n\t\t\t\t\"indices\": [\n\t\t\t\t\t0,\n\t\t\t\t\t23\n\t\t\t\t]\n\t\t\t}]\n\t\t},\n\t\t\"description\": {\n\t\t\t\"urls\": []\n\t\t}\n\t},\n\t\"protected\": false,\n\t\"followers_count\": 6133636,\n\t\"friends_count\": 12,\n\t\"listed_count\": 12936,\n\t\"created_at\": \"Wed May 23 06:01:13 +0000 2007\",\n\t\"favourites_count\": 31,\n\t\"utc_offset\": null,\n\t\"time_zone\": null,\n\t\"geo_enabled\": null,\n\t\"verified\": true,\n\t\"statuses_count\": 3656,\n\t\"lang\": null,\n\t\"contributors_enabled\": null,\n\t\"is_translator\": null,\n\t\"is_translation_enabled\": null,\n\t\"profile_background_color\": null,\n\t\"profile_background_image_url\": null,\n\t\"profile_background_image_url_https\": null,\n\t\"profile_background_tile\": null,\n\t\"profile_image_url\": null,\n\t\"profile_image_url_https\": \"https:\\/\\/pbs.twimg.com\\/profile_images\\/942858479592554497\\/BbazLO9L_normal.jpg\",\n\t\"profile_banner_url\": null,\n\t\"profile_link_color\": null,\n\t\"profile_sidebar_border_color\": null,\n\t\"profile_sidebar_fill_color\": null,\n\t\"profile_text_color\": null,\n\t\"profile_use_background_image\": null,\n\t\"has_extended_profile\": null,\n\t\"default_profile\": false,\n\t\"default_profile_image\": false,\n\t\"following\": null,\n\t\"follow_request_sent\": null,\n\t\"notifications\": null,\n\t\"translator_type\": null\n}\n```\n\u003c/div\u003e\n\u003cbr\u003e\n\u003cdiv id=\"profileDetailArgument\"\u003e\n\u003cp\u003e\u003ccode\u003eget_profile_details()\u003c/code\u003e arguments:\u003c/p\u003e\n\n\u003ctable\u003e\n    \u003cthead\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eArgument\u003c/td\u003e\n            \u003ctd\u003eArgument Type\u003c/td\u003e\n            \u003ctd\u003eDescription\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody\u003e\n        \u003ctr\u003e\n            \u003ctd\u003etwitter_username\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eTwitter Username\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eoutput_filename\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eWhat should be the filename where output is stored?.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eoutput_dir\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eWhat directory output file should be saved?\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eproxy\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eOptional parameter, if user wants to use proxy for scraping. If the proxy is authenticated proxy then the proxy format is username:password@host:port.\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003c/div\u003e\n\u003chr\u003e\n\u003cbr\u003e\n\u003cdiv\u003e\n\u003ch4 id=\"profileDetailKeys\"\u003eKeys of the output:\u003c/p\u003e\nDetail of each key can be found \u003ca href=\"https://developer.twitter.com/en/docs/twitter-api/v1/data-dictionary/object-model/user\"\u003ehere\u003c/a\u003e.\u003c/h4\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\u003chr\u003e\n\u003ch3 id=\"profile\"\u003eTo scrape profile's tweets:\u003c/h3\u003e\n\u003cp id=\"profileJson\"\u003eIn JSON format:\u003c/p\u003e\n\n```python\nfrom twitter_scraper_selenium import scrape_profile\n\nmicrosoft = scrape_profile(twitter_username=\"microsoft\",output_format=\"json\",browser=\"firefox\",tweets_count=10)\nprint(microsoft)\n```\nOutput:\n```javascript\n{\n  \"1430938749840629773\": {\n    \"tweet_id\": \"1430938749840629773\",\n    \"username\": \"Microsoft\",\n    \"name\": \"Microsoft\",\n    \"profile_picture\": \"https://twitter.com/Microsoft/photo\",\n    \"replies\": 29,\n    \"retweets\": 58,\n    \"likes\": 453,\n    \"is_retweet\": false,\n    \"retweet_link\": \"\",\n    \"posted_time\": \"2021-08-26T17:02:38+00:00\",\n    \"content\": \"Easy to use and efficient for all \\u2013 Windows 11 is committed to an accessible future.\\n\\nHere's how it empowers everyone to create, connect, and achieve more: https://msft.it/6009X6tbW \",\n    \"hashtags\": [],\n    \"mentions\": [],\n    \"images\": [],\n    \"videos\": [],\n    \"tweet_url\": \"https://twitter.com/Microsoft/status/1430938749840629773\",\n    \"link\": \"https://blogs.windows.com/windowsexperience/2021/07/01/whats-coming-in-windows-11-accessibility/?ocid=FY22_soc_omc_br_tw_Windows_AC\"\n  },...\n}\n```\n\u003chr\u003e\n\u003cp id=\"profileCSV\"\u003eIn CSV format:\u003c/p\u003e\n\n```python\nfrom twitter_scraper_selenium import scrape_profile\n\n\nscrape_profile(twitter_username=\"microsoft\",output_format=\"csv\",browser=\"firefox\",tweets_count=10,filename=\"microsoft\",directory=\"/home/user/Downloads\")\n\n\n```\n\nOutput:\n\u003cbr\u003e\n\u003ctable class=\"table table-bordered table-hover table-condensed\" style=\"line-height: 14px;overflow:hidden;white-space: nowrap\"\u003e\n\u003cthead\u003e\u003ctr\u003e\u003cth title=\"Field #1\"\u003etweet_id\u003c/th\u003e\n\u003cth title=\"Field #2\"\u003eusername\u003c/th\u003e\n\u003cth title=\"Field #3\"\u003ename\u003c/th\u003e\n\u003cth title=\"Field #4\"\u003eprofile_picture\u003c/th\u003e\n\u003cth title=\"Field #5\"\u003ereplies\u003c/th\u003e\n\u003cth title=\"Field #6\"\u003eretweets\u003c/th\u003e\n\u003cth title=\"Field #7\"\u003elikes\u003c/th\u003e\n\u003cth title=\"Field #8\"\u003eis_retweet\u003c/th\u003e\n\u003cth title=\"Field #9\"\u003eretweet_link\u003c/th\u003e\n\u003cth title=\"Field #10\"\u003eposted_time\u003c/th\u003e\n\u003cth title=\"Field #11\"\u003econtent\u003c/th\u003e\n\u003cth title=\"Field #12\"\u003ehashtags\u003c/th\u003e\n\u003cth title=\"Field #13\"\u003ementions\u003c/th\u003e\n\u003cth title=\"Field #14\"\u003eimages\u003c/th\u003e\n\u003cth title=\"Field #15\"\u003evideos\u003c/th\u003e\n\u003cth title=\"Field #16\"\u003epost_url\u003c/th\u003e\n\u003cth title=\"Field #17\"\u003elink\u003c/th\u003e\n\u003c/tr\u003e\u003c/thead\u003e\n\u003ctbody\u003e\u003ctr\u003e\n\u003ctd\u003e1430938749840629773\u003c/td\u003e\n\u003ctd\u003eMicrosoft\u003c/td\u003e\n\u003ctd\u003eMicrosoft\u003c/td\u003e\n\u003ctd\u003ehttps://twitter.com/Microsoft/photo\u003c/td\u003e\n\u003ctd align=\"right\"\u003e64\u003c/td\u003e\n\u003ctd align=\"right\"\u003e75\u003c/td\u003e\n\u003ctd align=\"right\"\u003e521\u003c/td\u003e\n\u003ctd\u003eFalse\u003c/td\u003e\n\u003ctd\u003e \u003c/td\u003e\n\u003ctd\u003e2021-08-26T17:02:38+00:00\u003c/td\u003e\n\u003ctd\u003eEasy to use and efficient for all – Windows 11 is committed to an accessible future.\u003cbr/\u003e\u003cbr/\u003eHere\u0026#39;s how it empowers everyone to create, connect, and achieve more: https://msft.it/6009X6tbW \u003c/td\u003e\n\u003ctd\u003e[]\u003c/td\u003e\n\u003ctd\u003e[]\u003c/td\u003e\n\u003ctd\u003e[]\u003c/td\u003e\n\u003ctd\u003e[]\u003c/td\u003e\n\u003ctd\u003ehttps://twitter.com/Microsoft/status/1430938749840629773\u003c/td\u003e\n\u003ctd\u003ehttps://blogs.windows.com/windowsexperience/2021/07/01/whats-coming-in-windows-11-accessibility/?ocid=FY22_soc_omc_br_tw_Windows_AC\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e...\u003c/p\u003e\n\n\u003cbr\u003e\u003chr\u003e\n\u003cdiv id=\"profileArgument\"\u003e\n\u003cp\u003e\u003ccode\u003escrape_profile()\u003c/code\u003e arguments:\u003c/p\u003e\n\n\u003ctable\u003e\n    \u003cthead\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eArgument\u003c/td\u003e\n            \u003ctd\u003eArgument Type\u003c/td\u003e\n            \u003ctd\u003eDescription\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody\u003e\n        \u003ctr\u003e\n            \u003ctd\u003etwitter_username\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eTwitter username of the account\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003ebrowser\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eWhich browser to use for scraping?, Only 2 are supported Chrome and Firefox. Default is set to Firefox\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eproxy\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eOptional parameter, if user wants to use proxy for scraping. If the proxy is authenticated proxy then the proxy format is username:password@host:port.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003etweets_count\u003c/td\u003e\n            \u003ctd\u003eInteger\u003c/td\u003e\n            \u003ctd\u003eNumber of posts to scrape. Default is 10.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eoutput_format\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eThe output format, whether JSON or CSV. Default is JSON.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003efilename\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eIf output parameter is set to CSV, then it is necessary for filename parameter to passed. If not passed then the filename will be same as username passed.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003edirectory\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eIf output_format parameter is set to CSV, then it is valid for directory parameter to be passed. If not passed then CSV file will be saved in current working directory.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eheadless\u003c/td\u003e\n            \u003ctd\u003eBoolean\u003c/td\u003e\n            \u003ctd\u003eWhether to run crawler headlessly?. Default is \u003ccode\u003eTrue\u003c/code\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003c/div\u003e\n\u003chr\u003e\n\u003cbr\u003e\n\u003cdiv id=\"profileOutput\"\u003e\n\u003cp\u003eKeys of the output\u003c/p\u003e\n\n\u003ctable\u003e\n    \u003cthead\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eKey\u003c/td\u003e\n            \u003ctd\u003eType\u003c/td\u003e\n            \u003ctd\u003eDescription\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody\u003e\n        \u003ctr\u003e\n            \u003ctd\u003etweet_id\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003ePost Identifier(integer casted inside string)\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eusername\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eUsername of the profile\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003ename\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eName of the profile\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eprofile_picture\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eProfile Picture link\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003ereplies\u003c/td\u003e\n            \u003ctd\u003eInteger\u003c/td\u003e\n            \u003ctd\u003eNumber of replies of tweet\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eretweets\u003c/td\u003e\n            \u003ctd\u003eInteger\u003c/td\u003e\n            \u003ctd\u003eNumber of retweets of tweet\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003elikes\u003c/td\u003e\n            \u003ctd\u003eInteger\u003c/td\u003e\n            \u003ctd\u003eNumber of likes of tweet\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eis_retweet\u003c/td\u003e\n            \u003ctd\u003eboolean\u003c/td\u003e\n            \u003ctd\u003eIs the tweet a retweet?\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eretweet_link\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eIf it is retweet, then the retweet link else it'll be empty string\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eposted_time\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eTime when tweet was posted in ISO 8601 format\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003econtent\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003econtent of tweet as text\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003ehashtags\u003c/td\u003e\n            \u003ctd\u003eArray\u003c/td\u003e\n            \u003ctd\u003eHashtags presents in tweet, if they're present in tweet\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003ementions\u003c/td\u003e\n            \u003ctd\u003eArray\u003c/td\u003e\n            \u003ctd\u003eMentions presents in tweet, if they're present in tweet\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eimages\u003c/td\u003e\n            \u003ctd\u003eArray\u003c/td\u003e\n            \u003ctd\u003eImages links, if they're present in tweet\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003evideos\u003c/td\u003e\n            \u003ctd\u003eArray\u003c/td\u003e\n            \u003ctd\u003eVideos links, if they're present in tweet\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003etweet_url\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eURL of the tweet\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003elink\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eIf any link is present inside tweet for some external website. \u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\u003chr\u003e\n\u003cdiv id=\"to-scrape-user-tweets-with-api\"\u003e\n\n\u003cp\u003eTo Scrap profile's tweets with API:\u003c/p\u003e\n\n```python\nfrom twitter_scraper_selenium import scrape_profile_with_api\n\nscrape_profile_with_api('elonmusk', output_filename='musk', tweets_count= 100)\n```\n\u003c/div\u003e\n\u003cbr\u003e\n\u003cdiv id=\"users_api_parameter\"\u003e\n\u003cp\u003e\u003ccode\u003escrape_profile_with_api()\u003c/code\u003e Arguments:\u003cp\u003e\n\u003ctable\u003e\n    \u003cthead\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eArgument\u003c/td\u003e\n            \u003ctd\u003eArgument Type\u003c/td\u003e\n            \u003ctd\u003eDescription\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eusername\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eTwitter's Profile username\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003etweets_count\u003c/td\u003e\n            \u003ctd\u003eInteger\u003c/td\u003e\n            \u003ctd\u003eNumber of tweets to scrape.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eoutput_filename\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eWhat should be the filename where output is stored?.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eoutput_dir\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eWhat directory output file should be saved?\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eproxy\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eOptional parameter, if user wants to use proxy for scraping. If the proxy is authenticated proxy then the proxy format is username:password@host:port.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003ebrowser\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eWhich browser to use for extracting out graphql key. Default is firefox.\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eheadless\u003c/td\u003e\n            \u003ctd\u003eString\u003c/td\u003e\n            \u003ctd\u003eWhether to run browser in headless mode?\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\u003cdiv id=\"scrape_user_with_api_args_keys\"\u003e \u003cp\u003eOutput:\u003cp\u003e\n\n```js\n{\n  \"1608939190548598784\": {\n    \"tweet_url\" : \"https://twitter.com/elonmusk/status/1608939190548598784\",\n    \"tweet_details\":{\n      ...\n    },\n    \"user_details\":{\n      ...\n    }\n  }, ...\n}\n```\n\n\u003c/div\u003e\n\u003cbr\u003e\n\u003chr\u003e\n\u003c/div\u003e\n\n\u003ch3 id=\"proxy\"\u003e Using scraper with proxy (http proxy) \u003c/h3\u003e\n\n\u003cdiv id=\"unauthenticatedProxy\"\u003e\n\u003cp\u003eJust pass \u003ccode\u003eproxy\u003c/code\u003e argument to function.\u003c/p\u003e\n\n```python\nfrom twitter_scraper_selenium import scrape_profile\n\nscrape_profile(\"elonmusk\", headless=False, proxy=\"66.115.38.247:5678\", output_format=\"csv\",filename=\"musk\") #In IP:PORT format\n\n```\n\u003c/div\u003e\n\n\u003cbr\u003e\n\u003cdiv id=\"authenticatedProxy\"\u003e\n\u003cp\u003e Proxy that requires authentication: \u003c/p\u003e\n\n```python\n\nfrom twitter_scraper_selenium import scrape_profile\n\nmicrosoft_data = scrape_profile(twitter_username=\"microsoft\", browser=\"chrome\", tweets_count=10, output=\"json\",\n                      proxy=\"sajid:pass123@66.115.38.247:5678\")  #  username:password@IP:PORT\nprint(microsoft_data)\n\n\n```\n\n\u003c/div\u003e\n\u003cbr\u003e\n\u003chr\u003e\n\u003cdiv id=\"privacy\"\u003e\n\u003ch2\u003ePrivacy\u003c/h2\u003e\n\n\u003cp\u003e\nThis scraper only scrapes public data available to unauthenticated user and does not holds the capability to scrape anything private.\n\u003c/p\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\u003chr\u003e\n\u003cdiv id=\"license\"\u003e\n\u003ch2\u003eLICENSE\u003c/h2\u003e\n\nMIT\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshaikhsajid1111%2Ftwitter-scraper-selenium","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshaikhsajid1111%2Ftwitter-scraper-selenium","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshaikhsajid1111%2Ftwitter-scraper-selenium/lists"}