{"id":23325138,"url":"https://github.com/mratanusarkar/twitter-sentiment-analysis","last_synced_at":"2025-06-19T21:36:06.519Z","repository":{"id":128639609,"uuid":"597446214","full_name":"mratanusarkar/twitter-sentiment-analysis","owner":"mratanusarkar","description":"a demo poc for sentiment analysis of tweets","archived":false,"fork":false,"pushed_at":"2024-06-07T19:44:52.000Z","size":3609,"stargazers_count":0,"open_issues_count":8,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-07T06:24:37.707Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mratanusarkar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-04T15:25:09.000Z","updated_at":"2023-02-07T16:21:10.000Z","dependencies_parsed_at":"2024-12-20T18:32:17.679Z","dependency_job_id":"6067070d-6f5b-4673-8411-ba83f9084734","html_url":"https://github.com/mratanusarkar/twitter-sentiment-analysis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mratanusarkar/twitter-sentiment-analysis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mratanusarkar%2Ftwitter-sentiment-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mratanusarkar%2Ftwitter-sentiment-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mratanusarkar%2Ftwitter-sentiment-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mratanusarkar%2Ftwitter-sentiment-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mratanusarkar","download_url":"https://codeload.github.com/mratanusarkar/twitter-sentiment-analysis/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mratanusarkar%2Ftwitter-sentiment-analysis/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260836866,"owners_count":23070553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-20T18:29:40.043Z","updated_at":"2025-06-19T21:36:01.489Z","avatar_url":"https://github.com/mratanusarkar.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# twitter-sentiment-analysis\nThis is a demo poc for sentiment analysis of tweets. \nThe repo is divided into:\n- [Notebooks](https://github.com/mratanusarkar/twitter-sentiment-analysis/tree/main/Notebooks)\n- [Scripts](https://github.com/mratanusarkar/twitter-sentiment-analysis/tree/main/Runner)\n\nWhere you can find: \n- various experiments with Twitter API\n- ways to scrape and collect tweet data using various kinds of search parameters \n- perform analysis and visualizations using the collected data\n- Sentiment analysis and Insights using NLP\n\nIn the **Notebooks**, and a script/module format of the same in **Runner** folder for background running jobs.\n\n\n# Features\nThis repo is still a work-in-progress. \u003cbr\u003e\nSome of the features currently implemented are as follows:\n\n- Tweet Scraper\n- Twitter Word Cloud\n- (more features coming soon...)\n\n---\n\n# Tweet Scraper\nThis is a scrapper function used to gather and collect tweets, powered by [snscrape](https://github.com/JustAnotherArchivist/snscrape). \u003cbr\u003e\ncompared to tweet API v2, this enables us to get unlimited tweets without any restrictions and without the need to get API tokens and secrets.\n\n## usage:\nHere is a sample usage:\n```python\nfrom module.scraper import TweetScraper\n\n# create helper objects\ntweet_scraper = TweetScraper()\n\n# set parameters\nquery = '@isro'\nlimit = 1000\n\n# scrape tweets\nrawData = tweet_scraper.get_tweets(query, limit)\n```\nThis will return a pandas dataframe containing last 1000 tweets from @isro. \u003cbr\u003e\nsee the function signature below to get more details on function parameters.\n\n## parameters\n\n| Parameter | Data Type        | Description                                               | More Details                                                                                                                                                               |\n|-----------|------------------|-----------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| query     | string           | twitter search query as per https://twitter.com/search?q= | it can be a user mention like: `@user` or hashtag like `#tag` or a word like `text` or a complex query joined by  `AND`, `OR`, or statement enclosed in `()`. Explore twitter.com/search-advanced to know more. |\n| limit     | int              | number of tweets you want to scrape                       | depending on number of tweets, the script will take time to execute. example: 100 tweets will be collected in 1s, where as 10,000 might take 5min and 1,00,000 may take 1h. |\n| return    | pandas dataframe | a pandas dataframe with the tweets                        | as of now, the following data fields are collected: `id`, `date`, `username`, `content`, `view_count`, `like_count`, `reply_count`, `retweet_count`, `quote_Count`, `url`  |\n\n---\n\n# Twitter Word Cloud\nThis is a visualization tool powered by [word_cloud](https://github.com/amueller/word_cloud).\nCombined with the scraper function above, this tool gives you the capability to visualize what's going on in twitter at a glance!\nIn short, it uses all the tweets and counts the most occurring words in the tweets. It discards the common english words, and non-english characters, does pre-processing and data cleaning,\nand In the end, you get a word cloud that gives insight into your search query.\n\nFor example:\n- you can input @user and see what's going on with the user's timeline at one go!\n- you may input a trending #hashtag and take a look on what twitter has to say on the trend/issue/event at one go!\n- it's left to the end user on how they may use this tool and get powerful visualization. the possibilities are limitless!\n\nI am sharing a few use-cases below.\n\n## sample use case:\nHere is a sample word cloud generated using `limit: 10,000` and `query: ISRO (#SSLVD2 OR #ISRO)` at resolution: `width, height: 1080, 720` during the SSLV-D2 Launch on 10th Feb, 2023. You can clearly see how Twitter was looking that day during the Launch, in just one snapshot!\n\n![ISRO SLVD2 Launch](https://user-images.githubusercontent.com/34891206/219942847-9329f7b1-7913-4d23-9222-a0553f50d9ff.png)\n\n\n## usage:\nHere is a sample usage:\n```python\nfrom module.scraper import TweetScraper\nfrom module.generator import TwitterWordCloud\n\n# create helper objects\ntweet_scraper = TweetScraper()\ntweet_wc = TwitterWordCloud()\n\n# set parameters\ntopic_title = 'ISRO During SSLV-D2 Launch'\nquery = 'ISRO (#SSLVD2 OR #ISRO)'\nlimit = 1000\nexclude_words = ['amp', 'eval']\n\n# scrape tweets\nrawData = tweet_scraper.get_tweets(query, limit)\ntweet_wc.generate_word_cloud_v2(rawData, topic_title, exclude_words, 1080, 720)\n```\n\nThis will generate a wordcloud using last 1000 tweets made during the ISRO SSLV-D2 Launch.\nsee the function signature below to get more details on function parameters.\n\n## parameters\n\n### function: generate_word_cloud():\na simple generator function with with only one required parameter (the dataframe) for quick easy word cloud generation. \u003cbr\u003e\nThe output image is (1000px, 500px) in a (15, 8) inch canvas.\n\n| Parameter           | Data Type        | Description                                              | More Details                                                                                                            |\n|---------------------|------------------|----------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|\n| rawData             | pandas dataframe | pandas dataframe from scraper function                   |                                                                                                                         |\n| force_exclude_words | list of strings  | words you wish to exclude from word cloud                | after seeing an output, if you feel some words from the image that you wish to exclude, you can do so using this option |\n| return              | None             | it generates and display the wordcloud, and saves as png |                                                                                                                         |\n\n### generate_word_cloud_v2():\na move customizable and generic function with the following parameters\n\n| Parameter           | Required         | Data Type        | Description                                                    | More Details                                                                                                            |\n|---------------------|------------------|------------------|----------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|\n| rawData             | Yes              | pandas dataframe | pandas dataframe from scraper function                         |                                                                                                                         |\n| topic_title         | Yes              | string           | a short string describing the topic of tweets in the dataframe | the output files will have the same name as the topic                                                                   |\n| force_exclude_words | No, default []   | list of strings  | words you wish to exclude from word cloud                      | after seeing an output, if you feel some words from the image that you wish to exclude, you can do so using this option |\n| width               | No, default 1000 | int              | number of pixels wide of the output image                      |                                                                                                                         |\n| height              | No, default 500  | int              | number of pixels height of the output image                    |                                                                                                                         |\n| dpi                 | No, default 100  | int              | pixel density per inch                                         |                                                                                                                         |\n| return              | NA               | None             | it generates and display the wordcloud, and saves as png       |                                                                                                                         |\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmratanusarkar%2Ftwitter-sentiment-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmratanusarkar%2Ftwitter-sentiment-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmratanusarkar%2Ftwitter-sentiment-analysis/lists"}