{"id":13478775,"url":"https://github.com/sanghviharshit/pocket-tagger","last_synced_at":"2025-07-31T22:40:03.046Z","repository":{"id":50393254,"uuid":"194172654","full_name":"sanghviharshit/pocket-tagger","owner":"sanghviharshit","description":"📖👓🏷Tag your getpocket.com articles automatically using natural language processing","archived":false,"fork":false,"pushed_at":"2019-07-02T01:51:54.000Z","size":13,"stargazers_count":44,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-19T20:46:40.399Z","etag":null,"topics":["articles","getpocket","google-cloud","natural-language-processing","nlp","pocket","scraper","tag"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/pocket-tagger/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sanghviharshit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-27T22:50:31.000Z","updated_at":"2025-01-25T00:59:58.000Z","dependencies_parsed_at":"2022-08-29T06:51:41.968Z","dependency_job_id":null,"html_url":"https://github.com/sanghviharshit/pocket-tagger","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanghviharshit%2Fpocket-tagger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanghviharshit%2Fpocket-tagger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanghviharshit%2Fpocket-tagger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanghviharshit%2Fpocket-tagger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sanghviharshit","download_url":"https://codeload.github.com/sanghviharshit/pocket-tagger/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245809709,"owners_count":20676038,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["articles","getpocket","google-cloud","natural-language-processing","nlp","pocket","scraper","tag"],"created_at":"2024-07-31T16:02:02.688Z","updated_at":"2025-03-27T08:30:59.514Z","avatar_url":"https://github.com/sanghviharshit.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Auto Pocket Tagger\n\nUse Google cloud's Natural Language Processing API to automatically analyze the webpage from articles saved in your Pocket list, derive tags/keywords based on the content of the page, and add tags to the articles in Pocket list for free.\n\n\u003e Pocket has suggested tags service for their paid premium plans. You can find more about it [here](https://help.getpocket.com/article/906-pocket-premium-suggested-tags). This still requires manual work of adding the tags to each article one-by-one. This package automates all of it for free.\n\n## Features\n- Uses [Python wrapper](https://github.com/tapanpandita/pocket) for [Pocket API](http://getpocket.com/api/docs) to retrieve articles in the `My List`\n- Uses [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/) to scrape webpages\n- Uses Google Cloud's [Natural Language Processing API](https://cloud.google.com/natural-language/) to generate list of categories and entities from the content of the webpage\n- Uses Pocket API to add tags to articles in your `My List`\n\n\n## Usage\n\n### Installation\n\n#### Install published version from pypi\n```shell\n$ pip install pocket-tagger\n```\n\n#### Install latest version from git\n```shell\n$ pip install git+https://github.com/sanghviharshit/pocket-tagger\n```\n\n\n### Prerequisites\n#### [Google Cloud](https://cloud.google.com/natural-language/docs/quickstart)\n\nThis package relies on Google cloud natural language processing API, which requires billing enabled on your project.\nYou can find the quickstart instructions [here](https://cloud.google.com/natural-language/docs/quickstart)\n**Options:**\n1. Create a service account and download the credentials file - https://cloud.google.com/video-intelligence/docs/common/auth\n```python\ntagger = PocketTagger(gcloud_credentials_file=\"gcloud_credentials_file.json\")\n```\n2. or Configure gloud locally - https://cloud.google.com/sdk/gcloud/reference/init\n```python\ntagger = PocketTagger()\n```\n\n#### [Pocket API](https://getpocket.com/developer/)\n\nTo fetch the articles list and add tags, you need a developer key from [here](https://getpocket.com/developer/)\nCreate a new Application with `modify` and `retrieve` permissions. Save the Consumer Key and Access Token.\n```python\ntagger = PocketTagger(consumer_key='your-consumer-key',\n                access_token='your-access-token')\n```\n\n### [Examples](./examples)\n\n```python\n# Initialize PocketTagger with GCloud and Pocket API Credentials\ntagger = PocketTagger(gcloud_credentials_file=\"gcloud_credentials_file.json\",\n                consumer_key='pocket-consumer-key',\n                access_token='pocket-access-token')\n\n# Check https://getpocket.com/developer/docs/v3/retrieve for additional list of options you can pass for retrieving pocket list\narticles = tagger.get_articles_from_api(count=10, offset=10, detailType='complete')\n\n# Alternatively you can load the articles from file if you saved them previously using save_articles_to_file\n# articles = tagger.get_articles_from_file(\"20190621.json\")\n# Generate tags for each article\narticles_with_tags = tagger.get_tags_for_articles(articles)\n\n# Save the articles with tags to file. You can use this file to verify it looks good before running the final step to tag the articles.\ntagger.save_articles_to_file(today.strftime('%Y%m%d-with-tags.json'), articles_with_tags)\n\n# You can skip this step if you want to do a dry run. Verify the tags in the file we generated in the previous step.\ntagger.add_tags_to_articles(articles_with_tags)\n```\n\n### Optional overrides\nYou can override the default thresholds for [entity](https://cloud.google.com/natural-language/docs/reference/rest/v1/Entity\n) salience and [category](https://cloud.google.com/natural-language/docs/reference/rest/v1/ClassificationCategory) confidence\n\n```python\nthresholds = {\n  'entity_salience_threshold': 0.7\n  'category_confidence_threshold': 0.3\n}\narticles_with_tags = tagger.get_tags_for_articles(articles, thresholds)\n```\n\n## Sample\n\nSample output from running it for my 490 items long Pocket list\n\u003e `X` under Entities or Categories denotes the NLP client returned those as potential candidates, but we skipped them because it didn't meet the threshold. You can see the last line `Tags: abc, xyz` for list of tags pocket-tagger added for each URL.\n\n```\n(1/490) https://www.reddit.com/r/explainlikeimfive/comments/bvweym/eli5_why_do_coffee_drinkers_feel_more_clear/?utm_source=share\u0026utm_medium=ios_app\n         Title: ELI5: Why do coffee drinkers feel more clear headed after consuming caffeine? Why do some get a headache without it? Does caffeine cause any permanent brain changes and can the brain go back to 'normal' after years of caffeine use? : explainlikeimfive\n         Description: r/explainlikeimfive: **Explain Like I'm Five is the best forum and archive on the internet for layperson-friendly explanations.** \u0026nbsp; Don't Panic!\n         Entities:\n            X Coffee Drinkers: 0.2438652664422989\n            X Eli5: 0.14941969513893127\n            X Caffeine: 0.12065556645393372\n            X Caffeine: 0.0874909833073616\n            X Some: 0.06917785853147507\n            X Headache: 0.0606028214097023\n            X Brain: 0.03606536239385605\n            X Explainlikeimfive: 0.033727116882801056\n            X Brain Changes: 0.03211209550499916\n            X Caffeine Use: 0.029848895967006683\n            X R: 0.02966366335749626\n            X Forum: 0.028598546981811523\n            X Internet: 0.022404097020626068\n            X Archive: 0.022404097020626068\n            X Explainlikeimfive: 0.017647551372647285\n            X Don'T Panic: 0.009302889928221703\n            X Five: 0.007013489492237568\n            X Five: 0.0\n         Categories:\n              /Food \u0026 Drink/Beverages/Coffee \u0026 Tea: 0.6700000166893005\n         Tags: Food \u0026 Drink, Beverages, Coffee \u0026 Tea\n(2/490) https://www.reddit.com/r/television/comments/bnpwe3/enjoy_three_full_minutes_of_the_cast_of_game_of/?utm_source=share\u0026utm_medium=ios_app\n         Title: Enjoy three full minutes of the cast of 'Game of Thrones' expressing disappointment in Season 8. : television\n         Description: r/television:\n         Entities:\n            X Cast: 0.31218624114990234\n            X Disappointment: 0.20341947674751282\n            X Season: 0.20341947674751282\n            X Game Of Thrones: 0.13265934586524963\n            X Television: 0.08712445199489594\n            X Television: 0.06119102984666824\n            X 8: 0.0\n            X Three: 0.0\n         Categories:\n              /Arts \u0026 Entertainment/TV \u0026 Video/TV Shows \u0026 Programs: 0.75\n         Tags: Arts \u0026 Entertainment, TV \u0026 Video, TV Shows \u0026 Programs\n(3/490) https://www.reddit.com/r/homeautomation/comments/awvf5r/local_realtime_person_detection_for_rtsp_cameras/\n         Title: Local realtime person detection for RTSP cameras : homeautomation\n         Description: r/homeautomation: A subreddit focused on automating your home, housework or household activity. Sensors, switches, cameras, locks, etc. Any …\n         Entities:\n            X Realtime Person Detection: 0.3057926297187805\n            X Homeautomation: 0.15315502882003784\n            X Cameras: 0.14035314321517944\n            X Rtsp: 0.07461880147457123\n            X Homeautomation: 0.051411159336566925\n            X Home: 0.047811269760131836\n            X Housework: 0.04366889223456383\n            X Subreddit: 0.04183248057961464\n            X R: 0.04132793843746185\n            X Cameras: 0.032860007137060165\n            X Locks: 0.028899790719151497\n            X Household Activity: 0.012798599898815155\n            X Switches: 0.012735127471387386\n            X Sensors: 0.012735127471387386\n         Categories:\n              /Computers \u0026 Electronics: 0.7900000214576721\n         Tags: Computers \u0026 Electronics\n```\n\n## References\n- [Pocket API Wrapper for Python](https://github.com/tapanpandita/pocket)\n- [Pocket API Docs](http://getpocket.com/api/docs)\n- [Google Cloud Natural Language Processing](https://cloud.google.com/natural-language/)\n- [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/)\n- [Complete list of content categories from Google Natural Language API](https://cloud.google.com/natural-language/docs/categories)\n\n## Analytics\n[![Analytics](https://ga-beacon.appspot.com/UA-59542024-4/pocket-tagger/)](https://github.com/igrigorik/ga-beacon)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsanghviharshit%2Fpocket-tagger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsanghviharshit%2Fpocket-tagger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsanghviharshit%2Fpocket-tagger/lists"}