{"id":42776543,"url":"https://github.com/xiffy/thcrrspndnt","last_synced_at":"2026-01-29T22:08:51.369Z","repository":{"id":48402921,"uuid":"245890862","full_name":"xiffy/thcrrspndnt","owner":"xiffy","description":":see_no_evil: :hear_no_evil: :speak_no_evil: But now for english","archived":false,"fork":false,"pushed_at":"2025-04-11T18:05:39.000Z","size":263,"stargazers_count":5,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-10-13T12:44:36.560Z","etag":null,"topics":["correspondent","flask","python","rss","twitter-bot","website"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xiffy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-08T21:29:15.000Z","updated_at":"2025-04-11T18:05:43.000Z","dependencies_parsed_at":"2024-03-29T12:28:51.245Z","dependency_job_id":"2d2fcff5-26e2-41f7-b5b8-aa47b01b75c2","html_url":"https://github.com/xiffy/thcrrspndnt","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/xiffy/thcrrspndnt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiffy%2Fthcrrspndnt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiffy%2Fthcrrspndnt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiffy%2Fthcrrspndnt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiffy%2Fthcrrspndnt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xiffy","download_url":"https://codeload.github.com/xiffy/thcrrspndnt/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiffy%2Fthcrrspndnt/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28886972,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-29T21:06:44.224Z","status":"ssl_error","status_checked_at":"2026-01-29T21:06:42.160Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["correspondent","flask","python","rss","twitter-bot","website"],"created_at":"2026-01-29T22:08:50.875Z","updated_at":"2026-01-29T22:08:51.353Z","avatar_url":"https://github.com/xiffy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# :see_no_evil: :hear_no_evil: :speak_no_evil: thcrrspndnt \nthcrrspndnt is first of all a twitter-bot. It searches for a configurable `query` the twitter-api on behalf of your twitter account. (Did I mention you must have a twitter account and register an app to get this baby qorking?). Each found tweet is then parsed for the presence of a share url and unshortens any in between url shortners. \nIf a valid URL is found in the tweet, the tweet get's stored. \nThe URL is fetched (once) and vital information is collected with Beautiful Soup and stored in the database. If this is the first time the URL is seen, a tweet is (optionally) sent. \n\nThis service is specifically designed for the Correspondent.nl/.com a Dutch / English site. \nYou can see it in action here: https://molecule.nl/dcrrspndnt (dutch) and https://molecule.nl/thcrrspndnt theyare accompanied by [@dcrrspndnt](https://twitter.com/dcrrspndnt) and [@thcrsspndnton](https://twitter.com/thcrsspndnt) twitter respectively. \n\n#### I wants it\n```git clone git@github.com:xiffy/thcrrspndnt.git\ncd thcrrspndnt\nvirtualenv -p python3 venv\n. venv/bin/activate\npip install -r requirements.txt\n```\n\nnow edit settings.sample.py and fill in your twitter-secrets and your desired searchstring and save as settings.py\n\nstarting the bot, i've got the next lines inside my crontab\n \n    */5 * * * *  cd /\u003chidden\u003eproject/thcrrspndnt/thcrrspndnt; /\u003chidden\u003e/project/thcrrspndnt/venv/bin/python searcher.py \u003e\u003e /var/log/thcrrspndnt.log 2\u003e\u00261\n     export CORRES_VERSION=NL; cd /\u003chidden\u003e/project/thcrrspndnt/thcrrspndnt; /\u003chidden\u003e/project/thcrrspndnt/venv/bin/python searcher.py \u003e\u003e /var/log/dcrrspndnt.log 2\u003e\u00261 \n\nand this creates some logging for each site and crawls the web for each site.\n\nWhen you invoke `start.sh` the english variant server will get started and will listen to port 5000,`startNL.sh` will start a server on port 5001. These are Flask servers and will offer a web interface to the cached articles. And you can access all articles for free :-)\n \n \n\n:todo: tell how to obtain app-aproval\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiffy%2Fthcrrspndnt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxiffy%2Fthcrrspndnt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiffy%2Fthcrrspndnt/lists"}