{"id":16724656,"url":"https://github.com/eswzy/telegram-news","last_synced_at":"2025-03-21T21:31:01.501Z","repository":{"id":55885284,"uuid":"250285276","full_name":"ESWZY/telegram-news","owner":"ESWZY","description":"Python package for automatically fetching and pushing news by Telegram.","archived":false,"fork":false,"pushed_at":"2023-10-18T16:01:35.000Z","size":2037,"stargazers_count":16,"open_issues_count":2,"forks_count":8,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-28T23:11:11.042Z","etag":null,"topics":["newsfeed","pypi","telegram","telegram-bot"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ESWZY.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-03-26T14:44:40.000Z","updated_at":"2025-01-22T21:45:00.000Z","dependencies_parsed_at":"2022-08-15T08:40:13.395Z","dependency_job_id":"25feb49d-59c7-49c4-bae3-e8819475be59","html_url":"https://github.com/ESWZY/telegram-news","commit_stats":{"total_commits":255,"total_committers":5,"mean_commits":51.0,"dds":0.09019607843137256,"last_synced_commit":"198c6e2167f19804d0db4937020b6a6f5b63ca19"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESWZY%2Ftelegram-news","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESWZY%2Ftelegram-news/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESWZY%2Ftelegram-news/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESWZY%2Ftelegram-news/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ESWZY","download_url":"https://codeload.github.com/ESWZY/telegram-news/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244160042,"owners_count":20408020,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["newsfeed","pypi","telegram","telegram-bot"],"created_at":"2024-10-12T22:46:10.787Z","updated_at":"2025-03-21T21:31:00.496Z","avatar_url":"https://github.com/ESWZY.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/ESWZY/telegram-news/master/docs/images/banner.png\" alt=\"Telegram-news\"\u003e\n  \u003cbr\u003eTelegram-news\u003cbr\u003e\n\u003c/h1\u003e\n\n\u003cdiv align=\"center\"\u003e\n\nPython package for automatically fetching and pushing news by Telegram.\n\n[![PyPI](https://img.shields.io/pypi/v/telegram-news)](https://pypi.org/project/telegram-news/)\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/telegram-news?logo=python)\n[![License](https://img.shields.io/github/license/ESWZY/telegram-news)](https://github.com/ESWZY/telegram-news/blob/master/LICENSE)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/telegram-news)\n\n[![Build Status](https://img.shields.io/travis/ESWZY/telegram-news/master?logo=travis)](https://travis-ci.org/ESWZY/telegram-news)\n[![Codacy Badge](https://api.codacy.com/project/badge/Grade/3c07fed525da42e89dd3d0376457b4d2)](https://app.codacy.com/manual/ESWZY/telegram-news?utm_source=github.com\u0026utm_medium=referral\u0026utm_content=ESWZY/telegram-news\u0026utm_campaign=Badge_Grade_Dashboard)\n![Last commit](https://img.shields.io/github/last-commit/ESWZY/telegram-news)\n[![https://t.me/eswzy](https://img.shields.io/badge/Telegram-ESWZY-blue.svg?logo=telegram)](https://t.me/eswzy)\n\n\u003c/div\u003e\n\n## Introduction\n\nThis is a easy-to-learn, flexible and standardized message fetching and pushing framework, especially for [Telegram](http://www.telegram.org) and [Telegram Bot](https://core.telegram.org/bots).\n\nThe target news source can be HTML page, JSON and XML. We also provide customized process for unknown data format.\n\nPush the latest news to your channel or group once it happens!\n\n## Install\n\n```shell\n$ pip install telegram-news\n```\n\nOr, you can install by cloning this repository:\n\n```shell\n$ git clone https://github.com/ESWZY/telegram-news.git\n$ cd telegram-news\n$ python setup.py install\n```\n\n## Prepare\n\nIt does not need much so that you can run your code anywhere.\n\nFirst, ask [@BotFather](https://t.me/botfather) for a bot and bot token. After that, create a public [channel](https://telegram.org/tour/channels) or [group](https://telegram.org/tour/groups), and remember chat id you just named. Do not forget to invite your bot into your channel or group and make it an admin.\n\nYou also need a SQL database. Any SQL database is OK. Especially, I recommend [PostgreSQL](https://www.postgresql.org/).\n\n## Quick deploy on Heroku\n\nClick 👇 button to deploy an example for free. Python environment and PostgreSQL database have been prepared.\n\n\u003ca href=\"https://heroku.com/deploy?template=https://github.com/ESWZY/telegram-news-getting-started\"\u003e\n  \u003cimg src=\"https://www.herokucdn.com/deploy/button.svg\" alt=\"Deploy\"\u003e\n\u003c/a\u003e\n\nAfter deployment, start the worker in \"Resources\" tab, and then you can see the effect in your channel/group, which contains both SCMP news and Wiki news at same time.\n\nAlso, you can have a look at the quick deployment source code of this project in [ESWZY/telegram-news-getting-started](https://github.com/ESWZY/telegram-news-getting-started).\n\n## Usage\n\nThose are 3 examples for you to understand how to use the framework.\n\n### Basic Example\n\n```python\nimport os\nfrom sqlalchemy import create_engine\nfrom sqlalchemy.orm import Session\n\nfrom telegram_news.template import InfoExtractor, NewsPostman\n\n# Three required fields:\n# Your bot token obtained from @BotFather\nbot_token = os.getenv(\"TOKEN\")\n# Add your bots into a channel as an administrator\nchannel = os.getenv(\"CHANNEL\")\n# Your database to store old messages.\nDATABASE_URL = os.getenv(\"DATABASE_URL\")\n\n# Create a database session\nengine = create_engine(DATABASE_URL)\ndb = Session(bind=engine.connect())\n\n# The news source\nurl = \"https://en.wikinews.org/wiki/Main_Page\"\ntag = \"Wiki News\"\ntable_name = \"wikinews\"\n\n# Info extractor to process data format\nie = InfoExtractor()\n\n# Select select element by CSS-based selector\nie.set_list_selector('#MainPage_latest_news_text \u003e ul \u003e li')\nie.set_title_selector('#firstHeading')\nie.set_paragraph_selector('#mw-content-text \u003e div \u003e p:not(p:nth-child(1))')\nie.set_time_selector('#mw-content-text \u003e div \u003e p:nth-child(1) \u003e strong')\nie.set_source_selector('span.sourceTemplate')\n\n# Set a max length for post, Max is 4096\nie.max_post_length = 2000\n\n# News postman to manage sending affair\nnp = NewsPostman(listURLs=[url, ], sendList=[channel, ], db=db, tag=tag)\nnp.set_bot_token(bot_token)\nnp.set_extractor(ie)\nnp.set_table_name(table_name)\n\n# Start to work!\nnp.poll()\n```\n\nTypical results:\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/ESWZY/telegram-news/master/docs/images/demo1.png\" alt=\"Demo 1\" width=\"40%\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/ESWZY/telegram-news/master/docs/images/demo2.png\" alt=\"Demo 2\" width=\"40%\"\u003e\n\u003c/div\u003e\n\n### Advanced Example\n\n```python\nimport os\nfrom sqlalchemy import create_engine\nfrom sqlalchemy.orm import Session\nfrom telegram_news.template import InfoExtractor, NewsPostman\nbot_token = os.getenv(\"TOKEN\")\nchannel = os.getenv(\"CHANNEL\")\nDATABASE_URL = os.getenv(\"DATABASE_URL\")\nengine = create_engine(DATABASE_URL)\ndb = Session(bind=engine.connect())\n\n# Above code is as same as the basic example, you can reuse those code directly\n\nurl_2 = \"https://www.cnbeta.com/\"\ntag_2 = \"cnBeta\"\ntable_name_2 = \"cnbetanews\"\n\nie_2 = InfoExtractor()\nie_2.set_list_selector('.items-area \u003e div \u003e dl \u003e dt \u003e a')\nie_2.set_title_selector('header \u003e h1')\n\n# Select many target at same time    \nie_2.set_paragraph_selector('div.cnbeta-article-body \u003e div.article-summary \u003e p, '  # Summary only\n                            'div.cnbeta-article-body \u003e div.article-content \u003e p')   # Content only\nie_2.set_time_selector('header \u003e div \u003e span:nth-child(1)')\nie_2.set_source_selector('header \u003e div \u003e span.source')\n\n# Select image to display, then the max length is down to 1024\nie_2.set_image_selector('div.cnbeta-article-body \u003e div.article-summary \u003e p img, '  # From summary only\n                        'div.cnbeta-article-body \u003e div.article-content \u003e p img')   # From content only\nie_2.max_post_length = 1000\n\nnp_2 = NewsPostman(listURLs=[url_2, ], sendList=[channel], tag=tag_2, db=db)\nnp_2.set_extractor(ie_2)\nnp_2.set_table_name(table_name_2)\nnp_2.poll()\n```\n\nTypical results:\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/ESWZY/telegram-news/master/docs/images/demo3.png\" alt=\"Demo 3\" width=\"40%\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/ESWZY/telegram-news/master/docs/images/demo4.png\" alt=\"Demo 4\" width=\"40%\"\u003e\n\u003c/div\u003e\n\n### Advanced Example for JSON and XML\n\nThe handle for JSON and XML are quite similar. You can convert XML to JSON by function `telegram_news.utils.xml_to_json`, and use `NewsPostmanJSON` and `InfoExtractorJSON`. Or, you can use `NewsPostmanXML` and `InfoExtractorXML` directly.\n\nYou should use key list to recursively route to the information you want.\n\n```python\nimport hashlib\nimport json\nimport os\nfrom sqlalchemy import create_engine\nfrom sqlalchemy.orm import Session\nfrom telegram_news.template import InfoExtractorJSON, NewsPostmanJSON\nfrom telegram_news.utils import xml_to_json\nbot_token = os.getenv(\"TOKEN\")\nchannel = os.getenv(\"CHANNEL\")\nDATABASE_URL = os.getenv(\"DATABASE_URL\")\nengine = create_engine(DATABASE_URL)\ndb = Session(bind=engine.connect())\n\nurl_3 = \"https://www.scmp.com/rss/91/feed\"\ntag_3 = \"SCMP\"\ntable_name_3 = \"scmpnews\"\n\nie_3 = InfoExtractorJSON()\n\n# Pre-process the XML string, convert to JSON string\ndef list_pre_process(text):\n    text = json.loads(xml_to_json(text))\n    return json.dumps(text)\nie_3.set_list_pre_process_policy(list_pre_process)\n\n# Route by key list\nie_3.set_list_router(['rss', 'channel', 'item'])\nie_3.set_link_router(['link'])\nie_3.set_title_router(['title'])\nie_3.set_paragraphs_router(['description'])\nie_3.set_time_router(['pubDate'])\nie_3.set_source_router(['author'])\nie_3.set_image_router(['media:thumbnail', '@url'])\n\n# Customize ID for news item\ndef id_policy(link):\n    return hashlib.md5(link.encode(\"utf-8\")).hexdigest()\nie_3.set_id_policy(id_policy)\n\nnp_3 = NewsPostmanJSON(listURLs=[url_3], sendList=[channel], db=db, tag=tag_3)\nnp_3.set_extractor(ie_3)\nnp_3.set_table_name(table_name_3)\nnp_3.poll()\n```\n\nTypical results:\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/ESWZY/telegram-news/master/docs/images/demo5.png\" alt=\"Demo 5\" width=\"40%\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/ESWZY/telegram-news/master/docs/images/demo6.png\" alt=\"Demo 6\" width=\"40%\"\u003e\n\u003c/div\u003e\n\n### Parallel Program\n\nIf you use the same database and send to the same channel, you can simply joint each part of code block, and call `poll()` function simultaneously.\n\nAn example you can find in our Heroku deploy template repo: \n\nhttps://github.com/ESWZY/telegram-news-getting-started/blob/master/main.py\n\n## Example Channel\n\nA Telegram channel of [basic example](https://github.com/ESWZY/telegram-news#basic-example) for English Wikinews: [~~@wikinews_en~~](https://t.me/joinchat/T7TbJUWpgUpGmarY) (in English)\n\nA Telegram channel for realtime earthquake warning powered by Telegram-news: [@earthquake_alert](https://t.me/s/earthquake_alert) (in Chinese)\n\n## TODO\n\n- [x] HTML item list\n- [x] JSON item list\n- [x] XML item list\n- [x] Send Image\n- [x] Send Video\n- [x] Send media group\n- [ ] Send file\n- [ ] Send audio\n- [x] File sending retry\n- [ ] CC as e-mail\n- [ ] Webhook\n- [ ] Update message by message ID\n- [ ] Document\n- [ ] GUI\n\n## Feedback\n\nFeel free to contact with me if you have any question. Also welcome any contribute.\n\nIf you build a channel by this, don't forget to share that good news with us!\n\n## License\n\nLicensed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feswzy%2Ftelegram-news","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feswzy%2Ftelegram-news","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feswzy%2Ftelegram-news/lists"}