{"id":13744274,"url":"https://github.com/dc-aichara/DS-ML-Public","last_synced_at":"2025-05-09T02:33:15.314Z","repository":{"id":34887350,"uuid":"173027799","full_name":"dc-aichara/DS-ML-Public","owner":"dc-aichara","description":"Python Scripts and Jupyter Notebooks ","archived":false,"fork":false,"pushed_at":"2024-04-17T00:01:26.000Z","size":3307,"stargazers_count":51,"open_issues_count":5,"forks_count":38,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-11-15T15:42:34.256Z","etag":null,"topics":["bayesian-optimization","beautifulsoup","bitcoin","catboost","dash","dashboard","data-analysis","data-mining","data-science","data-visualisation","hyperparameter-tuning","hyperparameters-optimization","lightgbm","machine-learning","news","plotly","python","telegram","web-scraping","xgboost"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dc-aichara.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-28T02:40:07.000Z","updated_at":"2024-08-12T19:46:12.000Z","dependencies_parsed_at":"2024-01-15T00:17:06.556Z","dependency_job_id":"ec5de77b-9193-4856-91b9-227f5d909e10","html_url":"https://github.com/dc-aichara/DS-ML-Public","commit_stats":null,"previous_names":[],"tags_count":0,"template":true,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dc-aichara%2FDS-ML-Public","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dc-aichara%2FDS-ML-Public/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dc-aichara%2FDS-ML-Public/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dc-aichara%2FDS-ML-Public/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dc-aichara","download_url":"https://codeload.github.com/dc-aichara/DS-ML-Public/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253177897,"owners_count":21866417,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-optimization","beautifulsoup","bitcoin","catboost","dash","dashboard","data-analysis","data-mining","data-science","data-visualisation","hyperparameter-tuning","hyperparameters-optimization","lightgbm","machine-learning","news","plotly","python","telegram","web-scraping","xgboost"],"created_at":"2024-08-03T05:01:06.607Z","updated_at":"2025-05-09T02:33:12.439Z","avatar_url":"https://github.com/dc-aichara.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"\n\u003cimg src=\"images/wordcloud.png\"\u003e\n\n---\n\n# [Medium Files](https://github.com/dc-aichara/DS-ML-Public/tree/master/Medium_Files)\n\n## 1. [hyp_tune.ipynb](https://github.com/dc-aichara/DS-ML-Public/blob/master/Medium_Files/hyp_tune.ipynb)\n\n- A notebook to guide hyperparameters optimization using Bayesian model based optimization. \n\nExample hyperparameters optimization results table for LightGBM Regressor on Boston Housing data. \n\n\u003cimg src=\"images/hyp_lgbm.png\"\u003e\n\nRead complete article on [Medium](https://medium.com/analytics-vidhya/hyperparameters-optimization-for-lightgbm-catboost-and-xgboost-regressors-using-bayesian-6e7c495947a9). \n\n## 2. [Bar_Plot_Animation.py ](https://github.com/dc-aichara/DS-ML-Public/blob/master/Medium_Files/Bar_Plot_Animation.py)\n- Creates animation of bar plots using matplotlib python package \u003cbr\u003e\n[Read complete tutorial on Medium](https://medium.com/@dc.aichara/making-animated-bar-plots-of-top-16-cryptocurrencies-market-capitalization-price-and-volume-7889788af264)\u003cbr\u003e\n[Watch animation on YouTube](https://www.youtube.com/watch?v=jqSuaRpCnro) \u003cbr\u003e\n[![YouTube Video](http://img.youtube.com/vi/jqSuaRpCnro/0.jpg)](http://www.youtube.com/watch?v=jqSuaRpCnro)\n\n## 3. [telegram.py](https://github.com/dc-aichara/DS-ML-Public/blob/master/Medium_Files/telegram.py)\n- Telegram chat data extraction and data processing \u003cbr\u003e\n[Medium Tutorial](https://medium.com/@dc.aichara/telegram-channel-data-extraction-users-information-chats-and-specific-messages-and-data-21bb54710fd3)  \u003cbr\u003e\n\n\u003e An example of telegram chats which contain keyword 'bitcoin' or 'btc'\n\n\u003cimg src='images/Bitcoin_text_ts.png'\u003e\n\n\n## 4. [Bitcoin_messages_telegram.py](https://github.com/dc-aichara/DS-ML-Public/blob/master/Medium_Files/Bitcoin_messages_telegram.py)\n- Get text messages from telegram groups and channels which contain word 'bitcoin' or 'btc'. \u003cbr\u003e\n\nExample: \n\n```python\n\u003e\u003e\u003e tele_btc_messages.head()\n\n```\n\u003cimg src='images/tele_text.png'\u003e\n\n## 5. [telegram_user_status.py](https://github.com/dc-aichara/DS-ML-Public/blob/master/Medium_Files/telegram_user_status.py)\n- Use to get users who were online in last 24 hours.\n\nExample: \n```\n$ cd DS-ML-Public\n$ python telegram_user_status.py 12345 fe3922d77g6wgwgwyu35g46c9 bitgrit\nNumber of active users in last 24 hours is 1530.\n          User               status\n0  Dayal Chand               online\n1       Sameer             recently\n2  Dikesh Shah  2019-07-02 01:13:19\n3       Crypto  2019-07-02 00:47:50\n4        Billy  2019-07-02 01:32:49\n\n```\n## 6. [GA.ipynb](https://github.com/dc-aichara/DS-ML-Public/blob/master/Medium_Files/GA.ipynb)\n\n- A  Jupyter Notebook for Google Analytics Reporting API tutorial. \n\n## 7. [dashboard_demo](https://github.com/dc-aichara/DS-ML-Public/blob/master/Medium_Files/dashboard_demo/)\n\n- A dashboard demo app  \n\n*** \n\n## [WebScrapers](https://github.com/dc-aichara/DS-ML-Public/tree/master/WebScrapers) \n\n## 1. [lok_sabha_results.py](https://github.com/dc-aichara/DS-ML-Public/blob/master/WebScrapers/lok_sabha_results.py)\n- Get [Lok Sabha elections 2019 results](https://results.eci.gov.in/pc/en/partywise/index.htm) using web scraping technique with requests and BeautifulSoup python packages. \n\n## 2. [crypto_news_scraper.py](https://github.com/dc-aichara/DS-ML-Public/blob/master/WebScrapers/crypto_news_scraper.py)\n- Python script to extract cryptocurries related news from [CoinDesk](https://www.coindesk.com), [Cointelegraph](https://cointelegraph.com) and [cryptonewsz](https://www.cryptonewsz.com). \n\n### Usages\n$ git clone https://github.com/dc-aichara/DS-ML-Public.git \u003cbr\u003e\n$ cd DS-ML-Public/WebScrapers \u003cbr\u003e\n$ python3 \n\n```python\n\u003e\u003e\u003e from crypto_news_scraper import NewsScrap\n\u003e\u003e\u003e news = NewsScrap()\n\u003e\u003e\u003e df_coindesk = news.coin_desk_news()\n\u003e\u003e\u003e df_coindesk.head()\n  category                                            heading  ...                time    source\n0     news  Dapp.com Closes $1 Million Investment Round Le...  ... 2019-09-06 22:00:00  CoinDesk\n1     news  Telegram Finally Releases Code for Its $1.7 Bi...  ... 2019-09-06 21:46:00  CoinDesk\n2     news  Massive $1 Billion Bitcoin Whale Transaction M...  ... 2019-09-06 19:00:00  CoinDesk\n3     news  Ethereum Picks Early October for Testnet Activ...  ... 2019-09-06 18:00:00  CoinDesk\n4     news  Dapp Data Site DappRadar Raises $2.33 Million ...  ... 2019-09-06 17:00:00  CoinDesk\n\n[5 rows x 6 columns]\n\u003e\u003e\u003e df_cointelegraph = news.cointelegraph_news()\n\u003e\u003e\u003e df_cointelegraph.head()\n  category                                            heading  ...                 time         source\n0     News  Crypto and Blockchain Adoption Grows: 5 Import...  ...  2019-09-09 11:15:03  CoinTelegraph\n1     News  World’s ‘First’ Blockchain Smartphone to Becom...  ...  2019-09-09 08:15:03  CoinTelegraph\n2     News  Ethereum's Istanbul Hard Fork Implementation D...  ...  2019-09-09 08:15:03  CoinTelegraph\n3     News  Blockchain Startup DappRadar Raises $2.33M Fro...  ...  2019-09-09 08:15:03  CoinTelegraph\n4     News  Huobi’s Research Arm to Partner with the Unive...  ...  2019-09-09 07:15:03  CoinTelegraph\n\n[5 rows x 6 columns]\n\u003e\u003e\u003e df_all = news.get_all_news()\nGetting news from CoinDesk!!\nGetting news from Cointelegraph!!\nGetting news from cryptonewsz!! This will take 1-2 mintues. 😉\n\u003e\u003e\u003e df_all.head()\n  category                                            heading  ...                 time    source\n0     news  Dapp.com Closes $1 Million Investment Round Le...  ...  2019-09-06 22:00:00  CoinDesk\n1     news  Telegram Finally Releases Code for Its $1.7 Bi...  ...  2019-09-06 21:46:00  CoinDesk\n2     news  Massive $1 Billion Bitcoin Whale Transaction M...  ...  2019-09-06 19:00:00  CoinDesk\n3     news  Ethereum Picks Early October for Testnet Activ...  ...  2019-09-06 18:00:00  CoinDesk\n4     news  Dapp Data Site DappRadar Raises $2.33 Million ...  ...  2019-09-06 17:00:00  CoinDesk\n\n[5 rows x 6 columns]\n\n```\n## 3. [inshorts_news_scraper.py](https://github.com/dc-aichara/DS-ML-Public/blob/master/WebScrapers/inshorts_news_scraper.py)\n\n### Usages\n$ git clone https://github.com/dc-aichara/DS-ML-Public.git \u003cbr\u003e\n$ cd DS-ML-Public/WebScrapers \u003cbr\u003e\n$ python3 \n\n```python\n\u003e\u003e\u003e from inshorts_news_scraper import InshortsNews\n\u003e\u003e\u003e news = InshortsNews('business')\n\u003e\u003e\u003e df_b = news.get_news()\n\u003e\u003e\u003e df_b.head()\n                                            headings                                               news       short_by                time  category\n0  BSNL plans to fire 30% contract staff unpaid s...  BSNL is reportedly planning to lay off about 3...  Anushka Dixit 2019-09-09 23:35:00  business\n1  SAT overturns SEBI's 2 year-ban on PwC in ₹7,8...  The Securities Appellate Tribunal (SAT) on Mon...  Anushka Dixit 2019-09-09 21:29:00  business\n2  Nissan CEO Hiroto Saikawa to step down on Sept...  Nissan CEO Hiroto Saikawa will step down on Se...         Dharna 2019-09-09 21:08:00  business\n3  British Airways pilots begin 2-day strike over...  British Airways pilots began a two-day strike ...  Anushka Dixit 2019-09-09 20:18:00  business\n4  SEBI making e-voting app for retail investors ...  Markets regulator SEBI is working on an e-voti...         Dharna 2019-09-09 18:04:00  business\n\u003e\u003e\u003e df_all = news.get_all_news()\n\u003e\u003e\u003e df_all.head()\n                                            headings                                               news        short_by                time  category\n0  Conflict between India, Pak less heated now th...  Speaking about tensions between India and Paki...  Arshiya Chopra 2019-09-10 08:50:00  national\n1  Bengaluru woman loses ₹95,000 after calling fa...  A Bengaluru woman lost ₹95,000 after calling a...  Pragya Swastik 2019-09-10 08:25:00  national\n2  IAS officer who resigned is traitor, should go...  BJP MP Anantkumar Hegde has called IAS officer...    Apaar Sharma 2019-09-09 23:28:00  national\n3  Stop drama, stand up, CISF allegedly tells wom...  Virali Modi, a disability rights activist, has...    Anmol Sharma 2019-09-09 23:10:00  national\n4  Tech firms may be allowed to sell users' publi...  India is reportedly mulling guidelines which w...          Dharna 2019-09-09 23:00:00  national\n\n```\n## 4. [Japanese News Scraper](https://github.com/dc-aichara/DS-ML-Public/blob/master/WebScrapers/japanese_news_scraper.py)\n\n### Usage:\n```python\n\u003e\u003e\u003e from japanese_news_scraper import JapaneseNewsScrap\n\u003e\u003e\u003e jp_news = JapaneseNewsScrap(24*60*60)\n\u003e\u003e\u003e df_coinpost = jp_news.get_coin_post_news()\n\u003e\u003e\u003e df_coinpost.head()\n                  time                                         heading  ...                           link    source\n0  2019-10-08 15:30:48              米リップル社、大学ブロックチェーン研究イニシアチブで年次大会を初開催  ...  https://coinpost.jp/?p=111090  CoinPost\n1  2019-10-08 15:29:29                     米NBAのキングス、ファン向けの独自仮想通貨発行を発表  ...  https://coinpost.jp/?p=111088  CoinPost\n2  2019-10-08 14:59:53             金融庁がブロックチェーン実験結果を公表、金融機関の顧客KYC情報を共有  ...  https://coinpost.jp/?p=111189  CoinPost\n3  2019-10-08 14:26:52    Chainlinkの新フレームワーク発表で、仮想通貨LINKが高騰 協賛にIntelなど  ...  https://coinpost.jp/?p=111080  CoinPost\n4  2019-10-08 14:04:36  イーサリアム企業連合、ブロックチェーン仕様の新バージョン発表 「Devcon 5」で検証実施  ...  https://coinpost.jp/?p=111170  CoinPost\n\n[5 rows x 5 columns]\n\n```\n## 5. [Website Pages Scraper](https://github.com/dc-aichara/DS-ML-Public/blob/master/WebScrapers/website_pages_scraper.py)\n\n- A python script to extract text from pages of given website. \n\n### Usage:\n\n```python\n\u003e\u003e\u003e from website_pages_scraper import WebScraper\n\u003e\u003e\u003e webscraper = WebScraper(main_page_url='https://www.example.com/')\n\u003e\u003e\u003e df = webscraper.scrap_website(depth=2)\n\n```\n\n## [My Medium Articles](https://medium.com/@dcaichara)\n-  [Hyperparameters Optimization for LightGBM, CatBoost and XGBoost Regressors using Bayesian Optimization.](https://medium.com/game-of-data/hyperparameters-optimization-for-lightgbm-catboost-and-xgboost-regressors-using-bayesian-6e7c495947a9)\n-  [Google Spreadsheets Automation with Python](https://medium.com/@dcaichara/play-with-google-spreadsheets-with-python-301dd4ee36eb)\n-  [Telegram Group/Channel Data Extraction (User’s information, chats, and specific messages), and Data Processing](https://medium.com/@dcaichara/telegram-channel-data-extraction-users-information-chats-and-specific-messages-and-data-21bb54710fd3)\n-  [12 Things to know about Jupyter Notebook Markdown](https://medium.com/@dcaichara/12-things-to-know-about-jupyter-notebook-markdown-3f6cef811707)\n-  [Making Animated Bar Plots of Top 16 CryptoCurrencies (Market Capitalization, Price and Volume)](https://medium.com/@dcaichara/telegram-channel-data-extraction-users-information-chats-and-specific-messages-and-data-21bb54710fd3)\n-  [A Complete Date-Time Guide for Data Scientist in Python](https://medium.com/analytics-vidhya/a-complete-date-time-guide-for-data-scientist-in-python-cd0b1cfb5e74) \n-  [How to Get Google Analytics Report Data as Pandas DataFrame?](https://medium.com/analytics-vidhya/how-to-get-google-analytics-report-data-as-pandas-dataframe-e3930831fba)\n-  [Coin Hopping Attack : How did c0ban (First ICO from Japan) deal with it using LWMA ?](https://medium.com/@dcaichara/coin-hopping-attack-how-did-c0ban-first-ico-from-japan-deal-with-it-using-lwma-9facda2f02b6)\n-  [Building a Dashboard App using Plotly’s Dash: A Complete Guide from Beginner to Pro-1](https://medium.com/analytics-vidhya/building-a-dashboard-app-using-plotlys-dash-a-complete-guide-from-beginner-to-pro-61e890bdc423)\n-  [Building a Dashboard App using Plotly’s Dash: A Complete Guide from Beginner to Pro-2](https://medium.com/analytics-vidhya/building-a-dashboard-app-using-plotlys-dash-a-complete-guide-from-beginner-to-pro-e7657a4eb707)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdc-aichara%2FDS-ML-Public","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdc-aichara%2FDS-ML-Public","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdc-aichara%2FDS-ML-Public/lists"}