{"id":18860692,"url":"https://github.com/paulseperformance/cryptopanic_scraper","last_synced_at":"2025-04-14T12:30:51.058Z","repository":{"id":50184858,"uuid":"169201491","full_name":"pAulseperformance/cryptopanic_scraper","owner":"pAulseperformance","description":"cryptopanic news feed scraper","archived":false,"fork":false,"pushed_at":"2022-12-08T06:39:58.000Z","size":7844,"stargazers_count":23,"open_issues_count":4,"forks_count":3,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-03-28T01:50:23.104Z","etag":null,"topics":["chromedriver","crypto","jupyter","notebook","python","python3","selenium","selenium-python","selenium-webdriver","webscraper","webscraping"],"latest_commit_sha":null,"homepage":"https://www.cryptopanic.com","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pAulseperformance.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-05T06:52:12.000Z","updated_at":"2025-02-01T01:50:02.000Z","dependencies_parsed_at":"2023-01-24T16:45:17.652Z","dependency_job_id":null,"html_url":"https://github.com/pAulseperformance/cryptopanic_scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pAulseperformance%2Fcryptopanic_scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pAulseperformance%2Fcryptopanic_scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pAulseperformance%2Fcryptopanic_scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pAulseperformance%2Fcryptopanic_scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pAulseperformance","download_url":"https://codeload.github.com/pAulseperformance/cryptopanic_scraper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248881344,"owners_count":21176833,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chromedriver","crypto","jupyter","notebook","python","python3","selenium","selenium-python","selenium-webdriver","webscraper","webscraping"],"created_at":"2024-11-08T04:26:24.441Z","updated_at":"2025-04-14T12:30:50.556Z","avatar_url":"https://github.com/pAulseperformance.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n\n\u003c!-- PROJECT SHIELDS --\u003e\n\n[![Contributors][contributors-shield]][contributors-url]\n[![Forks][forks-shield]][forks-url]\n[![Stargazers][stars-shield]][stars-url]\n[![Issues][issues-shield]][issues-url]\n[![MIT License][license-shield]][license-url]\n[![LinkedIn][linkedin-shield]][linkedin-url]\n\n\n\n\u003c!-- PROJECT LOGO --\u003e\n\u003cbr /\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/grilledchickenthighs/cryptopanic_scraper\"\u003e\n    \u003cimg src=\"images/logo.png\" alt=\"Logo\" width=\"80\" height=\"80\"\u003e\n  \u003c/a\u003e\n\n  \u003ch3 align=\"center\"\u003eCryptopanic Scraper\u003c/h3\u003e\n\n  \u003cp align=\"center\"\u003e\n    Headless chromedriver for automatic scraping of cryptopanics asynchronous newsfeed.\n    \u003cbr /\u003e\n    \u003ca href=\"#usage\"\u003e\u003cstrong\u003eExplore the docs »\u003c/strong\u003e\u003c/a\u003e\n    \u003cbr /\u003e\n    \u003cbr /\u003e\n    \u003ca href=\"https://github.com/grilledchickenthighs/cryptopanic_scraper/issues\"\u003eReport Bug\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/grilledchickenthighs/cryptopanic_scraper/issues\"\u003eRequest Feature\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n\n\n\u003c!-- TABLE OF CONTENTS --\u003e\n## Table of Contents\n\n* [About the Project](#about-the-project)\n  * [Built With](#built-with)\n* [Getting Started](#getting-started)\n  * [Prerequisites](#prerequisites)\n  * [Installation](#installation)\n* [Usage](#usage)\n* [Roadmap](#roadmap)\n* [Contributing](#contributing)\n* [License](#license)\n* [Contact](#contact)\n\n\n\n\u003c!-- ABOUT THE PROJECT --\u003e\n## About The Project\n\n[![Product Name Screen Shot][product-screenshot]](https://cryptopanic.com/)\n\nCryptopanic is a crypto news aggregator that offers realtime news feeds of all things crypto as well \nas user input for ratings.\nThis project was designed to scrape the data from their website so it could be later analyzed using NLP.\n\n### Built With\n\n* [Python](https://github.com/topics/python)\n* [Selenium](https://github.com/topics/selenium)\n\n\n\n\u003c!-- GETTING STARTED --\u003e\n## Getting Started\n\nTo get a local copy up and running follow these simple steps.\n\n### Prerequisites\n\n\n* python 3\n* pip \n\n\n### Installation\n \n1. Clone the cryptopanic_scraper\n    ```sh\n    git clone https:://github.com/grilledchickenthighs/cryptopanic_scraper.git\n    ```\n2. Change directory\n    ```sh\n    cd cryptopanic_scraper\n    ```\n3. Install packages\n    ```sh\n    pip install -r requirements.txt\n    ```\n\n\n\n\u003c!-- USAGE EXAMPLES --\u003e\n## Usage\nSimply run:\n```sh\npython cryptopanic_scraper.py --headless\n```\nIf you want to see it in action, run the script without any flags.\n```sh\npython cryptopanic_scraper.py \n```\nIf you want to filter the type of news to scrape add the --filter flag and choose\na type. {all,hot,rising,bullish,bearish,lol,commented,important,saved}\n```sh\npython cryptopanic_scraper.py --filter hot\n```\nYou can always use the --help flag if you forget these commands:\n```sh\npython cryptopanic_scraper.py --help\n\nusage: cryptopanic_webdriver.py [-h] [-v]\n                                [-f {all,hot,rising,bullish,bearish,lol,commented,important,saved}]\n                                [-s]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -v, --verbose         increase output verbosity\n  -f {all,hot,rising,bullish,bearish,lol,commented,important,saved}, --filter {all,hot,rising,bullish,bearish,lol,commented,important,saved}\n                        Type of News filter\n  -s, --headless        Run Chrome driver headless\n```\n\nIf your interested in analyzing the data:\n\nPlease feel free to check out the [jupyter](https://github.com/GrilledChickenThighs/cryptopanic_scraper/tree/master/jupyter) directory for getting started.\n\n\u003c!-- ROADMAP --\u003e\n## Roadmap\n\nSee the [open issues](https://github.com/grilledchickenthighs/cryptopanic_scraper/issues) for a list of proposed features (and known issues).\n\n\n\n\u003c!-- CONTRIBUTING --\u003e\n## Contributing\n\nContributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are **greatly appreciated**.\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n\n\n\u003c!-- LICENSE --\u003e\n## License\n\nDistributed under the MIT License. See `LICENSE` for more information.\n\n\n\n\u003c!-- CONTACT --\u003e\n## Contact\n\n[Paul Mendes](https://grilledchickenthighs.github.io/) - [@BTCTradeNation](https://twitter.com/BTCTradeNation) - [paulsperformance@gmail.com](mailto:paulseperformance@gmail.com)\n\nProject Link: [https://github.com/grilledchickenthighs/cryptopanic_scraper](https://github.com/grilledchickenthighs/cryptopanic_scraper)\n\n\n\n\u003c!-- MARKDOWN LINKS \u0026 IMAGES --\u003e\n\u003c!-- https://www.markdownguide.org/basic-syntax/#reference-style-links --\u003e\n[contributors-shield]: https://img.shields.io/github/contributors/grilledchickenthighs/cryptopanic_scraper?style=flat-square\n[contributors-url]: https://github.com/GrilledChickenThighs/cryptopanic_scraper/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/grilledchickenthighs/cryptopanic_scraper?style=flat-sqaure\n[forks-url]: https://github.com/GrilledChickenThighs/cryptopanic_scraper/network/members\n[stars-shield]: https://img.shields.io/github/stars/grilledchickenthighs/cryptopanic_scraper?style=flat-square\n[stars-url]: https://github.com/grilledchickenthighs/cryptopanic_scraper/stargazers\n[issues-shield]: https://img.shields.io/github/issues/grilledchickenthighs/cryptopanic_scraper.svg?style=flat-square\n[issues-url]: https://github.com/grilledchickenthighs/cryptopanic_scraper/issues\n[license-shield]: https://img.shields.io/github/license/grilledchickenthighs/cryptopanic_scraper.svg?style=flat-square\n[license-url]: https://github.com/grilledchickenthighs/cryptopanic_scraper/blob/master/LICENSE.txt\n[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=flat-square\u0026logo=linkedin\u0026colorB=555\n[linkedin-url]: https://linkedin.com/in/paul-mendes\n[product-screenshot]: images/screenshot.png","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaulseperformance%2Fcryptopanic_scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaulseperformance%2Fcryptopanic_scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaulseperformance%2Fcryptopanic_scraper/lists"}