https://github.com/musadiq071/hacker-news_scraper
CLI scraper for Hacker News — get top stories, filter by keyword, save to CSV, or open in browser.
https://github.com/musadiq071/hacker-news_scraper
automation cli csv datacollection hacker-news open-source python selenium webscraping
Last synced: 2 months ago
JSON representation
CLI scraper for Hacker News — get top stories, filter by keyword, save to CSV, or open in browser.
- Host: GitHub
- URL: https://github.com/musadiq071/hacker-news_scraper
- Owner: Musadiq071
- License: mit
- Created: 2025-08-25T20:50:16.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-08-25T21:26:17.000Z (11 months ago)
- Last Synced: 2025-08-25T23:26:44.310Z (11 months ago)
- Topics: automation, cli, csv, datacollection, hacker-news, open-source, python, selenium, webscraping
- Language: Python
- Homepage:
- Size: 6.84 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Hacker News Scraper (CLI)
A lightweight Python CLI tool that scrapes top stories from [Hacker News](https://news.ycombinator.com) using Selenium (Safari WebDriver).
Stories are saved to CSV with timestamps, and you can filter by keywords or automatically open them in your browser.
---
## Features
- Scrape any number of top Hacker News stories (e.g., 10, 30, 50)
- Filter stories by keyword
- Save results to CSV with timestamped filenames
- Optionally open all links in your default browser
---
## Installation
1. **Clone the repository**:
```bash
git clone https://github.com/YOUR-USERNAME/hacker-news_scraper.git
cd hacker-news_scraper
```
2. install dependencies:
```bash
pip install -r requirements.txt
```
3. Usage
Run the scraper
```bash
python3 automation.py
```
You will be prompted to enter:
Number of top stories to scrape (default 10)
Whether to open links in your browser
Optional keyword filter
-Example output in terminal:
1: Google's Liquid Cooling -> https://chipsandcheese.com/p/googles-liquid-cooling-at-hot-chips
2: chipsandcheese.com -> https://news.ycombinator.com/from?site=chipsandcheese.com
-Example csv file:
hackernews_top10_20250825_090015.csv
Notes
Requires Safari with "Allow Remote Automation" enabled:
Safari > Develop > Allow Remote Automation
Run once in terminal:
```bash
safaridriver --enable
```
Example Output
Check the hackernews_top10(example_file).csv in the repo for a sample of scraped stories.