https://github.com/youstinus/car-scrape

Scrapes website content, puts to sqlite3 database, downloads preview picture
https://github.com/youstinus/car-scrape

pprof scraper sqlite3 sqlite3-database webpage-scraper website-scraper

Last synced: 1 day ago
JSON representation

Scrapes website content, puts to sqlite3 database, downloads preview picture

• Host: GitHub
• URL: https://github.com/youstinus/car-scrape
• Owner: youstinus
• License: mit
• Created: 2021-12-25T12:30:56.000Z (almost 4 years ago)
• Default Branch: master
• Last Pushed: 2021-12-25T12:46:15.000Z (almost 4 years ago)
• Last Synced: 2025-01-29T13:27:22.536Z (8 months ago)
• Topics: pprof, scraper, sqlite3, sqlite3-database, webpage-scraper, website-scraper
• Language: Go
• Homepage:
• Size: 6.84 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Car Scrape

Using `github.com/PuerkitoBio/goquery` package to scrape webpages.

Application used to scrape website data, parse and write to database.

All data is written to sqlite3 database and some pictures can be downloaded as well.

Sometimes it does not work because of site restrictions or captchas.

Some basic performance analysis used to measure application using pprof tool.

Project from 2021 archive