https://github.com/prophetru/webscraper
Web Scraper - just for fun
https://github.com/prophetru/webscraper
cpp cpr csv gtest http https libxml2 scraper vcpkg web
Last synced: 6 months ago
JSON representation
Web Scraper - just for fun
- Host: GitHub
- URL: https://github.com/prophetru/webscraper
- Owner: ProphetRu
- License: apache-2.0
- Created: 2024-12-22T19:31:48.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2024-12-23T07:33:50.000Z (about 1 year ago)
- Last Synced: 2025-06-17T03:06:36.311Z (7 months ago)
- Topics: cpp, cpr, csv, gtest, http, https, libxml2, scraper, vcpkg, web
- Language: C++
- Homepage:
- Size: 28.3 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Web Scraper
WebScraper - a program that parses (scrapes) a website (`fust for fun`)
* Sends a request to a URL
* Receives an HTML page in response
* Parses the HTML page and finds the necessary data
* Generates and saves data in CSV format
## Dependencies
* [googletest](https://github.com/google/googletest)
* [cpr](https://github.com/libcpr/cpr)
* [libxml2](https://github.com/GNOME/libxml2)
## Build local Windows/Linux
```shell
vcpkg install gtest cpr libxml2
vcpkg integrate install
cd WebScraper
mkdir build && cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE="path/to/vcpkg/scripts/buildsystems/vcpkg.cmake"
cmake --build . --config Release
```
## Testing
```shell
ctest
```