https://github.com/linux0hat/cpp-web-crawler

Explore the web.
https://github.com/linux0hat/cpp-web-crawler

cpp crawler sqlite3

Last synced: 15 days ago
JSON representation

Explore the web.

• Host: GitHub
• URL: https://github.com/linux0hat/cpp-web-crawler
• Owner: Linux0Hat
• License: apache-2.0
• Created: 2024-05-14T21:00:08.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-05-14T21:20:59.000Z (about 2 years ago)
• Last Synced: 2025-03-02T09:44:50.923Z (over 1 year ago)
• Topics: cpp, crawler, sqlite3
• Language: C++
• Homepage:
• Size: 57.6 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Web Crawler

Web crawler explore the web and save every pages he found.

## Build

### Dependences

- `cmake`

- `ninja` or `make`

- `gcc`

- `sqlite3`

- `curl`

- [`SQLiteCpp`](https://github.com/SRombauts/SQLiteCpp) clone on /libs/SQLiteCpp

### Build

- **with Ninja :**

```

mkdir build

cd build

cmake -G Ninja ..

ninja

```

- **with Make**

```

mkdir build

cd build

cmake ..

make

```

## Use

```

web-crawler  --clear-waitlist