https://github.com/install-it/it-claws
Automates the process of find and download the latest softwares and tools.
https://github.com/install-it/it-claws
driver python scraper selenium webscrapping
Last synced: 9 days ago
JSON representation
Automates the process of find and download the latest softwares and tools.
- Host: GitHub
- URL: https://github.com/install-it/it-claws
- Owner: install-it
- License: gpl-2.0
- Created: 2025-05-15T05:07:29.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2026-06-12T17:03:11.000Z (12 days ago)
- Last Synced: 2026-06-12T19:06:10.439Z (12 days ago)
- Topics: driver, python, scraper, selenium, webscrapping
- Language: Python
- Homepage:
- Size: 248 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# it-claws
[![Tag][tag-shield]][tag-url]
[![Contributors][contributors-shield]][contributors-url]
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
[![License][license-shield]][license-url]
it-claws
Automated concurrent scraper for staging PC driver deployment environments
Report Bug
·
Request Feature
## About The Project
it-claws is a Python-based tool that automatically downloads the latest PC hardware drivers, diagnostic tools, and common software from official vendor websites. Using Selenium for browser automation and httpx for static requests, it navigates vendor sites to retrieve up-to-date installation packages — suitable for staging driver packs for enterprise deployment.
Unlike simple download utilities, it-claws runs concurrently with configurable retry logic, supports dynamic and static page scraping, and can pack everything into a compressed ZIP archive via 7z.
The tool serves as a companion to [install-it](https://github.com/install-it/install-it/). See the [Usage](#usage) section for more information.
it-claws also supports **Docker** deployment with tmpfs RAM disk and automated cloud upload via rclone. See [scripts](scripts/) for the automation pipeline.
### Built With
[
](https://www.7-zip.org/)
[
](https://www.python.org/)
[
](https://www.selenium.dev/)
[
](https://www.python-httpx.org/)
## Usage
### Basic flow
1. **Select targets**
- Use `-t` to specify space-separated target names
- Use `-i` for interactive selection
- Use `--all` to select all targets
- Use `--target-from ` to load a preset text file
2. **Download**
- The tool resolves download URLs (static or dynamic)
- Files are downloaded concurrently with configurable retries
3. **Archive & output**
- Results are staged in the output directory
- Optionally pack into a compressed ZIP archive
### Selecting targets
```sh
it-claws -t "AMD Chipset Drivers" "Realtek HD Universal Audio"
it-claws -i
it-claws --all
```
Use a preset file (one target per line, `#` for comments):
```sh
it-claws --target-from presets/default.txt
```
Combine with interactive to tweak the preset before running:
```sh
it-claws --target-from presets/default.txt -i
```
### Output options
```sh
it-claws -o ./my-drivers -f my-custom-folder --max-concurrent 10
```
### Resilience & archiving
```sh
it-claws -o ./downloads -a ./driver-pack.zip --retries 2 -l 9
```
- `--retries`: retry attempts per failed download (default: `1`)
- `-a` / `--archive-path`: destination for the output ZIP archive
- `-l` / `--compress-level`: 7z compression level `0`–`9` (default: `5`)
### Extract RAR files on Linux
The `7z` package may not support RAR format. Install `p7zip-full p7zip-rar` as an alternative.
## Deployment
This project can be deployed via Docker with tmpfs RAM disk and automated rclone upload. See [`scripts/`](scripts/) for the automation pipeline and environment variable reference.
```bash
docker run --rm --privileged \
-v /my/host/config:/config \
-e RC_REMOTE_PATH="onedrive:PC_Deployments" \
-e RETRIES="2" \
-e COMPRESS_LEVEL="9" \
it-claws
```
Override the default command:
```bash
docker run --name=it-claws \
-v /path/to/config:/config \
ghcr.io/install-it/it-claws:latest \
it-claws -t "AMD Chipset Drivers" "Realtek HD Universal Audio"
```
## Getting Started
### Prerequisites
- [uv](https://docs.astral.sh/uv/)
- [7zip](https://www.7-zip.org/download.html)
- A supported web browser (Chrome, Edge, or Firefox)
### Install dependencies
```bash
uv sync
uv sync --group dev # install dev dependencies
```
### Common commands
**Run the scraper**
```bash
it-claws
```
**Display help**
```bash
it-claws -h
```
[tag-url]: https://github.com/install-it/it-claws/releases
[tag-shield]: https://img.shields.io/github/v/tag/install-it/it-claws?style=for-the-badge&label=LATEST&color=%23B1B1B1
[contributors-shield]: https://img.shields.io/github/contributors/install-it/it-claws.svg?style=for-the-badge
[contributors-url]: https://github.com/install-it/it-claws/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/install-it/it-claws.svg?style=for-the-badge
[forks-url]: https://github.com/install-it/it-claws/network/members
[stars-shield]: https://img.shields.io/github/stars/install-it/it-claws.svg?style=for-the-badge
[stars-url]: https://github.com/install-it/it-claws/stargazers
[issues-shield]: https://img.shields.io/github/issues/install-it/it-claws.svg?style=for-the-badge
[issues-url]: https://github.com/install-it/it-claws/issues
[license-shield]: https://img.shields.io/github/license/install-it/it-claws.svg?style=for-the-badge
[license-url]: https://github.com/install-it/it-claws/blob/master/LICENSE.txt