https://github.com/openzim/ifixit
iFixit to ZIM scraper
https://github.com/openzim/ifixit
ifixit kiwix openzim scraper zim
Last synced: 10 months ago
JSON representation
iFixit to ZIM scraper
- Host: GitHub
- URL: https://github.com/openzim/ifixit
- Owner: openzim
- License: gpl-3.0
- Created: 2022-04-01T07:29:52.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2025-03-27T10:06:28.000Z (about 1 year ago)
- Last Synced: 2025-05-01T02:39:05.942Z (about 1 year ago)
- Topics: ifixit, kiwix, openzim, scraper, zim
- Language: Python
- Homepage:
- Size: 1.14 MB
- Stars: 30
- Watchers: 3
- Forks: 5
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# iFixit
`ifixit2zim` is an [openZIM](https://openzim.org) scraper to create offline versions of [iFixit](https://www.ifixit.com/) website, in all its supported languages.
[](https://www.codefactor.io/repository/github/openzim/ifixit)
[](https://www.gnu.org/licenses/gpl-3.0)
[](https://codecov.io/gh/openzim/ifixit)
[](https://pypi.org/project/ifixit2zim/)
[](https://pypi.org/project/ifixit2zim)
[](https://ghcr.io/openzim/ifixit)
This scraper downloads the iFixit resources (categories, guides, ...) and puts them in a ZIM file, a clean and user friendly format for storing content for offline usage.
## Usage
`ifixit2zim` works off a *language version* that you must provide via the `--language` argument. The list of supported languages is visible in the `--help` message.
### Docker
```bash
docker run -v my_dir:/output ghcr.io/openzim/ifixit ifixit2zim --help
```
### Python
`ifixit2zim` is a Python3 (**3.6+**) software. If you are not using the [Docker](https://docker.com) image, you are advised to use it in a virtual environment to avoid installing software dependencies on your system. In addition to Python3, you also need to have an up-to-date installation of pip, setuptools and wheel as recommanded [here](https://packaging.python.org/en/latest/tutorials/installing-packages/#id14) (wheel is important since you will have to build some dependencies).
```bash
python3 -m venv .venv
source .venv/bin/activate
# using published version
pip3 install ifixit2zim
ifixit2zim --help
# running from source
pip3 install -e ".[dev]"
python3 ifixit2zim/ --help
```
Call `deactivate` to quit the virtual environment.
See `requirements.txt` for the list of python dependencies.
## Contributing
**All contributions are welcome!**
Please open an issue on Github and/or submit a Pull-request.
This project adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing).
This project has implemented openZIM's [Python bootstrap, conventions and policies](https://github.com/openzim/_python-bootstrap/blob/main/docs/Policy.md) **v1.0.0**.
### Guidelines
- Don't take assigned issues. Comment if those get staled.
- If your contribution is far from trivial, open an issue to discuss it first.
- Ensure your code passed [black formatting](https://pypi.org/project/black/), [isort](https://pypi.org/project/isort/) and [flake8](https://pypi.org/project/flake8/) (88 chars)
### Create an appropriate Python environment
First time:
```
python3 -m venv .venv
source .venv/bin/activate
pip3 install -e ".[dev]"
```
Next times:
```
source .venv/bin/activate
```
NOTA : there is some limitations to the execution of the underlying libzim library on
MacOS with some known bugs. The main issue is that the full-text index is not working,
so this shouldn't be a problem for quick tests. In doubt, execute the scraper in a
Docker container as explained below.
### Test the scraper in a Docker container
First, build the Docker image (to be ran in the main folder of this repo):
```
docker build -t local-ifixit .
```
Then run the scraper with CLI arguments needed for your test (everything after `ifixit2zim` in the example below).
For instance, if you want to run a scrape of only the `Apple_PDA` category, including its guides,
in French :
```
docker run -it -v $(pwd)/output:/output --rm local-ifixit ifixit2zim --language fr --output /output --tmp-dir /tmp --category Apple_PDA
```
This will produce a ZIM in the output folder of your current directory.
### Test the ZIM produced
To test if the ZIM produced is OK, you should run kiwix-serve, once more with Docker.
For instance, if you produced a file named `ifixit_fr_selection_2022-04.zim` in the
`output` subfolder, and port 1256 is unused on your machine, you might run:
```
docker run -it --rm -v $(pwd)/output:/data -p 1256:80 ghcr.io/kiwix/kiwix-tools kiwix-serve /data/ifixit_fr_selection_2022-04.zim
```
And then navigate to (https://localhost:1256) on your favorite browser.
Once test are complete, you might stop the Docker container by pressing Ctrl-C