https://github.com/scrapinghub/web-poet
Web scraping Page Objects core library
https://github.com/scrapinghub/web-poet
hacktoberfest page-objects python web-scraping
Last synced: about 1 year ago
JSON representation
Web scraping Page Objects core library
- Host: GitHub
- URL: https://github.com/scrapinghub/web-poet
- Owner: scrapinghub
- License: bsd-3-clause
- Created: 2020-04-22T15:39:41.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2025-02-10T16:44:53.000Z (over 1 year ago)
- Last Synced: 2025-04-03T14:09:34.541Z (about 1 year ago)
- Topics: hacktoberfest, page-objects, python, web-scraping
- Language: Python
- Homepage: https://web-poet.readthedocs.io/en/stable/
- Size: 1.18 MB
- Stars: 99
- Watchers: 7
- Forks: 15
- Open Issues: 31
-
Metadata Files:
- Readme: README.rst
- Changelog: CHANGELOG.rst
- Contributing: CONTRIBUTING.rst
- License: LICENSE
Awesome Lists containing this project
README
========
web-poet
========
.. image:: https://img.shields.io/pypi/v/web-poet.svg
:target: https://pypi.python.org/pypi/web-poet
:alt: PyPI Version
.. image:: https://img.shields.io/pypi/pyversions/web-poet.svg
:target: https://pypi.python.org/pypi/web-poet
:alt: Supported Python Versions
.. image:: https://github.com/scrapinghub/web-poet/actions/workflows/tests-ubuntu.yml/badge.svg
:target: https://github.com/scrapinghub/web-poet/actions/workflows/tests-ubuntu.yml
:alt: Tox Ubuntu
.. image:: https://github.com/scrapinghub/web-poet/actions/workflows/tests-windows.yml/badge.svg
:target: https://github.com/scrapinghub/web-poet/actions/workflows/tests-windows.yml
:alt: Tox Windows
.. image:: https://codecov.io/github/scrapinghub/web-poet/coverage.svg?branch=master
:target: https://codecov.io/gh/scrapinghub/web-poet
:alt: Coverage report
.. image:: https://readthedocs.org/projects/web-poet/badge/?version=stable
:target: https://web-poet.readthedocs.io/en/stable/?badge=stable
:alt: Documentation Status
.. intro starts
``web-poet`` is a Python 3.9+ implementation of the `page object pattern`_ for
web scraping. It enables writing portable, reusable web parsing code.
.. _page object pattern: https://martinfowler.com/bliki/PageObject.html
.. intro ends
See the documentation_.
.. _documentation: https://web-poet.readthedocs.io
Developing
==========
Setup your local Python environment via:
1. `pip install -r requirements-dev.txt`
2. `pre-commit install`
Now everytime you perform a `git commit`, these tools will run against the
staged files:
* `black`
* `isort`
* `flake8`
You can also directly invoke `pre-commit run --all-files` or `tox -e linters`
to run them without performing a commit.