Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
https://github.com/scrapy/scrapy
crawler crawling framework hacktoberfest python scraping web-scraping web-scraping-python
Last synced: 25 days ago
JSON representation
Scrapy, a fast high-level web crawling & scraping framework for Python.
- Host: GitHub
- URL: https://github.com/scrapy/scrapy
- Owner: scrapy
- License: bsd-3-clause
- Created: 2010-02-22T02:01:14.000Z (over 14 years ago)
- Default Branch: master
- Last Pushed: 2024-05-08T19:08:49.000Z (26 days ago)
- Last Synced: 2024-05-09T11:04:05.495Z (25 days ago)
- Topics: crawler, crawling, framework, hacktoberfest, python, scraping, web-scraping, web-scraping-python
- Language: Python
- Homepage: https://scrapy.org
- Size: 24.6 MB
- Stars: 51,015
- Watchers: 1,775
- Forks: 10,342
- Open Issues: 660
-
Metadata Files:
- Readme: README.rst
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
- Authors: AUTHORS
Lists
- awesome-for-beginners - scrapy - first-issue)_ <br> A fast high-level web crawling & scraping framework for Python. (Python)
- awesome-asyncio-cn - Scrapy - 一个为了爬取网站数据,提取结构性数据而编写的应用框架。 (爬虫)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 44538 | (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 44538 | (Python)
- awesome-crawler - Scrapy - A fast high-level screen scraping and web crawling framework. (Python)
- my-awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 48605 | (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-crawler-cn - Scrapy - 一种高效的屏幕,网页数据采集框架。 (Python)
- awesome-list - scrapy - high-level library to write (Data Gathering)
- awesome - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-repositories - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-starred - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 44538 | (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 38318 | (Python)
- awesome-five23 - scrapy - level web crawling & scraping framework for Python. | scrapy | 39032 | (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- Starred-Repo - Scrapy: Python Web Crawler
- my-awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars-copy - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-resources - scrapy
- my-awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 51273 | (Python)
- awesome-starts - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 45007 | (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- alex-mikhalev-awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 43238 | (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stenudd - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-web-scraper - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- docsify-awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- my-awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- artsz-awesome - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 36652 | (Python)
- my-awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-starts - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (python)
- awesome-from-stars - scrapy/scrapy - level web crawling & scraping framework for Python. (HarmonyOS / Windows Manager)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-projects - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-python-again -
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-crawler - Scrapy - A fast high-level screen scraping and web crawling framework. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 40033 | (Python)
- my-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-starred - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 51280 | (Python)
- awesome-stars - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome - scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-starred - scrapy - level web crawling & scraping framework for Python. | scrapy | 51221 | (Python)
- awesome-list - Scrapy - A fast high-level web crawling & scraping framework for Python. (Data Processing / Data Pre-processing & Loading)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-python - scrapy - A fast high-level screen scraping and web crawling framework. (Web Crawling)
- awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 51266 | (Python)
- my-awesome - scrapy/scrapy - scraping,web-scraping-python pushed_at:2024-05 star:51.3k fork:10.4k Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy/scrapy - `★51299` Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- my-awesome-stars - scrapy/scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python. (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 51168 | (Python)
- awesome-stars - scrapy - level web crawling & scraping framework for Python. | scrapy | 51273 | (Python)
- awesome-crawlers - Scrapy - 08-25 | A fast high-level screen scraping and web crawling framework. | (Python)
README
.. image:: https://scrapy.org/img/scrapylogo.png
:target: https://scrapy.org/
======
Scrapy
======.. image:: https://img.shields.io/pypi/v/Scrapy.svg
:target: https://pypi.python.org/pypi/Scrapy
:alt: PyPI Version.. image:: https://img.shields.io/pypi/pyversions/Scrapy.svg
:target: https://pypi.python.org/pypi/Scrapy
:alt: Supported Python Versions.. image:: https://github.com/scrapy/scrapy/workflows/Ubuntu/badge.svg
:target: https://github.com/scrapy/scrapy/actions?query=workflow%3AUbuntu
:alt: Ubuntu.. .. image:: https://github.com/scrapy/scrapy/workflows/macOS/badge.svg
.. :target: https://github.com/scrapy/scrapy/actions?query=workflow%3AmacOS
.. :alt: macOS.. image:: https://github.com/scrapy/scrapy/workflows/Windows/badge.svg
:target: https://github.com/scrapy/scrapy/actions?query=workflow%3AWindows
:alt: Windows.. image:: https://img.shields.io/badge/wheel-yes-brightgreen.svg
:target: https://pypi.python.org/pypi/Scrapy
:alt: Wheel Status.. image:: https://img.shields.io/codecov/c/github/scrapy/scrapy/master.svg
:target: https://codecov.io/github/scrapy/scrapy?branch=master
:alt: Coverage report.. image:: https://anaconda.org/conda-forge/scrapy/badges/version.svg
:target: https://anaconda.org/conda-forge/scrapy
:alt: Conda VersionOverview
========Scrapy is a BSD-licensed fast high-level web crawling and web scraping framework, used to
crawl websites and extract structured data from their pages. It can be used for
a wide range of purposes, from data mining to monitoring and automated testing.Scrapy is maintained by Zyte_ (formerly Scrapinghub) and `many other
contributors`_... _many other contributors: https://github.com/scrapy/scrapy/graphs/contributors
.. _Zyte: https://www.zyte.com/Check the Scrapy homepage at https://scrapy.org for more information,
including a list of features.Requirements
============* Python 3.8+
* Works on Linux, Windows, macOS, BSDInstall
=======The quick way:
.. code:: bash
pip install scrapy
See the install section in the documentation at
https://docs.scrapy.org/en/latest/intro/install.html for more details.Documentation
=============Documentation is available online at https://docs.scrapy.org/ and in the ``docs``
directory.Releases
========You can check https://docs.scrapy.org/en/latest/news.html for the release notes.
Community (blog, twitter, mail list, IRC)
=========================================See https://scrapy.org/community/ for details.
Contributing
============See https://docs.scrapy.org/en/master/contributing.html for details.
Code of Conduct
---------------Please note that this project is released with a Contributor `Code of Conduct `_.
By participating in this project you agree to abide by its terms.
Please report unacceptable behavior to [email protected].Companies using Scrapy
======================See https://scrapy.org/companies/ for a list.
Commercial Support
==================See https://scrapy.org/support/ for details.