https://github.com/stummjr/flake8-scrapy

A Flake8 plugin to catch common issues on Scrapy spiders
https://github.com/stummjr/flake8-scrapy

flake8 flake8-plugin linter linter-plugin scrapy

Last synced: 3 months ago
JSON representation

A Flake8 plugin to catch common issues on Scrapy spiders

• Host: GitHub
• URL: https://github.com/stummjr/flake8-scrapy
• Owner: stummjr
• License: mit
• Created: 2018-09-30T20:27:09.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2022-04-11T15:31:51.000Z (over 2 years ago)
• Last Synced: 2024-07-08T12:09:55.126Z (4 months ago)
• Topics: flake8, flake8-plugin, linter, linter-plugin, scrapy
• Language: Python
• Homepage:
• Size: 24.4 KB
• Stars: 19
• Watchers: 3
• Forks: 2
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-flake8-extensions - flake8-scrapy - Plugin to catch common issues on Scrapy spiders. (Library-specific checks)

README

        
# scrapy-flake8

![](https://github.com/stummjr/flake8-scrapy/workflows/CI/badge.svg)

[![Downloads](https://pepy.tech/badge/flake8-scrapy)](https://pepy.tech/project/scrapy-fieldstats)

A Flake8 plugin to catch common issues on Scrapy spiders.

## Issue types

| Code  | Meaning |

| ---   | --- |

| SCP01 | There are URLs in `start_urls` whose netloc is not in `allowed_domains` |

| SCP02 | There are URLs in `allowed_domains` |

| SCP03 | Usage of `urljoin(response.url, '/foo')` instead of `response.urljoin('/foo')` |

| SCP04 | Usage of `Selector(response)` in callback |

This is a work in progress, so new issues will be added to this list.

## Installation

To run this in your project, please make sure you have flake8 installed first:

```

$ pip install flake8

```

And then install flake8-scrapy:

```

$ pip install flake8-scrapy

```

Now, all you have to do is run it on your project:

```

$ flake8

```

And Flake8 will run the checks defined in this plugin.