https://github.com/sgerodes/scrapy-dot-items

https://github.com/sgerodes/scrapy-dot-items

Last synced: 6 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/sgerodes/scrapy-dot-items
• Owner: sgerodes
• Created: 2022-07-23T12:54:59.000Z (almost 4 years ago)
• Default Branch: main
• Last Pushed: 2022-07-26T08:27:29.000Z (almost 4 years ago)
• Last Synced: 2025-02-23T09:34:35.798Z (over 1 year ago)
• Language: Python
• Size: 49.8 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Title

## Description

For those who are tired to use scrapy's brackets and quotes. 

In one line of code this library will increase the readability of your code, make it more stable to human error.

And will introduce IDEs code completion features to your scrapy items. 

You will be able to use scrapy items as normal python objects using dot syntax.

The regular bulky way to do it:

    regular_scrapy_item['some_field'] = 42

    print(regular_scrapy_item.get('some_field'))

The simple dot-items way:

    scrapy_dot_item.some_field = 42

    print(scrapy_dot_item.some_field)

## Installation

    pip install scrapy-dot-items

## Usage

### Apply to a single scrapy item class

You can add functionality with the dot_item decorator to a single class

    

    from scrapy_dot_items import dot_item

    @dot_item

    class RealEstateItem(scrapy.Item):

        price = scrapy.Field()

    dot_scrapy_item = RealEstateItem()

    dot_scrapy_item.price = 1000

    print(dot_scrapy_item.price)  # prints 1000

### Apply to all scrapy items 

You can apply this functionality globally

    from scrapy_dot_items import dot_items_globally

    dot_items_globally()  # now every scrapy.Item class will have this functionality

    class RealEstateItem(scrapy.Item):

        price = scrapy.Field()

    dot_scrapy_item = RealEstateItem()

    dot_scrapy_item.price = 1000

    print(dot_scrapy_item.price)  # prints 1000

## Backwards compatibility

scrapy-dot-items are backwards compatible. You can install it into your old project and mix regular and dot style. 

    

    dot_scrapy_item['price'] = 2000  # you can still set and get the items the regular way too

    print(dot_scrapy_item.get('price'))  # prints 2000

    print(dot_scrapy_item.price)  # prints 2000

## For contributers

This project uses pipenv instead of pip. Setup:

- intall pipenv with 'pip install pipenv'

- create a virtual environment using pipenv with 'pipenv shell' in the root directory

- install the dependencies with 'pipenv install'

Attention:

Please install all packages with 'pipenv install '. Do NOT use 'pip install ...'.

The same for unistalling 'pipenv uninstall '