https://github.com/ifoukarakis/jobscrapper

An automated job scrapper
https://github.com/ifoukarakis/jobscrapper

git-scraping scrapy

Last synced: 5 days ago
JSON representation

An automated job scrapper

• Host: GitHub
• URL: https://github.com/ifoukarakis/jobscrapper
• Owner: ifoukarakis
• Created: 2020-08-31T14:48:01.000Z (about 4 years ago)
• Default Branch: master
• Last Pushed: 2022-11-29T09:47:31.000Z (almost 2 years ago)
• Last Synced: 2024-08-02T12:44:55.515Z (3 months ago)
• Topics: git-scraping, scrapy
• Language: Python
• Homepage:
• Size: 708 KB
• Stars: 32
• Watchers: 3
• Forks: 11
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-starred - ifoukarakis/jobscrapper - An automated job scrapper (others)

README

        
# Openings

Experimenting with scraping job openings from the web.

# Prerequisites

- Python 3.6+ (was developed using python 3.8)

- pip/virtualenv installed

# Installing

Create a virtualenv and install requirements. You can do this by running

```bash

virtualenv env

. env/bin/activate

pip install -r requirements

```

# Running spiders

There are two spiders that you can use for now, one for Workable-backed career pages and one for Recruitee.

To run Workable-based crawler, edit openings/spiders/workable.py run: 

```bash

scrapy runspider jobscrapper/spiders/workable.py

```

Similarly for recruitee just run:

```bash

scrapy runspider jobscrapper/spiders/recruitee.py

```

All jobs will be stored under `data/`, on a separate file per company.