https://github.com/raphapassini/pyjobs

Its a crawler with the goal of extract offers of python jobs from websites, mostly Brazilian websites.
https://github.com/raphapassini/pyjobs

Last synced: 12 days ago
JSON representation

Its a crawler with the goal of extract offers of python jobs from websites, mostly Brazilian websites.

• Host: GitHub
• URL: https://github.com/raphapassini/pyjobs
• Owner: raphapassini
• License: mit
• Created: 2014-07-16T04:25:01.000Z (over 10 years ago)
• Default Branch: master
• Last Pushed: 2015-08-24T14:40:10.000Z (about 9 years ago)
• Last Synced: 2024-08-01T22:53:47.312Z (3 months ago)
• Language: JavaScript
• Size: 1.01 MB
• Stars: 20
• Watchers: 3
• Forks: 5
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
pyjobs

======

Its a crawler with the goal of extract offers of python jobs from websites, mostly Brazilian websites.

How to install

---------------

1) Check if you have libxml2-dev, libffi-dev, libssl-dev libxml2-dev libxslt-dev and mongodb, if you doesn't install it:

```sudo apt-get install libxml2-dev libffi-dev libssl-dev libxml2-dev libxslt-dev mongodb```

2) Install project requirements

```pip install -r requirements.txt```

Please, be kind with yourself and install it in an virtualenv! :)

How to run it

--------------

```scrapy crawl ceviu```

```scrapy crawl catho```

```scrapy crawl vagas```

```scrapy crawl empregos```

ROADMAP

-------

[x] - Iterate over CEVIU search pages

[x] - Store items in database, preferably a NoSQL database such as MongoDB

[x] - Implement Catho.com.br spider

[x] - Implement Empregos.com.br spider

[x] - Implement Vagas.com.br spider

[] - Build an web interface to search for jobs