https://github.com/arthurnn/podcast

web crawler to grab itunes podcast info
https://github.com/arthurnn/podcast

Last synced: 14 days ago
JSON representation

web crawler to grab itunes podcast info

• Host: GitHub
• URL: https://github.com/arthurnn/podcast
• Owner: arthurnn
• Created: 2011-12-11T07:16:13.000Z (almost 13 years ago)
• Default Branch: master
• Last Pushed: 2013-11-04T23:30:11.000Z (about 11 years ago)
• Last Synced: 2024-10-04T13:25:36.801Z (about 1 month ago)
• Language: Python
• Homepage:
• Size: 528 KB
• Stars: 20
• Watchers: 6
• Forks: 6
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
## Dependencies

	1. Python

	2. python-MySQLdb

		http://sourceforge.net/projects/mysql-python/

	3. python Scrapy

		http://scrapy.org/download/

## Installation

	1. create db

		$ mysql -uroot -proot

		mysql> create database podcastdb;

	2. create table

		$ mysql -uroot -proot podcastdb < db/mysql.sql

	

	3. start crawler server

		$ scrapy server

		

	4. register the spider in the server

		$ curl http://localhost:6800/schedule.json -d project=default -d spider=itunes

		

	5. Just keep watching the spiders making their work:

		http://localhost:6800/

[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/arthurnn/podcast/trend.png)](https://bitdeli.com/free "Bitdeli Badge")