https://github.com/sribalakumar/justdialscrapper

Scrapper and D3 visualization
https://github.com/sribalakumar/justdialscrapper

Last synced: 9 days ago
JSON representation

Scrapper and D3 visualization

• Host: GitHub
• URL: https://github.com/sribalakumar/justdialscrapper
• Owner: sribalakumar
• Created: 2013-08-14T18:46:01.000Z (about 11 years ago)
• Default Branch: master
• Last Pushed: 2022-01-19T13:09:41.000Z (almost 3 years ago)
• Last Synced: 2024-08-02T14:05:59.921Z (3 months ago)
• Language: CSS
• Size: 271 KB
• Stars: 2
• Watchers: 2
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
#Python Scraper and D3 visualization using Play 1.2.5

This is a Play 1.2.5 app and the scraper written in python resides in app/controllers/JustDialScrap.py

>TO RUN SCRAPER

repohome#>cd app/controllers/

repohome#>python JustDialScrap.py

City: User Input I

Category: User Input II 

>Example:

repohome#>python JustDialScrap.py

city: chennai

Category: cabs

Above commands runs the scraper and puts the scraped data to Chennai-Cabs.csv file and also generates Chennai-Cabs.json file which has the json data from JustDials top 5 pages(max) verified vs nonverified count for the city Chennai and category cabs. 

>TO RUN PLAY

repohome#> path to play 1.2.5/play run

which runs the webapp with default play dev settings on port 9000.

>TO VIEW THE CHART

open broswer url: localhost:9000

Json files generated by scraper is read by Play controller(app/controllers/Application.java) and then passed via Groovy templating to D3 chart scripts which is inside /app/views/Application/index.html. The play app uses D3 Pie-Chart to visualize the verified vs nonverified count.