https://github.com/rahul-jha98/restauranttrends.stats-backend

Application that scrapes the Zomato Dataset and enables the user to visualise the results.
https://github.com/rahul-jha98/restauranttrends.stats-backend

data-analysis data-extraction firebase-storage web-scraping zomato-api

Last synced: about 2 months ago
JSON representation

Application that scrapes the Zomato Dataset and enables the user to visualise the results.

• Host: GitHub
• URL: https://github.com/rahul-jha98/restauranttrends.stats-backend
• Owner: rahul-jha98
• Created: 2020-04-12T16:17:53.000Z (almost 5 years ago)
• Default Branch: master
• Last Pushed: 2022-01-22T12:31:38.000Z (almost 3 years ago)
• Last Synced: 2023-09-07T10:21:04.301Z (over 1 year ago)
• Topics: data-analysis, data-extraction, firebase-storage, web-scraping, zomato-api
• Language: HTML
• Homepage:
• Size: 37.7 MB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Backend Repostiory for web app: RestaurantTrends.stats 

This repostiory is the backend server code for powering the web app RestaurantTrends.stats: https://github.com/rahul-jha98/RestaurantTrends.stats

## Understanding the role of the Backend:

 - **FlaskServer.py**: This file is the Flask Server which will provide the API for the frontend to request new anlysis of a given city name. Whenever a city is requested we spawn a new child process and tell it to execute ZomatoDataAnalyzer passing in the city name as param. This way if we have more than one requests we can handle it with multiple child processes.

 - **ZomatoDataAnalyzer.py** : This file has two responsibilities. First it uses the Zomato API and web scraping to prepare a csv of necessary information for a given city. Then we read the csv and perform data analysis on it. All the plots are saved in a directory and a dictionary containing the titles is also dumped in a json. 

 With both of these done we have our analysis done. But the data has to be uploaded to firebase for our frontend to access it. So, what we do is we call a node file DataUploader.js to upload the data to firebase storage.

 - **DataUploader.js** : This part handles the job of uploading the files to firebase storage and save the link to it in firebase for our frontend to access. 

 

![Terminal when Request is made in backend](Notebooks/ServerSnap.png)

The reason we chose this apporach is that the Data Analysis requires lot of dependecies and good computing power which would mean the cost of such server would be higher. On the other hand for simply fetching the images the extra computing power is not necessary. So, separating the data hosting backend from analysis we ensure that the very low sized server with good CPU can be used to host this one since images and data are not permanently saved here.