https://github.com/douglasdrake/mars
A Flask app to scrape information about Mars
https://github.com/douglasdrake/mars
beautifulsoup4 flask mars nasa splinter
Last synced: 1 day ago
JSON representation
A Flask app to scrape information about Mars
- Host: GitHub
- URL: https://github.com/douglasdrake/mars
- Owner: douglasdrake
- Created: 2019-05-18T07:49:46.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2019-10-21T15:57:05.000Z (over 6 years ago)
- Last Synced: 2025-05-19T19:55:55.321Z (about 1 year ago)
- Topics: beautifulsoup4, flask, mars, nasa, splinter
- Language: Jupyter Notebook
- Homepage:
- Size: 5.41 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# mars
# Description and Methods
* A Flask app is provided that scrapes five sites for information and images about Mars:
1. [NASA's Mars Exploration Program](https://mars.nasa.gov/news)
2. [Jet Propulsion Laboaratory Mars Images](https://www.jpl.nasa.gov/spaceimages/?search=&category=Mars)
3. [Mars Weather Twitter Feed](https://twitter.com/MarsWxReport?lang=en)
4. [Mars Facts](https://space-facts.com/mars/)
5. [USGS Astrogeology Center](https://astrogeology.usgs.gov)
* `splinter` is used to visit the sites.
* Beautiful Soup, `bs4`, is used to parse and scrape the sites.
* The results of the scraping are saved in a MongoDB.
* Finally, the results are rendered in an HTML template.
`app.py` provides the Flask app and `scrape_mars.py` contains the functions used for scraping.
# Results
* [Jupyter notebook with results from 10-21-2019 scrape](https://nbviewer.jupyter.org/github/douglasdrake/mars/blob/master/scrape_mars.ipynb).