https://github.com/kevinknights29/web-scrapper--real-estate-data

This project scrapes data from real estate websites, loads it into a database, and enables this data to be retrieve through a REST API.
https://github.com/kevinknights29/web-scrapper--real-estate-data

data-engineering python webscraping

Last synced: 23 days ago
JSON representation

This project scrapes data from real estate websites, loads it into a database, and enables this data to be retrieve through a REST API.

• Host: GitHub
• URL: https://github.com/kevinknights29/web-scrapper--real-estate-data
• Owner: kevinknights29
• License: mit
• Created: 2022-04-10T21:43:30.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-02-27T04:57:20.000Z (8 months ago)
• Last Synced: 2024-05-08T00:23:59.688Z (6 months ago)
• Topics: data-engineering, python, webscraping
• Language: Python
• Homepage:
• Size: 1.71 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Web-Scrapper---Real-Estate-Data

- This project scrapes data from real estate websites.

- Store results as a .csv and load it into an S3 Bucket.

- From the S3 Bucket, it goes into Amazon Redshift for data warehousing.

- And finally, enables this data to be retrieved through a REST API.

## Scraped Sites

- Encuentra24

- Mitula

![image](https://github.com/kevinknights29/Web-Scrapper--Real-Estate-Data/assets/74464814/911b5e03-8dd0-4b02-83fe-60a0049d3ae8)

## Getting Started

### Docker

- Build container with:

    ```bash

    docker build . -t web-scrapper-real-estate

    ```

- Run the containerized application with:

    ```bash

    docker run web-scrapper-real-estate

    ```

You should start seeing the following output from the processing script.

![image](https://github.com/kevinknights29/Web-Scrapper--Real-Estate-Data/assets/74464814/e742bf91-eef2-40d2-aa89-7533d73c27fc)