https://github.com/aconstandinou/data-warehouse-build
https://github.com/aconstandinou/data-warehouse-build
data data-warehouse postgresql-database python quantitative-finance
Last synced: 3 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/aconstandinou/data-warehouse-build
- Owner: aconstandinou
- Created: 2018-07-11T01:25:19.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2018-07-25T19:43:56.000Z (over 6 years ago)
- Last Synced: 2024-08-14T07:09:20.135Z (6 months ago)
- Topics: data, data-warehouse, postgresql-database, python, quantitative-finance
- Language: Python
- Size: 9.77 KB
- Stars: 26
- Watchers: 3
- Forks: 22
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- jimsghstars - aconstandinou/data-warehouse-build - (Python)
README
# Building an Equity Data Warehouse in PostgreSQL with Python
###### This project is meant to use python and SQL in building out an equity data warehouse that will be used for future quantitative research projects.
### Getting Started
This code repository will build a Postgres database on your local machine.
The code is meant to build out the database, SQL schema, scrape the internet for our list of stocks, load their data into the PostgreSQL database and retrieve data as a test.
### Prerequisites
You need to have PostgreSQL and Python installed.
Here are the python libraries used in this database builder. Version numbers are included as well.
* psycopg2 version 2.7.5 (dt dec pq3 ext)
* requests version 2.18.4
* bs4 version 4.6.0
* os
* datetime
### Steps to build this database locally
1. Download this repository into it's own folder.
2. Edit the second line of the `database_info.txt` file with your personal credentials needed to connect
to a PostgreSQL database. There are 3 required credential details that you need to edit.
1. localhost name (your local host name on your machine)
2. username (the user name associated with connecting to a PostgreSQL database)
3. connection password (the password associated with connecting to a PostgreSQL database)
4. Ignore the 'db_name' as this will be the name of the database and **cannot be changed** for future scripts to work.
3. Run `db_part_one_schema_builder.py`
4. Run `db_part_two_sp500_stock_builder.py`
5. Run `db_part_three_retrieve_prices.py`
6. Run `db_part_four_output_dates.py`
### Development Environment
* Spyder IDE version 3.2.8
* Python 3.6.5
* PostgreSQL 9.5.9
### Authors
Antonio Constandinou