{"id":23734075,"url":"https://github.com/woojiahao/pds-analysis","last_synced_at":"2025-07-27T04:38:23.964Z","repository":{"id":90942515,"uuid":"144087253","full_name":"woojiahao/pds-analysis","owner":"woojiahao","description":null,"archived":false,"fork":false,"pushed_at":"2018-08-14T12:17:47.000Z","size":386,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-04-12T02:36:28.601Z","etag":null,"topics":["data-science","database","matplotlib","numpy","pandas","python3"],"latest_commit_sha":null,"homepage":"https://populationanalysis.herokuapp.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/woojiahao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-08-09T01:42:56.000Z","updated_at":"2024-02-01T03:31:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"351d1bf3-5914-4c08-a201-678c88f8b4eb","html_url":"https://github.com/woojiahao/pds-analysis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/woojiahao/pds-analysis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woojiahao%2Fpds-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woojiahao%2Fpds-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woojiahao%2Fpds-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woojiahao%2Fpds-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/woojiahao","download_url":"https://codeload.github.com/woojiahao/pds-analysis/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woojiahao%2Fpds-analysis/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267300098,"owners_count":24066107,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-27T02:00:11.917Z","response_time":82,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","database","matplotlib","numpy","pandas","python3"],"created_at":"2024-12-31T05:33:51.260Z","updated_at":"2025-07-27T04:38:23.959Z","avatar_url":"https://github.com/woojiahao.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PopulationAnalysis\nWebsite: [PopulationAnalysis](https://populationanalysis.herokuapp.com/)\n\nAnalysis of the data sets found at [https://data.gov.sg/](https://data.gov.sg/), the key topic of this analysis is the declining population and potential causes of it.\n\n## Installation Guide:\n### Required Modules:\n1. flask\n2. flask-sqlalchemy\n3. flask-migrate\n4. SQLAlchemy\n5. gunicorn \n6. psycopg2\n7. python-dotenv\n8. numpy\n9. matplotlib\n10. pandas\n11. pygal\n\n```bash\npip install flask flask-sqlalchemy flask-migrate sqlalchemy gunicorn numpy matplotlib pandas psycopg2 python-dotenv pygal\n```\n\n**Explanation:**\n* `flask` will be used to create the web page\n* `flask-sqlalchemy` is a wrapper over `SQLAlchemy` to reduce overheard upfront for setting up SQLAlchemy to work with the Flask app\n* `flask-migrate` is a module that assists with database migrations \n* `SQLAlchemy` is a Object Relational Mapper(ORM) that provides a layer of abstraction for the developer to reduce interaction with raw SQL and instead allow the developer to interact with the database through objects in Python\n* `gunicorn` is Heroku's preferred library for running a Flask application on\n* `psycopg2` is a wrapper to provide connectivity to the PostgreSQL database and it is used with `SQLAlchemy` and by extension of that, `flask-sqlalchemy`\n* `python-dotenv` is used to provide support for `.env` files that the local copy of the application will be working on\n* `numpy`, `matplotlib` and `pandas` these are all for the data science part of the application\n* `pygal` is a data visualization library to reduces overhead for making interactive diagrams, instead allowing you to focus on making a nice plot\n\n***\n\n### Pre-requisites:\n1. PostgreSQL installed and setup for this project on your local machine, [guide here](https://github.com/woojiahao/pds-analysis/#setting-up-postgresql)\n2. [Modules needed](https://github.com/woojiahao/pds-analysis#required-modules) \n3. Python3 \n\nYou are able to load the application both on your local machine and on a Heroku server (**adviced**) and the installation guide will go into detail on how to do both:\n\n***\n\n### Local Instance\n1. Clone this repository and navigate into it\n```bash\ngit clone https://github.com/woojiahao/pds-analysis.git\ncd pds-analysis\n```\n2. If you have modified aspects of the postgres database, you will have to configure the `config.py`, and alter the second `or` condition to match the modified connection string, otherwise, if you followed the set-up guide for PostgreSQL earlier, then skip this step\n```bash\ncd pds-analysis\nvim config.py\n\nimport os\nclass Config:\n\tDATABASE_CONNECTION_STRING = os.getenv('DATABASE_URL') or '\u003cyour database connection string\u003e'\n```\n3. Now to load the data into the database from the `.csv` files\n```bash\ncd pds-analysis\nflask shell\n\n\u003e\u003e\u003e db_manager.load_data()\n...\nData loaded\n\n\u003e\u003e\u003e exit()\n```\n4. Verify that the database has the tables for `enrolment`, `gdp`, `mothers_occupation` and `live_births`, by opening pgAdmin and verifying that the database contains these tables and using the following queries to ensure that the data is correctly loaded:\n```sql\nSELECT * FROM enrolment;\nSELECT * FROM gdp;\nSELECT * FROM mothers_occupation;\nSELECT * FROM live_births;\n```\n5. Load the plots into the project\n```bash\nflask shell\n\u003e\u003e\u003e plot_loader.load_plots()\n\u003e\u003e\u003e exit()\n```\n6. Run the application:\n```bash\ncd pds-analysis\nflask run\n```\n6. Go to the localhost link, which should be: [http:localhost:5000/](http:localhost:5000/)\n\n***\n\n### Heroku instance (Adviced!)\nThis guide will be broken down into 2 segments, the first for [setting up Heroku](https://github.com/woojiahao/pds-analysis#setting-up-heroku) and the second to [set up this application](https://github.com/woojiahao/pds-analysis#setting-up-application):\n\n#### Setting up Heroku: \n1. Install Heroku and the Heroku CLI to your local machine, in-depth instructions found [here](https://devcenter.heroku.com/articles/heroku-cli)\n2. Clone this repository and navigate into it\n```bash\ngit clone https://github.com/woojiahao/pds-analysis.git\ncd pds-analysis\n```\n3. Create a heroku application for this repository, you can optionally specify a custom site name:\n```bash\nheroku create \u003coptional custom name\u003e\n```\n4. Verify that the set up is correct by making sure that there is now a `heroku` remote that your repository:\n```bash\ngit remote -v\n\nheroku  https://git.heroku.com/populationanalysis.git (fetch)\nheroku  https://git.heroku.com/populationanalysis.git (push)\norigin  https://github.com/woojiahao/pds-analysis (fetch)\norigin  https://github.com/woojiahao/pds-analysis (push)\n```\n5. Add Heroku-Postgres with:\n```bash\nheroku addons:create heroku-postgresql:hobby-dev \n```\n6. Verify that you are connected to the new PostgreSQL database provided by Heroku using `heroku config` and you should see that now there is an [environment variable](https://devcenter.heroku.com/articles/config-vars) for the database connection string, called `DATABASE_URL`\n\n#### Setting up application:\n1. Follow [these steps](https://github.com/woojiahao/pds-analysis/#setting-up-postgresql) to set up PostgreSQL on your local machine\n2. Load the database data and plots \n```bash\nflask shell\n\n\u003e\u003e\u003e db_manager.load_data()\n\u003e\u003e\u003e plot_loader.load_plots()\n\u003e\u003e\u003e exit()\n```\n3. After you have configured Heroku, push this repository to the `heroku` remote, you will see that Heroku handles all the installation and configuring of the needed modules specified in the [`requirements.txt`](https://github.com/woojiahao/pds-analysis/blob/master/requirements.txt) file\n```bash\ngit add .\ngit commit -m \"Setting up\"\ngit push -u heroku master\n```\n2. Load the data into the Heroku database in this manner:\n```bash\nheroku run flask shell\n\n\u003e\u003e\u003e manager = Manager()\n\u003e\u003e\u003e manager.load_data()\n...\nData loaded\n```\n3. Exit the flask shell:\n```bash\n\u003e\u003e\u003e exit()\n```\n4. Activate the web [dyno](https://www.heroku.com/dynos) to start the website\n```bash\nheroku ps:scale web=1\n\nScaling dynos... done, now running web at 1:Free\n```\n5. Open the webpage\n```bash\nheroku open\n```\n\n***\n\n### Setting up PostgreSQl\n1. Download PostgreSQL for the operating system you are on: [Download](https://www.postgresql.org/download/), make sure you download pgAdmin as well \n2. You are free to select any username, password and port for the installation, but take note that modifying it would require some additional configurations when running the application locally\n3. After PostgreSQL is installed, open pgAdmin and create a new database called `pds` (feel free to name it whatever, but like the previous step, changing this will require additional configuration)\n4. Continue with the [installation guide](https://github.com/woojiahao/pds-analysis#pre-requisites)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwoojiahao%2Fpds-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwoojiahao%2Fpds-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwoojiahao%2Fpds-analysis/lists"}