{"id":22538278,"url":"https://github.com/cfpb/aurora","last_synced_at":"2025-04-09T20:20:51.887Z","repository":{"id":29471418,"uuid":"33008016","full_name":"cfpb/aurora","owner":"cfpb","description":"An open source enterprise data warehousing and analysis platform.","archived":false,"fork":false,"pushed_at":"2021-11-08T22:12:23.000Z","size":2533,"stargazers_count":21,"open_issues_count":6,"forks_count":33,"subscribers_count":9,"default_branch":"develop","last_synced_at":"2025-03-23T22:12:27.740Z","etag":null,"topics":["ansible","data-science","data-warehousing"],"latest_commit_sha":null,"homepage":"https://github.com/cfpb/aurora","language":"Jinja","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cfpb.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-03-27T20:37:23.000Z","updated_at":"2022-09-23T20:57:44.000Z","dependencies_parsed_at":"2022-09-06T20:40:56.396Z","dependency_job_id":null,"html_url":"https://github.com/cfpb/aurora","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Faurora","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Faurora/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Faurora/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Faurora/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cfpb","download_url":"https://codeload.github.com/cfpb/aurora/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248104103,"owners_count":21048282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","data-science","data-warehousing"],"created_at":"2024-12-07T11:11:18.164Z","updated_at":"2025-04-09T20:20:51.852Z","avatar_url":"https://github.com/cfpb.png","language":"Jinja","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Aurora - An Enterprise Data Platform\n\n**Description**:  This repository is a collection of Ansible scripts and other\nsupporting code required to build a scalable, secure, and powerful data\nprocessing platform.\n\n  - **Technology stack**: Ansible is used for deployment.\n  - **Status**:  Under active development.  Once we've reached \"Alpha\", further\n  changes will be tracked in the [CHANGELOG](CHANGELOG.md).\n\n## Dependencies\n\nThe Aurora data platform was designed to work on a network of RHEL 6.5 servers, and\nhas only been tested in that environment.  Additionally, you must have Ansible\ninstalled to deploy, and Vagrant to run locally.\n\n## Installation\n\nTo install locally, simply run \"vagrant up\" from the /deploy directory.  To deploy\nto a remote environment, a custom inventory file is required along with a custom\ngroup_vars file to go with it.  Once that has been added, simply run\n\"ansible-playbook site.yml -i inventories/{{ your_environment }}\"\n\n  * Note: if there isn't a postgres instance running on your machine, you'll need to pass an environment variable to install it\n  \t- Ex: ```EXTRA_VARS='{pp_install:true}' vagrant up [server_name]```\n\n## Configuration\n\nAs mentioned above, you can configure the deployment using Ansible's inventory\nand group_vars functionality.\n\n## Usage\n\nTBD - Likely will create more substantial documentation defining what each\nserver is for and how it is meant to be used.\n\n## How to test the software\n\n### Running Docker on a Macbook\n\n1. brew cask install docker-toolbox\n1. docker-machine start default\n1. docker-machine create --driver \"virtualbox\" default\n1. eval \"$(docker-machine env default)\"\n1. docker ps (to validate it works)\n\nIf docker starts running out of disk space, connect to the boot2docker VM (or Mac terminal) and run this:\n\ndocker ps -a -q | xargs -n 1 -I {} docker rm {}\n\nCommand to make sure the exited containers are deleted:\n\ndocker rm -v $(docker ps -a -q -f status=exited)\n\n### Setting up Test Environment\n\nWhen developing the Travis CI file, it can be helpful to test in travis's environment as described\nhere: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received\n* Note: you'll need to install the travis image with --privileged\n  * docker run --privileged -it quay.io/travisci/travis-ruby /bin/bash\n\nTo do this, follow the steps above up to actually running your commands.  Before doing so, Docker must\nbe installed in the Travis CI image, like so:\n\n1. sudo apt-get install apt-transport-https ca-certificates\n1. sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D\n1. echo \"deb https://apt.dockerproject.org/repo ubuntu-precise main\" | sudo tee /etc/apt/sources.list.d/docker.list\n1. sudo apt-get update\n1. apt-cache policy docker-engine\n1. sudo apt-get install docker-engine (May neeed --force-yes)\n1. sudo ln -s /bin/true /sbin/initctl\n1. sudo service docker start -\u003e\u003e\n1. docker daemon -H unix:///var/run/docker.sock\u0026\u003e/var/log/docker.log \u0026\n1. git clone https://github.com/[githubfork]/aurora /aurora\n1. cd /aurora\n1. git checkout travis\n1. Run commands in travis.yml file\n\n\nTBD\n\n## Role-specific documentation\n\nSome Ansible roles in this project have role-specific documentation:\n\n* (postgresql-server)[deploy/roles/postgresql-server/README.md]\n* (python27-scl)[deploy/roles/python27-scl/README.md]\n* (python36-scl)[deploy/roles/python36-scl/README.md]\n\n## Known issues\n\n- Travis-CI hangs when jobs complete - [resolution](https://www.jeffgeerling.com/blog/2017/fix-ansible-hanging-when-used-docker-and-tty)\n- R package installation takes too long (currently skipped)\n\n## Getting help\n\nOpen an issue on Github if you need help, have a feature request, or have\ncode to contribute.\n\n## Getting involved\n\nRefer to [CONTRIBUTING](CONTRIBUTING.md) if you'd like to help!\n\n----\n\n## Open source licensing info\n1. [TERMS](TERMS.md)\n2. [LICENSE](LICENSE)\n3. [CFPB Source Code Policy](https://github.com/cfpb/source-code-policy/)\n\n----\n\n## Credits and references\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcfpb%2Faurora","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcfpb%2Faurora","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcfpb%2Faurora/lists"}