{"id":18880116,"url":"https://github.com/datafire/on-prem","last_synced_at":"2025-07-11T08:37:55.369Z","repository":{"id":77016037,"uuid":"129152917","full_name":"DataFire/On-Prem","owner":"DataFire","description":"Sample project for on premise and white label DataFire.io installations","archived":false,"fork":false,"pushed_at":"2021-12-13T08:35:52.000Z","size":208,"stargazers_count":6,"open_issues_count":1,"forks_count":5,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-28T08:20:54.430Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataFire.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-11T20:53:28.000Z","updated_at":"2024-03-12T12:43:08.000Z","dependencies_parsed_at":"2023-03-04T17:45:38.303Z","dependency_job_id":null,"html_url":"https://github.com/DataFire/On-Prem","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataFire%2FOn-Prem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataFire%2FOn-Prem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataFire%2FOn-Prem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataFire%2FOn-Prem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataFire","download_url":"https://codeload.github.com/DataFire/On-Prem/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248946035,"owners_count":21187434,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T06:41:57.610Z","updated_at":"2025-04-14T19:32:13.023Z","avatar_url":"https://github.com/DataFire.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DataFire.io On-Prem and White-Label\nThis is a sample repository for running DataFire.io on your own servers.\nYou'll need to [contact the DataFire team](https://app.datafire.io/contact) for an access key.\n\n## Setup\n```bash\nsudo apt-get update\nsudo apt-get install -y curl git python-pip python-dev build-essential zip gzip\npip install awscli --upgrade --user\n\n# Use the keys provided by the DataFire team to log into docker\nsudo `AWS_ACCESS_KEY_ID=YOUR_KEY AWS_SECRET_ACCESS_KEY=YOUR_SECRET aws ecr get-login --no-include-email --region us-west-2`\n```\n\n## Run the Backend\n\n### Start a MongoDB instance\nUsers and project data will be stored in MongoDB.\nYou'll need to create a directory to store the database (here `~/datafire-mongodb`).\n\n```bash\nmkdir ~/datafire-mongodb\nsudo docker run --name datafire-mongo -v ~/datafire-mongodb:/data/db -d mongo\n```\n\n### Start a Git server\nGit is used to store the code for each project.\nYou'll need to create a directory to store projects (here `~/datafire-git`).\n\n```bash\nmkdir ~/datafire-git\ngit clone --bare https://github.com/DataFire-repos/hello-world ~/datafire-git/_start.git\nsudo docker run -d  \\\n  --name datafire-git \\\n  -v ~/datafire-git:/var/lib/git \\\n  -p 3002:80 \\\n  cirocosta/gitserver-http\n\n# Touch the _start repo to initialize /refs/heads/master\ngit clone http://localhost:3002/_start.git \u0026\u0026 cd _start\necho \" \" \u003e\u003e README.md \u0026\u0026 git commit -a -m \"touch readme\"\ngit push \u0026\u0026 cd .. \u0026\u0026 rm -rf _start\n```\n\n### Start Docker (DinD)\nDocker is used to run projects (both in `dev` and `prod`). Alternatively, you can set\nAWS credentials in `./backend/DataFire-accounts.yml` to use AWS ECS and Lambda.\n\n```bash\nsudo docker run --privileged --name datafire-docker -p 32768-33000:32768-33000 -d docker:17-dind\nsudo docker pull 205639412702.dkr.ecr.us-west-2.amazonaws.com/datafire:latest\nsudo docker save 205639412702.dkr.ecr.us-west-2.amazonaws.com/datafire | gzip \u003e ./datafire-image.tar.gz\nsudo docker cp ./datafire-image.tar.gz datafire-docker:/home/dockremap/datafire-image.tar.gz\nsudo docker exec --privileged -it datafire-docker sh -c 'docker load \u003c /home/dockremap/datafire-image.tar.gz'\n```\n\n### Update Backend Settings\n\nNow we need to tell the DataFire backend where each of these services live.\n\nUse `docker inspect` to find the IP address for each container:\n```bash\nsudo docker inspect datafire-mongo  | grep IPAddress\n```\n\nAdd your MongoDB location to `./backend/DataFire-accounts.yml`. For example:\n\n\n```yaml\nmongodb:\n    url: 'mongodb://172.17.0.2:27017'\n    integration: mongodb\n```\n\n```bash\nsudo docker inspect datafire-docker | grep IPAddress\nsudo docker inspect datafire-git | grep IPAddress\n```\n\nAdd your Docker location to `./backend/settings.js`. You should also set\nyour machine's public or intranet IP address as `api_host` and `web_host`. For example:\n\n```js\nmodle.exports = {\n  web_host: 'http://datafire.acme-co.com',\n  api_host: 'http://datafire.acme-co.com:3001',\n  docker_host: 'http://172.17.0.4:2375',\n  git_host: 'http://172.17.0.5:3002',\n}\n```\n\n### Repository Access\nBy default, project code is only accessible to the project's creator.\nHowever, this means the backend **must be served over SSL**, or `git clone`\nwill not work, and you will not be able to deploy your projects.\n\nIf you don't have SSL set up (i.e. `api_host` doesn't start with `https://`),\nyou need to change `allow_public_repo_access` to `true`\nin `./backend/settings.js`. Credentials will still be kept private - only the project's\ncode will be available publicly.\n\n### Start the Backend\nThe backend will handle all API requests. Here we start it on port `3001`.\n\n```bash\nsudo docker build ./backend -t my-datafire-backend\nsudo docker run -d \\\n  --name datafire-backend \\\n  -p 3001:8080 \\\n  -v ~/datafire-git:/var/lib/git \\\n  my-datafire-backend forever server.js\n```\n\n## Run the Website\n\n### Update website settings\n\nWe need to tell the website where the API is hosted, and where deployments will live (i.e. the DinD IP address).\nWe also need to set `git_host` to be the same as `api_host`.\n\nUnless you're just visiting the website on `localhost`, this should be the public or intranet IP of your server.\n\nEdit `./web/settings.ts`\n```ts\nexport const settings:any = {\n  whitelabel: true,\n  web_host: 'http://localhost',        # The URL where the website will be hosted\n  api_host: 'http://localhost:3001',   # The address of the API server above\n  git_host: 'http://localhost:3001',   # Usually, the same address as the API server (which proxies the git server)\n  deployment_host: 'http://localhost', # The address where DIND can be reached (usually the same address as the API server, with no port)\n}\n```\n\n```bash\nsudo docker build ./web -t my-datafire-web\nsudo docker run --name datafire-web -it -p 3000:8080 -d my-datafire-web npm run serve:prod\n```\n\nVisit `http://localhost:3000` to see the website.\n\n### Customization\n\n### Styles\nYou can set the Bootstrap theme by editing ./bootstrap.scss. You can use a [GUI editor](http://bbrennan.info/strapping/) to generate the SASS file.\n\nBe sure to also set the variable `$brand-secondary` in bootstrap.scss.\n\nAdditional styles can be added to this file as well.\n\n### Assets\nEverything in the `./web/assets` folder will be made available at `http://localhost/assets`\n\n### Settings\nEdit `./web/settings.ts` to change your deployment's configuration\n\n#### Options\n* whitelabel: must be set to true\n* logo: a URL for your company's logo\n* web_host: where the website is hosted, e.g. 'https://app.datafire.io'\n* api_host: where the DataFire.io backend is hosted, e.g. 'https://api.datafire.io'\n* callback_url: URL for OAuth callbacks, e.g. 'https://api.datafire.io/oauth/provider/callback'\n* refresh_url: URL for getting OAuth refresh tokens, e.g. 'https://api.datafire.io/oauth/provider/refresh'\n* integrations: A whitelist of integrations available in the UI\n* client_ids: A list of client IDs for OAuth-enabled integrations\n\n## OAuth Applications\n\nThis is a partial list of OAuth providers that are supported. Add your `client_id` for each app\nto `./web/settings.ts`, and both `client_id` and `client_secret` to `./backend/DataFire-accounts.yml`\n\n* azure\n* authentiq\n* bitbucket\n* box\n* bufferapp\n* ebay\n* flat\n* furkot\n* github\n* google\n* instagram\n* lyft\n* medium\n* motaword\n* netatmo\n* reverb\n* runscope\n* slack\n* spotify\n* squareup\n* stackexchange\n* facebook\n* fitbit\n* producthunt\n* heroku\n* linkedin\n* reddit\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatafire%2Fon-prem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatafire%2Fon-prem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatafire%2Fon-prem/lists"}