{"id":22609074,"url":"https://github.com/bitfinexcom/bfx-reports-framework","last_synced_at":"2025-04-11T06:19:55.420Z","repository":{"id":37976740,"uuid":"165242329","full_name":"bitfinexcom/bfx-reports-framework","owner":"bitfinexcom","description":null,"archived":false,"fork":false,"pushed_at":"2024-04-15T10:11:56.000Z","size":2328,"stargazers_count":10,"open_issues_count":4,"forks_count":13,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-04-16T10:46:56.765Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bitfinexcom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-11T12:44:15.000Z","updated_at":"2024-04-29T11:00:46.600Z","dependencies_parsed_at":"2023-11-20T05:24:07.974Z","dependency_job_id":"1a0ae3cf-3369-4a83-b0a8-961d3ace4087","html_url":"https://github.com/bitfinexcom/bfx-reports-framework","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fbfx-reports-framework","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fbfx-reports-framework/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fbfx-reports-framework/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitfinexcom%2Fbfx-reports-framework/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bitfinexcom","download_url":"https://codeload.github.com/bitfinexcom/bfx-reports-framework/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248352042,"owners_count":21089374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-08T15:10:48.962Z","updated_at":"2025-04-11T06:19:55.392Z","avatar_url":"https://github.com/bitfinexcom.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bfx Reports Framework\n\n## Description\n\nBfx Reports Framework is a Reactjs and Nodejs open source framework with License Apache 2.0 for Bitfinex and Ethfinex users that can be used to build financial Reports. \u003c/br\u003e\nWhen running the software the result is similar to what is hosted on https://www.bitfinex.com/reports \u003c/br\u003e\nIncluding main structures and libraries gives the possibility for developers to create their personalized reports. \u003c/br\u003e\n\nBfx Reports Framework can be run by its own, without need of adding code, it already includes some exiting features not included on what is hosted on website.\n\n### Included Features\n- All reports and tools from https://www.bitfinex.com/reports\n- Possibility to work without a connection to internet.\n- All ledgers and wallet values expressed in USD or other Forex currency\n\n### Composition\n- https://github.com/bitfinexcom/bfx-report-ui UI components\n- https://github.com/bitfinexcom/bfx-report-express Express server\n- https://github.com/bitfinexcom/bfx-report Base from where back-end service extends from\n\n## Setup\n\n### Install\n\n- Clone Github repository and install projects dependencies :\n\n```console\ngit clone https://github.com/bitfinexcom/bfx-reports-framework.git\ncd bfx-reports-framework\nnpm run init\n```\n- As to run framework on a network different that localhost, add to init the IP or URL as following:\n\n```console\n./init.sh -i XX.XX.XX.XX (being XX.XX.XX.XX the ip)\nor\n./init.sh -u URL\n```\n\n### Configure service\n\nWhen running `node init.sh` configuration is done automatically. \u003c/br\u003e\nAs to check instructions of how to configure each component, visit the git repositories of the components listed above.\n\n## Other Requirements\n\n### Grenache network\n\n- Install `Grenache Grape`: \u003chttps://github.com/bitfinexcom/grenache-grape\u003e:\n\n```console\nnpm i -g grenache-grape\n```\n\n## Run\n\nAs to run the software, is needed to run the network and services separately.\n\n### Run grenache network\n\n- Run two Grapes\n\n```console\ngrape --dp 20001 --aph 30001 --bn '127.0.0.1:20002'\ngrape --dp 20002 --aph 40001 --bn '127.0.0.1:20001'\n```\n\n### Run all framework services\n\n- From main folder, run:\n\n```console\nnpm run start\n```\n\n\n## Testing\n\n### Run tests\n\n```console\nnpm test\n```\n\n## Docker release\n\nA pre-configured [Docker/docker-compose](https://www.docker.com) infrastructure is provided to run the reports framework on an independent Linux server for individual user use. Functionality has been tested on `Ubuntu 20.04 LTS`\n\n### Main Structure\n\nThe following Docker containers are launched:\n\n- `grape1` and `grape2` to run the [grenache-grape](https://github.com/bitfinexcom/grenache-grape) network with two grapes. For it provides `Dockerfile.grenache-grape` file\n- `worker` to run the main grenache `bfx-reports-framework` worker which contains the general business logic. For it provides `Dockerfile.worker` file\n- `express` to run the lightweight server for proxying requests to the grenache `worker`. For it provides `Dockerfile.express` file\n- `ui-builder` to run a building proccess of UI static files. For it provides `Dockerfile.ui-builder` file\n- `nginx` to run the [reverse proxy server](https://www.nginx.com/resources/glossary/reverse-proxy-server)\n\nTo simplify setup/deploy processes the following bash scripts are provided:\n\n- `./scripts/setup.sh` - CLI as an easy way to get through the setup process\n- `./scripts/sync-repo.sh` - CLI to fetch the last changes of the repository/sub-modules from the main remote repo\n- `./scripts/launch.sh` - CLI to launch/re-launch docker-compose services to apply the last fetched changes\n- `./scripts/deploy.sh` - simple deploy script which sync all repo with remote and launch all services\n\n### Requirements\n\nThe setup was tested with the following dependencies:\n\n- Docker version 27.2.0\n- docker-compose version 2.24.6\n- git version 2.24.1\n\n### Setup process\n\nAfter cloning the repository there's needed to configure the app. For it can be used `./scripts/setup.sh` bash script\nAvailable the following arguments:\n\n```console\n./scripts/setup.sh -h\n\nUsage: ./scripts/setup.sh [options] [-d] | [-h]\n\nOptions:\n  -y    With this option, all questions are automatically answered with 'Yes'. In this case, the questions themselves will not be displayed\n  -n    Don't remove files of DBs\n  -h    Display help\n```\n\nDuring the setup process, the user will be asked some questions\n\n- if no Docker/docker-compose are found, the user will be prompted to install them\n- to remove all Log and DB and report files to setup the app from scratch\n- to choose syncing repository branch (master/staging), by default master\n- to set NGINX port, by default 80\n- to set NGINX host, by default localhost\n- to sync all repository/sub-modules (there will be run `./scripts/sync-repo.sh` script)\n\nBased on the responses, a `.env` file will be configured with the following default values:\n\n```console\nNODE_ENV=production\nUI_ENV=production\nNGINX_ENV=production\n\nNGINX_AUTOINDEX=on\n\nREPO_BRANCH=master\n\nNGINX_PORT=80\nNGINX_HOST=localhost\nSECRET_KEY=secretKey\n```\n\n\u003e Pay attention, for security reasons, don't recommend storing secret key value in the `.env` file for production, need to set it into `SECRET_KEY` environment variable!\n\n### Sync repo process\n\nIn case needs to fetch the last changes all repository/sub-modules might be used `./scripts/sync-repo.sh` bash script\nAvailable the following arguments:\n\n```console\n./scripts/setup.sh -h\n\nUsage: ./scripts/sync-repo.sh [options] | [-h]\n\nOptions:\n  -a    Sync all repositories\n  -w    Sync bfx-reports-framework only\n  -u    Sync bfx-report-ui only\n  -e    Sync bfx-report-express only\n  -h    Display help\n```\n\n### Launch process\n\nTo launch/re-launch the docker-compose services of the app available the `./scripts/launch.sh` bash script\nAvailable the following arguments:\n\n```console\n./scripts/launch.sh -h\n\nUsage: ./scripts/launch.sh [options] [-d] | [-h]\n\nOptions:\n  -a    Launch all repositories\n  -g    Launch grenache-grape network only\n  -w    Launch bfx-reports-framework worker only\n  -e    Launch bfx-report-express server only\n  -n    Launch NGINX reverse proxy server only\n  -u    Build bfx-report-ui static files only\n  -d    Detached mode: Run containers in the background\n  -h    Display help\n```\n\n\u003e To run containers of the app in the background, use `-d` argument for the `Detached mode`\n\n### Deploy process\n\nProvides the simple deploy bash script `./scripts/deploy.sh`\nIt provide the following steps:\n\n- add a maintenance flag to show maintenance HTML `./scripts/maintenance/index.html` page via NGINX when the deploy process is going on\n- sync all repository/sub-modules\n- relaunch all docker-compose services except `nginx` service\n- remove the maintenance flag\n\n## Terraform IaaS\n\nThis section describes the implementation of automated infrastructure setting-up in the [AWS](https://aws.amazon.com) cloud provider and the automated deployment process using [Terraform](https://www.terraform.io), it's an open-source infrastructure as code software tool, check [Intro to Terraform](https://www.terraform.io/intro). Terraform community has already written plenty of providers. All publicly available providers can be found on the Terraform Registry, including [Amazon Web Services (AWS)](https://registry.terraform.io/providers/hashicorp/aws/latest/docs). Functionality has been tested on `Ubuntu 20.04 LTS`\n\n### Main Modules Structure\n\nThe infrastructure configuration is located in the `./terraform` directory of the project root. It consists of modules with the following structure:\n\n- `app` - contains the main application configuration, and consists of submodules:\n  - `network` - creates VPC resources, based on [AWS VPC Terraform module](https://github.com/terraform-aws-modules/terraform-aws-vpc)\n  - `ec2` - creates resources to setup Ubuntu instance, attach volume for DB, set deployment process, generate bash script file as an easy way to have an ability to connect via SSH\n  - `ssh_key` - creates SSH private/public keys resources\n  - `kms_key` - creates AWS KMS key resources for encryption purposes of DB volume\n  - `ssm_param_secret_key` - creates AWS SSM parameter to have secure storage of generated on setup step `Private Key` used to encrypt user's `apiKey`/`apiSecret`\n- `backend` - creates resources to setup remote state management with S3 backend for your account. Based on [remote state S3 backend module](https://github.com/nozaq/terraform-aws-remote-state-s3-backend)\n\n### Requirements to use Terraform\n\nThe setup was tested with the following dependencies:\n\n- Terraform version 1.9.5\n\n\u003e To use Terraform you will need to install it. [This official tutorial](https://learn.hashicorp.com/tutorials/terraform/install-cli) will be useful to install Terraform.\n\n### Setting\n\nTo follow this instructions you will need AWS account and [associated credentials](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html) that allow you to create resources. Enter AWS Access keys into the `terraform/config/credentials.conf` file as shown below:\n\n```console\ncp terraform/config/credentials.conf.example terraform/config/credentials.conf\nvim terraform/config/credentials.conf\n```\n\nTo be able to override default values of cofigurable variables of infrastructure use `terraform/terraform.tfvars` file as shown below:\n\n```console\ncp terraform/terraform.tfvars.example terraform/terraform.tfvars\nvim terraform/terraform.tfvars\n```\n\nIn that `terraform/terraform.tfvars` file placed the main important variables useful for customization. More available variables may be seen in the `terraform/variables.tf` file. Be careful, overriding some variables without deep understanding can bring unexpected behavior.\n\n### Bootstrap the project with S3 backend management for tfstate files\n\nTerraform must store state about your managed infrastructure and configuration. This state is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures.\n\nThis state is stored by default in a local file named `terraform/terraform.tfstate`, but it can also be stored remotely, which works better in a team environment. For more info check the following links:\n\n- [Terraform State](https://www.terraform.io/language/state)\n- [Backends](https://www.terraform.io/language/settings/backends)\n\n**Why does this exist?**\n\n\u003e One of the most popular backend options for terraform is AWS (S3 for state, and DynamoDB for the lock table). If your project specifies an AWS/S3 backend, Terraform requires the existence of an S3 bucket in which to store state information about your project, and a DynamoDB table to use for locking (this prevents you, your collaborators, and CI from stepping on each other with terraform commands which either modify your state or the infrastructure itself).\n\nCurrently, this configuration provides a preset for [S3 Backend](https://www.terraform.io/language/settings/backends/s3)\n\nTo manage the project use the fillowing commands:\n\n```console\ncd terraform\n\n# Prepare your working directory for other commands. Whenever a configurations backend changes, you must run terraform init again to validate and configure the backend before you can perform any plans, applies, or state operations\nterraform init\n\n# Create backend infrastructure\nterraform apply -target=module.backend\n\n# Re-configure the backend\nterraform init -reconfigure -backend-config=config/backend.conf\n\n# Create or update infrastructure\nterraform apply\n\n# WARNING: The below commands are useful when you want to destroy previously-created infrastructure\nterraform destroy -target=module.app\nterraform apply -target=module.backend -var=is_backend_s3_config_removed=true\nterraform init -migrate-state\n```\n\n\u003e For the `production` environment strongly recommended to use flow with S3 Backend. But it can be redundant for the `development` or `test` env. Check the below section to get an easier way to bootstrap the project without S3 backend management.\n\n### Bootstrap the project without S3 backend management for tfstate files\n\nTo disable S3 backend management set `is_backend_s3_enabled=false` in the `terraform/terraform.tfvars` file. And right now you can use the default backend called `local` to store state as a local file on disk.\n\nTo manage the project use the fillowing commands:\n\n```console\ncd terraform\n\n# Prepare your working directory for other commands. Whenever a configurations backend changes, you must run terraform init again to validate and configure the backend before you can perform any plans, applies, or state operations\nterraform init\n\n# Create or update infrastructure\nterraform apply\n\n# WARNING: The below commands are useful when you want to destroy previously-created infrastructure\nterraform destroy\n```\n\n### Deployment\n\nTo setup the deployment process need to execute terraform apply of the plan. Each time the command is executed, it will execute [remote-exec Provisioner](https://www.terraform.io/language/resources/provisioners/remote-exec). That Provisioner will connect to the AWS EC2 instance via SSH and launch the `scripts/deploy.sh` bash script described above.\n\nTo deploy the project just use the fillowing command:\n\n```console\ncd terraform\n\nterraform apply\n```\n\n### Notes\n\n- After applying Terraform infrastructure, some useful outputs will be shown in the terminal. One of those being the `Public DNS`, the address as to access the react app. Check the available outputs in the `terraform/outputs.tf` file.\n\nExample of Outputs:\n\n```console\nApply complete! Resources: 1 added, 0 changed, 1 destroyed.\n\nOutputs:\n\nbfx_reports_framework_pub_dns = \"ec2-1-234-56-78.eu-central-1.compute.amazonaws.com\"\nbfx_reports_framework_pub_ip = \"1.234.56.78\"\n```\n\n- A SSH key and a bash script file will be created in Terraform folder as to connect via SSH to AWS EC2. The path to those files will be:\n\n- `terraform/bfx-ssh-key.pem`\n- `terraform/worker-connect.sh`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitfinexcom%2Fbfx-reports-framework","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbitfinexcom%2Fbfx-reports-framework","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitfinexcom%2Fbfx-reports-framework/lists"}