{"id":17648109,"url":"https://github.com/kiran94/dca-manager","last_synced_at":"2026-02-18T06:32:00.071Z","repository":{"id":43271808,"uuid":"430182602","full_name":"kiran94/dca-manager","owner":"kiran94","description":"Automated Dollar Cost Average Process","archived":false,"fork":false,"pushed_at":"2022-03-10T21:46:21.000Z","size":249,"stargazers_count":0,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-13T15:06:23.522Z","etag":null,"topics":["crypto","dca","glue","hudi","investing","kraken","trading"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kiran94.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-20T18:42:46.000Z","updated_at":"2022-01-23T16:05:07.000Z","dependencies_parsed_at":"2022-09-26T20:30:57.010Z","dependency_job_id":null,"html_url":"https://github.com/kiran94/dca-manager","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/kiran94/dca-manager","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiran94%2Fdca-manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiran94%2Fdca-manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiran94%2Fdca-manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiran94%2Fdca-manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kiran94","download_url":"https://codeload.github.com/kiran94/dca-manager/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiran94%2Fdca-manager/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29570333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T06:19:27.422Z","status":"ssl_error","status_checked_at":"2026-02-18T06:18:44.348Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crypto","dca","glue","hudi","investing","kraken","trading"],"created_at":"2024-10-23T11:16:19.712Z","updated_at":"2026-02-18T06:32:00.055Z","avatar_url":"https://github.com/kiran94.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dca-manager\n\n\u003e Dollar Cost Average Manager\n\n[![main](https://github.com/kiran94/dca-manager/actions/workflows/main.yml/badge.svg)](https://github.com/kiran94/dca-manager/actions/workflows/main.yml) [![Go Reference](https://pkg.go.dev/badge/github.com/kiran94/dca-manager.svg)](https://pkg.go.dev/github.com/kiran94/dca-manager) [![Go Report Card](https://goreportcard.com/badge/github.com/kiran94/dca-manager)](https://goreportcard.com/report/github.com/kiran94/dca-manager)\n\n\u003c!-- toc GFM --\u003e\n\n* [Getting Started](#getting-started)\n    * [Infrastructure](#infrastructure)\n    * [Hudi Integration](#hudi-integration)\n    * [Code](#code)\n    * [Running](#running)\n* [Configuration](#configuration)\n* [Schedules](#schedules)\n* [Architecture](#architecture)\n\n\u003c!-- /toc --\u003e\n\n## Getting Started\n\n### Infrastructure\n\nAssuming you are at the root of the repository.\n\n```sh\n# Export Terraform Variables\nexport TF_VAR_KRAKEN_API_KEY=your_key\nexport TF_VAR_KRAKEN_API_SECRET=your_secret\nexport TF_VAR_lambda_failure_dlq_email='[\"you@email.com\"]'\nexport TF_VAR_lambda_success_email='[\"you@email.com\"]'\n\ncd terraform\n\n# The aws and github providers require authentication\n# https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication\n# https://registry.terraform.io/providers/integrations/github/latest/docs#authentication\n\n# The terraform backend configured will also need to be updated\n# The aws region may also need to be updated\n\nterraform init\n\n# Apply Infrastructure\nterraform plan\nterraform apply\n\n# Add remote repository to local\ngit remote add origin $(terraform output -raw github_repository_ssh_clone_url)\n```\n\n*Note some coordination might be required here. Where the lambda function cannot be created without the zip file existing in the S3 location first. Therefore this may need to be at least uploaded to S3 first before `terraform apply` will complete successfully. Once everything is in place then CI will handle deploying any modifications*\n\n### Hudi Integration\n\nThis solution also loads processed transactions into an [Apache Hudi Table](https://hudi.apache.org/) which can be accessed with [Amazon Athena](https://aws.amazon.com/athena) for further analytics processing.\n\nIn order to do this, the [AWS Glue Connector for Apache Hudi](https://aws.amazon.com/marketplace/server/procurement?productId=e918d411-44a4-4b8d-b995-e101d7ef670b) must be applied to the target AWS account. Unfortunately I am not able to find a way to do this via terraform but you can follow these [instructions](https://aws.amazon.com/blogs/big-data/writing-to-apache-hudi-tables-using-aws-glue-connector/) to get setup. In this solution we use Spark 3 and therefore the default connection name we apply is `hudi-connection3`.\n\nOnce this has been completed, then the name of the setup connection should be set into the terraform variable `glue_connections`.\n\nOnce the Infrastructure has been applied/deployed then it can be run either with the following command or from the Glue UI.\n\n```sh\naws glue start-job-run --job-name $(terraform -chdir=terraform output -json | jq --raw-output '.glue_load_transactions_job.value')\n```\n\nNatually, you will need some transactions executed in order for this to be useful.\n\n### Code\n\nAssuming you are at the root of the repository.\n\n\nSet Required Environment Variables:\n\n```sh\nexport DCA_BUCKET=$(terraform -chdir=terraform output -raw bucket)\nexport DCA_CONFIG=$(terraform -chdir=terraform output -raw config_path)\nexport DCA_PENDING_ORDERS_QUEUE_URL=$(terraform -chdir=terraform output -raw pending_orders_queue_url)\nexport DCA_PENDING_ORDER_S3_PREFIX=$(terraform -chdir=terraform output -raw aws_lambda_pending_order_path)\n```\n\nThe `makefile` documents all the recommended commands:\n\n```sh\nmake install_tools\n\nmake build\nmake test\nmake lint\n```\n\n### Running\n\nOnce the infrastructure is up you can either run the code locally or via lambda. This repository consists of multiple lambdas, you can run them locally like so:\n\n```sh\ngo run cmd/execute_orders/main.go\ngo run cmd/process_orders/main.go\n```\n\nThis will pull data from a combination of sources such as the environment and SSM. Additionally the config uploaded in S3 will be used to determine what to do.\n\nBy default, dca-manager should not execute real transactions on an exchange without the `DCA_ALLOW_REAL` being set to any value.\n\n*Please make sure to inspect the code and make sure everything is in order before attaching a real account and running this in production to avoid unexpected transactions.*\n\n## Configuration\n\nThe configuration drives the orders which are executed regularly. At a given interval of time, the configuration is pulled and the process runs through the list of orders.\n\nFor example if you wanted to setup a regular market order for 5 `ADAGBP` via kraken then the configuration might look like this:\n\n*config.json*\n\n```json5\n{\n  \"orders\": [\n    {\n      \"exchange\": \"kraken\",\n      \"direction\": \"buy\",\n      \"ordertype\": \"market\",\n      \"volume\": \"5\",\n      \"pair\": \"ADAGBP\",\n      \"validate\": true,\n      \"enabled\": true\n    }\n  ]\n}\n```\n\nSee [example_config.json](./pkg/configuration/example_config.json) will by default upload to the designated location in S3 via terraform.\n\n## Schedules\n\nScheduling for execution of new orders can be found in the terraform variable `execute_orders_schedules`. Multiple schedules are supported.\n\nFor example, if we wanted to define a schedule for 6AM UTC on Friday and Wednesday then we could configure this:\n\n```terraform\nvariable \"execute_orders_schedules\" {\n  type = list(object({\n    description         = string\n    schedule_expression = string\n  }))\n\n  default = [\n    {\n      description         = \"At 6:00 UTC on every Friday\"\n      schedule_expression = \"cron(0 6 ? * FRI *)\"\n    },\n    {\n      description         = \"At 6:00 UTC on every Wednesday\"\n      schedule_expression = \"cron(0 6 ? * WED *)\"\n    }\n  ]\n}\n```\n\nSee [AWS Schedule Expressions](https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchevents-expressions.html) for all the supported options.\n\nSee [variables.tf](./terraform/variables.tf)\n\n## Logging\n\nWhen running within Lambda, functions are logging in JSON format to support filtering. Therfore you can filter using queries like this:\n\n```\n{ $.level = \"info\" }\n```\n\nSee more [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html#matching-terms-events)\n\n\n## Architecture\n\n```mermaid\ngraph TD\n\n    %% NODES\n    Kraken[[Kraken Exchange]]\n    CloudWatch(Amazon Cloudwatch)\n    S3[(Amazon S3)]\n    SQS(Amazon SQS)\n    ExecuteOrdersLambda{{Execute Orders Lambda}}\n    ProcessOrdersLambda{{Process Orders Lambda}}\n    Hudi(Glue / Hudi)\n    Athena(Amazon Athena)\n\n    %% EDGES\n    CloudWatch--\u003e|Time Schedule| ExecuteOrdersLambda\n\n    Kraken --\u003e ExecuteOrdersLambda\n    ExecuteOrdersLambda --\u003e|Store Pending Transaction| S3\n    ExecuteOrdersLambda --\u003e|Post Pending Transaction| SQS\n\n    SQS --\u003e ProcessOrdersLambda\n    ProcessOrdersLambda --\u003e|Store Transaction Details| S3\n    ProcessOrdersLambda --\u003e|Trigger Glue Job| Hudi\n    Hudi --\u003e S3\n\n    S3 --\u003e|Query Glue Tables| Athena\n    Athena --\u003eFurther(Potential Further Analystics)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiran94%2Fdca-manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkiran94%2Fdca-manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiran94%2Fdca-manager/lists"}