{"id":13649242,"url":"https://github.com/tuna/opentuna","last_synced_at":"2025-04-22T14:31:05.542Z","repository":{"id":39608356,"uuid":"277693443","full_name":"tuna/opentuna","owner":"tuna","description":"TUNA on AWS","archived":true,"fork":false,"pushed_at":"2023-12-11T01:42:22.000Z","size":1432,"stargazers_count":143,"open_issues_count":1,"forks_count":6,"subscribers_count":17,"default_branch":"master","last_synced_at":"2024-11-10T00:32:54.314Z","etag":null,"topics":["aws","aws-cdk","tuna"],"latest_commit_sha":null,"homepage":"https://opentuna.cn","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tuna.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-07T02:09:57.000Z","updated_at":"2024-11-09T19:22:46.000Z","dependencies_parsed_at":"2023-11-06T02:28:51.881Z","dependency_job_id":"ed22c51a-c117-4c45-bf9b-404bf6ff5d74","html_url":"https://github.com/tuna/opentuna","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tuna%2Fopentuna","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tuna%2Fopentuna/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tuna%2Fopentuna/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tuna%2Fopentuna/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tuna","download_url":"https://codeload.github.com/tuna/opentuna/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250258875,"owners_count":21400986,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-cdk","tuna"],"created_at":"2024-08-02T01:04:52.531Z","updated_at":"2025-04-22T14:31:00.533Z","avatar_url":"https://github.com/tuna.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"# NOTICE\n\n**OpenTUNA will be sunsetted from 20th Dec 2023**. This repository will then be archived and no longer maintained.\n\nIf you are interested in future maintenance work, feel free to reach us at \u003chttps://github.com/tuna/issues/issue\u003e.\n\n# Open TUNA project!\n\nThis is the infrastructure project of Open TUNA on AWS, orchestrated by [AWS CDK][aws-cdk].\n\nSee below for the overall architecture,\n![overall-arch](arch-digram.png)\n\nIt consists of the following independent [stacks][cfn-stack],\n\n- Network stack(optional)\n  - Create a dedicated VPC with public and private subnets across three AZs with NAT gateways\n  - Create S3 Endpoint\n- Storage stack(optional)\n  - EFS file system\n- Common stack\n  - SNS notification topic\n  - Slack webhook subscription(optional)\n- Open TUNA stack\n  - S3 asset bucket\n  - application load balancer\n  - CloudFront distribution\n  - log analysis for CloudFront distribution\n  - ECS cluster for content server and web portal\n  - issue SSL certificate from ACM(only for using Route53 as DNS resolver)\n  - create DNS record in R53 for ALB and CloudFront(only for using Route53 as DNS resolver)\n  - invalidate CloudFront cache when content updates\n  - S3 buckets for S3-backed repos e.g. rubygems\n  - Tunasync Manager stack\n    - auto scaling group for [tunasync][tunasync] manager\n    - intranet application load balancer for manager's API\n    - elasticache redis cluster for database storage\n  - Tunasync Worker stack\n    - auto scaling group for [tunasync][tunasync] worker\n    - install necessary third party tools for mirroring tasks\n    - use systemctl as daemon to start tunasync worker\n    - send custom CloudWatch metrics of tunasync process info\n    - publish large configuration files to s3 asset bucket\n  - Content Server stack\n    - build custom nginx container\n    - use Fargate service to serve mirror contents\n    - internet facing application load balancer\n    - collect network metrics by CloudWatch agent\n    - setup auto scaling rules based on metrics\n  - Web Portal stack\n    - use tuna/mirror-web\n    - route tunasync.json to tunasync manager\n    - create and run lambda function periodically to generate iso download links\n  - Monitor stack\n    - create several CodeBuild projects to verify integrity of index files\n    - run projects periodically and report upon failure\n    - trigger re-sync when failure is detected\n  - Certificate stack\n    - create CodeBuild project to manual renew Let's Encrypt certificate\n- [Pipeline stack](pipeline.md)\n\n## Prerequisites\n\n- VPC with both public and private subnets crossing two AZs at least and NAT gateway. You can [deploy the network stack](#deploy-network-stackoptional) if you don't have a VPC which satisfies the requirements.\n- EFS file system associated with above VPC. You can deploy stack with provisioning a EFS file system without specifying the existing filesystem id of EFS.\n\n## How to deploy it\n\n### Prerequisites\n\n- An AWS account\n- Configure [credential of aws cli][configure-aws-cli]\n- Install node.js LTS version, such as 12.x\n- Install npm 7.x\n- Install Docker Engine\n\n### Checkout submodules\n\n```shell\ngit submodule init\ngit submodule update\n```\n\n### Install project dependencies\n\n```shell\nnpm run init\n```\n\n### Deploy network stack(optional)\n\n```shell\nnpm run deploy-network\n```\n\n### Deploy storage stack(optional)\n\n```shell\nnpx cdk deploy OpenTunaStorageStack -c vpcId=\u003cexisting vpc Id\u003e\n```\n\n### Deploy open tuna stack\n\n```shell\n# deploy storage and common stack as well\nnpx cdk deploy OpenTunaStack -c vpcId=\u003cexisting vpc Id\u003e\n\n# subscribe SNS topic alaram to slack channel\n# Go to https://slack.com/apps/A0F7XDUAZ-incoming-webhooks to apply for webhook\nnpx cdk deploy OpenTunaStack -c vpcId=\u003cexisting vpc Id\u003e -c slackHookUrl=\u003cwebhook url\u003e\n\n# or deploy with existing EFS filesystem\nnpx cdk deploy OpenTunaStack -c vpcId=\u003cexisting vpc Id\u003e -c fileSystemId=\u003cexisting filesystem id\u003e -c fileSystemSGId=\u003cexisting sg id of the given file system\u003e\n\n# deploy with domain name and use Route53 as DNS resolver\nnpx cdk deploy OpenTunaStack -c vpcId=\u003cexisting vpc Id\u003e -c domainName=\u003cdomain name of site\u003e -c domainZone=\u003cpublic hosted zone of your domain in Route53\u003e\n\n# deploy with SSL cert of CloudFront for China regions\n# upload SSL cert to IAM, for China region only\naws iam upload-server-certificate --server-certificate-name my-domain --certificate-body file://cert.pem --private-key file://privkey.pem --certificate-chain file://chain.pem --path '/cloudfront/'\n# get cert id from above output\nnpx cdk deploy OpenTunaStack -c vpcId=\u003cexisting vpc Id\u003e -c domainName=\u003cdomain name of site\u003e -c domainZone=\u003cpublic hosted zone of your domain in Route53\u003e -c iamCertId=\u003ccert id\u003e\n```\n\nThe `CertificateStack` will be created when using IAM cert for the CloudFront SSL. The stack will create a CodeBuild project to issue new SSL certificates from Let's encrypt. And the build event can be pushed to the pipeline account when deploying the stack via the below context option,\n\n```\n-c certTopicArn=\u003carn of sns topic created by pipeline stack\u003e\n```\n\nDocker image for content server is automatically built and published. You can build and publish to ecr manually:\n\n```bash\n$ sudo docker build -t content-server:1.18-alpine .\n$ sudo docker tag content-server:1.18-alpine ${uid}.dkr.ecr.${region}.amazonaws.com/content-server:1.18-alpine\n$ sudo docker push ${uid}.dkr.ecr.${region}.amazonaws.com/content-server:1.18-alpine\n```\n\n## How to test\n\n```shell\nnpm run test\n```\n\n## Post deployment\n- Add email addresses or other subscribers to the notification topic created in the common stack. The alarm notifications related to Open TUNA will be sent to those subscribers.\n\n[aws-cdk]: https://aws.amazon.com/cdk/\n[cfn-stack]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacks.html\n[configure-aws-cli]: https://docs.aws.amazon.com/zh_cn/cli/latest/userguide/cli-chap-configure.html\n[tunasync]: https://github.com/tuna/tunasync\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftuna%2Fopentuna","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftuna%2Fopentuna","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftuna%2Fopentuna/lists"}