{"id":20874438,"url":"https://github.com/imagingdatacommons/throttleproxy","last_synced_at":"2025-05-12T15:30:32.626Z","repository":{"id":40236799,"uuid":"236112871","full_name":"ImagingDataCommons/ThrottleProxy","owner":"ImagingDataCommons","description":"(CORE REPO)","archived":false,"fork":false,"pushed_at":"2025-05-09T23:43:44.000Z","size":264,"stargazers_count":2,"open_issues_count":4,"forks_count":0,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-10T00:28:58.694Z","etag":null,"topics":["core-repo"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ImagingDataCommons.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-01-25T01:47:26.000Z","updated_at":"2025-05-09T23:43:48.000Z","dependencies_parsed_at":"2024-03-14T20:31:53.416Z","dependency_job_id":"2b1ac4c9-a907-4cbd-911e-8f7ca83c43b5","html_url":"https://github.com/ImagingDataCommons/ThrottleProxy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImagingDataCommons%2FThrottleProxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImagingDataCommons%2FThrottleProxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImagingDataCommons%2FThrottleProxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImagingDataCommons%2FThrottleProxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ImagingDataCommons","download_url":"https://codeload.github.com/ImagingDataCommons/ThrottleProxy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253765727,"owners_count":21960779,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["core-repo"],"created_at":"2024-11-18T06:32:29.649Z","updated_at":"2025-05-12T15:30:32.618Z","avatar_url":"https://github.com/ImagingDataCommons.png","language":"Python","readme":"# ThrottleProxy\n\nA basic throttling proxy, implemented in Flask. Enforces a configurable daily byte limit per IP address. Also\nenforces a global daily byte limit for the whole server. Prior to hitting the daily limit, the server\nis capable of degraded performance by inserting sleep requests between each chunk of relayed bytes. This\ndegradation occurs in two steps. This functionality is not actively used in IDC production, as sleeping the\nserver has implications on server performance. Thus, currently deployed configurations allows full performance \nup until the limit is reached. Once the daily limit is reached, the proxy returns HTTP status 429.\n\nYou need to deploy a *REDIS* server with Serverless VPC Access for this to work. There is a script \n`shell/prepareBackend.sh` that does this for you. You can customize this script by creating a sibling file\nnamed `shell/prepareBackend-SetEnv.sh` to set the needed env variables. (Note that `.gitignore` is configured\nto not upload files named *-SetEnv.sh.) You need a VPC connector is deploying to AppEngine standard; an\nenv variable can configure this.\n\nOnce completed you can see the set up in the Google console under `Memorystore-\u003eRedis` and \n`VPC Network-\u003eServerless VPC access`.\n\nThe values provided choose a 1 GB capacity in the REDIS Basic tier (no need for high availability; \nif we lost the data due to a failure, we would just end up doubling the quota for the day).\nFor the VPC connector, note that once the connector has spooled up to the max instances, *it never \nscales back down*.\n\nThe server can be deployed from your desktop using the script `shell/deployToCloud.sh`. You need to set\nthe environment variables in that script, or in a file `shell/deployToCloud-SetEnv.sh`.\n\nThe two files that need customization are:\n\n1) `config.txt`\n2) `app.yaml`\n\nYou can either edit those files in place, or set the \"LOAD_FROM_CLOUD=TRUE\" variable in `deployToCloud.sh` to keep the\nconfiguration files up in a cloud bucket in the deployed project. They will be then used instead of the demo files\nliving in the repo. The example `app.yaml` in this repo is good to go, needing only customizations to the \nvpc_access_connector name string, though you may want to tweak the scaling set-up. The example `config.txt` will \nneed extensive customization for your site. \n\nIf you need to do debugging by running locally, there is a `shell/run-proxy.sh` script, though it will not\nbe able to talk to the cache.\n\nNote that when you want to tear down the proxy, you will want to delete the Redis cache and the\nconnector, since they cost money to run. You will also want to disable AppEngine in the project (it\ncannot be deleted, only disabled).\n\nAs it is currently set up, the proxy will serve up content with this path, per the config file. Note how\nthe trailing \"/\" separator needs to be the last character of the USAGE_DECORATION value, and \"current\"\nis hardwired into the path:\n\nhttps://${ALLOWED_HOST}/current/${USAGE_DECORATION}${PATH_TAIL}/studies/....\n\n\n**Force Redeploy**\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimagingdatacommons%2Fthrottleproxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimagingdatacommons%2Fthrottleproxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimagingdatacommons%2Fthrottleproxy/lists"}