{"id":15831667,"url":"https://github.com/cleverdevil/healthlake","last_synced_at":"2025-04-01T17:32:48.047Z","repository":{"id":145921793,"uuid":"342021280","full_name":"cleverdevil/healthlake","owner":"cleverdevil","description":"Web service for ingesting Apple HealthKit data into an S3-backed data lake and making it accessible.","archived":false,"fork":false,"pushed_at":"2022-06-08T16:46:22.000Z","size":15,"stargazers_count":19,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-12T12:31:43.628Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cleverdevil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-24T20:04:51.000Z","updated_at":"2024-08-07T20:58:26.000Z","dependencies_parsed_at":"2023-04-09T19:02:53.454Z","dependency_job_id":null,"html_url":"https://github.com/cleverdevil/healthlake","commit_stats":{"total_commits":8,"total_committers":1,"mean_commits":8.0,"dds":0.0,"last_synced_commit":"054f8a2d9b9182383f0450a629beafd6580b796c"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cleverdevil%2Fhealthlake","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cleverdevil%2Fhealthlake/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cleverdevil%2Fhealthlake/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cleverdevil%2Fhealthlake/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cleverdevil","download_url":"https://codeload.github.com/cleverdevil/healthlake/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222744692,"owners_count":17031256,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-05T12:05:57.966Z","updated_at":"2024-11-02T17:05:29.061Z","avatar_url":"https://github.com/cleverdevil.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Health Lake\n===========\n\nHealth Lake is a work-in-progress web service that makes it possible to ingest\nApple HealthKit data into an Amazon S3 bucket, ready to be used as a queryable\ndata lake. The web service is dependent on an iOS app in the Apple App Store\ncalled [Health Auto Export](https://apps.apple.com/us/app/health-auto-export-json-csv/id1115567069),\nso you'll need to download, install, and configure it to use this service.\n\nIt is already in use by me for [my website](https://cleverdevil.io/health), but\nthere are some bits and pieces that are configured in my AWS environment,\nincluding setting up tables and views in AWS Glue that enable the service to\nquery the data set using Athena. If there is enough interest, I may put together\nsome IaC templates (CloudFormation or Terraform) that can be used to do a more\ncomplete deployment.\n\nFeel free to look at the code, ask questions, and provide feedback!\n\nOn Usage\n--------\n\nAs mentioned above, this is a work-in-progress, and some bits and pieces for the\ncomplete project aren't included, but here is a quick summary of how I\nuse/deploy the web service.\n\nInstall the [serverless framework](https://serverless.com) and then copy the\nsample configuration files to \"real\" ones. Notably, you'll need to have a file\ncalled `conf.py` for configuring the web service and `serverless.yml` which is\nused by the serverless framework to do the full deployment of the service to\nLambda with the appropriate API Gateway configuration and IAM settings to ensure\nthe function has access to S3, Athena, and Glue.\n\nOnce the serverless framework is installed and you've made the appropriate\nchanges to `serverless.yml` and `conf.py`, you need to create a Python virtual\nenvironment, activate it, and install requirements:\n\n```\n$ virtualenv -p python3.6 venv\n$ . venv/bin/activate\n$ pip install -r requirements.txt\n```\n\nAt this point, you need to install a few serverless framework plugins:\n\n```\n$ sls plugin install -n serverless-wsgi\n$ sls plugin install -n serverless-python-requirements\n```\n\nNow, you can test locally with `sls wsgi serve` or deploy to AWS with `sls\ndeploy`.\n\nWeb Service API \n---------------\n\nThere are only three routes for the API:\n\n### `HTTP POST /sync`\n\nThis is the endpoint that you'll configure in the iOS app. I recommend\nconfiguring the app to post once a day with daily granularity. I am not sure if\nthe service will work with other configurations.\n\nOnce the data is posted to this endpoint, the service will flatten the JSON data\nstructure and transform it to make it easy to leverage in AWS Glue and Athena.\n\n### `HTTP GET /detail/\u003cYYYY-MM-DD\u003e`\n\nThis endpoint will return all metrics, in JSON format, that are available on the\nspecified day. There is some intelligence in the service for generating rollups\nautomatically, on-demand, to reduce cost and improve performance. If provided\nwith a date where data is not yet available, or is in the future, the service\nwill return a 404 with a descriptive status message.\n\n### `HTTP GET /summary/\u003cYYYY-MM\u003e`\n\nThis endpoint will return all metrics, in JSON format, that are available on the\nspecified month. There is some intelligence in the service for generating rollups\nautomatically, on-demand, to reduce cost and improve performance. If provided\nwith a date where data is not yet available, or is in the future, the service\nwill return a 404 with a descriptive status message.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcleverdevil%2Fhealthlake","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcleverdevil%2Fhealthlake","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcleverdevil%2Fhealthlake/lists"}