{"id":41623751,"url":"https://github.com/climateimpactlab/coredot","last_synced_at":"2026-01-24T14:03:42.813Z","repository":{"id":180214783,"uuid":"664780002","full_name":"ClimateImpactLab/coredot","owner":"ClimateImpactLab","description":"Smart dot-product optimizer","archived":false,"fork":false,"pushed_at":"2023-07-28T20:10:37.000Z","size":5,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-09-10T03:51:53.958Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ClimateImpactLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-07-10T18:22:20.000Z","updated_at":"2023-07-28T20:10:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"9454d94f-ff56-4350-a5db-e39ed26a4675","html_url":"https://github.com/ClimateImpactLab/coredot","commit_stats":null,"previous_names":["climateimpactlab/coredot"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ClimateImpactLab/coredot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClimateImpactLab%2Fcoredot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClimateImpactLab%2Fcoredot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClimateImpactLab%2Fcoredot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClimateImpactLab%2Fcoredot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ClimateImpactLab","download_url":"https://codeload.github.com/ClimateImpactLab/coredot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClimateImpactLab%2Fcoredot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28729411,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-24T10:24:43.181Z","status":"ssl_error","status_checked_at":"2026-01-24T10:24:36.112Z","response_time":89,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-24T14:02:24.009Z","updated_at":"2026-01-24T14:03:42.804Z","avatar_url":"https://github.com/ClimateImpactLab.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# coredot\nSmart dot-product optimizer\n\nThis package provides two central pieces of logic for econometric\nprojections:\n\n - Multiple optimized dot-product calculators, which are\n   differentiated by CPU/GPU architecture, calculation size, memory\n   availability, and known duplication.\n   \n - A parallization framework, which selects the best optimized\n   dot-product calculator given initialization-time features like\n   those above and provides a general framework for handling\n   calculations based on what can and cannot be parallized.\n   \n## Context\n\nThe \"inner loop\" of our econometric projections that can be most\nthoroughly optimized is the dot product operation across all regions\nand possibly other dimensions. In some cases, this can be parallelized\nwith the GPU, although that decision can be made at run-time based on\nthe properties of the data.\n\nThe parallel dot-product operation will be compartmentalized to\nmaximize the opportunities for parallel, vectorized, and compiled\noptimizations.\n\nAt initialization, the features of the dot product operation will be\nset, including:\n\n- Dimension and sizes (e.g., number of weather variables; see parallelization above)\n- Known duplication (e.g., weather or coefficients used across multiple regions)\n- Input and output memory contexts (e.g., CPU or GPU)\n\nThis allows the core to develop an execution plan. We will estimate\ndask, numba, and GPU approaches to understand when each is\nappropriate.\n\nPointers to the relevant weather will generally be provided for all\nregions at once, and possibly left in as multi-file xarrays or copied\nto the GPU depending on the execution plan.\n\nThe memory context is important because memory transfer into and out\nof the GPU makes simple dot products inefficient on GPUs, relative to\nCPUs\n(https://blog.theincredibleholk.org/blog/2012/12/10/optimizing-dot-product/). However,\nif the data can be used for multiple dot products, we can get\nbenefits. This would occur if we do grid-level calculations, where\ncoefficients are duplicated and where regional aggregation is also\nneeded and can be done on the GPU. We can also find opportunities with\nlarger arrays (https://dournac.org/info/gpu_sum_reduction).\n\n## Dot-product selection\n\nWe will handle the following cases:\n - A single set of coefficients is applied to many observations (e.g.,\n   Labor).\n - Each observation has a different set of coefficients (e.g.,\n   Mortality).\n - Some coefficients are shared while others are observation-specific\n   (e.g., Agriculture). More generally, a final calculation is a sum\n   of sets of coefficients from the first two cases.\n   \nThe third case makes things interesting, because we need to know if\nthe grouping follows a common pattern across sets of coefficients. For\nexample, if one group of coefficients varies by Region and another by\nTime, then each Region-Time observation will end up with a different\nfinal set of coefficients. Or there could be two different groups of\ncoefficients, each of which varies by Region, but which do so\ndifferently.\n\nThe information will be provided as two dictionaries:\n - `{group: #predictors}`: Specifies the groups of constant\n   predictors. E.g., one could have {'USA': 3, 'IND': 3, ...} if the\n   coefficients vary by region only, or {'USA-2020': 3, ...} if they\n   vary by region-year. A special '*' group means that predictors vary\n   by each observation.\n - `{(group1, group2, ...): #observations}`: Specifies the number of\n   observations that get each sum of groups as their dot-product\n   calculation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclimateimpactlab%2Fcoredot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclimateimpactlab%2Fcoredot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclimateimpactlab%2Fcoredot/lists"}