{"id":21824743,"url":"https://github.com/shopify/seer-prototype","last_synced_at":"2025-10-04T18:50:50.963Z","repository":{"id":66009115,"uuid":"481989421","full_name":"Shopify/seer-prototype","owner":"Shopify","description":"Security Expert Elicitation of Risks","archived":false,"fork":false,"pushed_at":"2024-06-03T17:33:27.000Z","size":179,"stargazers_count":13,"open_issues_count":2,"forks_count":3,"subscribers_count":230,"default_branch":"main","last_synced_at":"2025-09-16T13:21:32.219Z","etag":null,"topics":["elicitation","expert-elicitation","security"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Shopify.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-04-15T14:45:19.000Z","updated_at":"2025-05-28T17:30:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"a215e5de-6f6c-4304-a336-952026c0ad98","html_url":"https://github.com/Shopify/seer-prototype","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Shopify/seer-prototype","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fseer-prototype","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fseer-prototype/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fseer-prototype/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fseer-prototype/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Shopify","download_url":"https://codeload.github.com/Shopify/seer-prototype/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fseer-prototype/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278358477,"owners_count":25973949,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elicitation","expert-elicitation","security"],"created_at":"2024-11-27T17:59:48.250Z","updated_at":"2025-10-04T18:50:50.930Z","avatar_url":"https://github.com/Shopify.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SEER: Security Expert Elicitation of Risks\n\n**This tool is a PROTOTYPE ONLY. This is NOT an official or supported Shopify project.**\n\nThe SEER prototype is a tool for security experts to provide estimates of risk\nfor open source software. It is intended to demonstrate how a production-ready\nservice might work, but is not production-ready in itself.\n\nThe core of SEER inspired by [FAIR](https://www.fairinstitute.org/), which is\nbased on a Monte Carlo method. The main departure from FAIR (and similar\napproaches like [Hubbard \u0026 Seiersen's](https://www.google.com/books/edition/How_to_Measure_Anything_in_Cybersecurity/AwD0BgAAQBAJ?hl=en)) is that SEER\ncan integrate _multiple_ estimates per subject of estimation.\n\nNote well: there are no tests. These figures may be nonsense.\n\nThe version you are looking at continues to evolve, meaning that it will have\ndiverged from the version demoed in my [_How Do We Rank Project Risk?_](https://ossna2022.sched.com/event/e992f6daac4b7c79223c101ab9ec7de1) talk at\nOS Summit North America 2022.\nFor convenience you may find [that version at this tag](https://github.com/Shopify/seer-prototype/tree/ranking-project-risk-presentation).\n\n## To run the software\n\n### Local development\n\nFirst set up the assets and database:\n\n```shell\n$ bin/rails assets:precompile db:migrate db:seed\n```\n\nSeeding takes several minutes.\n\nSecond, run the rails process:\n\n```shell\n$ bin/rails server\n```\n\nThere will now be a server listening at [localhost:3000](http://localhost:3000/).\n\n### Containerised\n\nIf you prefer to use Docker or podman, you can build a container image.\n\nFor podman:\n```shell\n$ podman build . --tag seer\n```\n\nFor Docker:\n```shell\n$ docker build . --tag seer\n```\n\nAfter building the image, you can run it.\n\nFor podman:\n```shell\n$ podman run -p 3000:3000 seer\n```\n\nFor Docker:\n```shell\n$ docker run -p 3000:3000 seer\n```\n\nThere will now be a server listening at [localhost:3000](http://localhost:3000/).\n\n## How the core estimation calculation works\n\nThe user provides two three-point estimates (min, mode/likely, max), one\nfor frequency and one for magnitude. To produce a risk value, a sample is\ntaken from each and multiplied to calculate a risk amount. This is done a\nmillion times to create an approximation of the combined distribution of\nfrequency and magnitude.\n\nThe Monte Carlo process is followed when an `Estimate` model is saved. The\n`#create_scenario_bins` method is set to be called `after_save`.\n\n`#create_scenario_bins` first clears existing estimate results from the DB.\nIt then creates `ThreePointEstimate` objects for each of the frequency and\nmagnitude estimate values given by the expert. These are passed into a\n`Scenarios` object.\n\nThe `ThreePointEstimate` and `Scenarios` live in `app/helpers/estimates_helper.rb`.\nThe `ThreePointEstimate` class represents the three basic data points for the\nestimate. Based on these it provides a `#sample` method, which takes a randomly\nchosen value from a triangular distribution configured with the minimum, modal\nand maximum values of the estimate. The triangular distribution sample is provided\nby the [simple-random](https://github.com/ealdent/simple-random) gem. It's not\nclear whether this is a CSRPNG, but it doesn't matter for our purposes.\n\nAlongside `ThreePointEstimate` is `Scenarios`. The `Scenarios#sample` method performs\nthe process of repeatedly calling `ThreePointEstimate#sample` multiple times, as\nset by the `number_of_samples` argument. `ThreePointEstimate#sample` is called\non the frequency estimate and on the magnitude estimate, and the results are\nmultiplied to get the risk for that scenario. Then the value is stored in an\narray of risks.\n\nOnce this sampling loop is complete, `Scenarios#sample` converts the results\ninto a histogram with 100 bins. This is then munged into a convenient format,\nwhich is an array of hashes of the form:\n`{value: \u003cboundary of bucket\u003e, count: \u003cnumber of results in bucket\u003e}`.\n\nOnce control returns from `Scenarios#sample` to `Estimate#create_scenario_bins`,\nthe method creates 100 `ScenarioBin` model records to represent the 100 histogram\nbins that were generated during the sampling process.\n\n## Copyright License\n\nThis repository is licensed under the Apache v2 License:\n\n```\nCopyright 2022-Present Shopify, Inc.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshopify%2Fseer-prototype","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshopify%2Fseer-prototype","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshopify%2Fseer-prototype/lists"}