{"id":50374906,"url":"https://github.com/outbrain-inc/ab-tea-lab","last_synced_at":"2026-05-30T09:02:10.598Z","repository":{"id":358896742,"uuid":"1218752766","full_name":"outbrain-inc/ab-tea-lab","owner":"outbrain-inc","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-19T13:34:15.000Z","size":59,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-19T16:19:53.657Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/outbrain-inc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-23T07:21:49.000Z","updated_at":"2026-05-19T13:38:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/outbrain-inc/ab-tea-lab","commit_stats":null,"previous_names":["outbrain-inc/ab-tea-lab"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/outbrain-inc/ab-tea-lab","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outbrain-inc%2Fab-tea-lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outbrain-inc%2Fab-tea-lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outbrain-inc%2Fab-tea-lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outbrain-inc%2Fab-tea-lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/outbrain-inc","download_url":"https://codeload.github.com/outbrain-inc/ab-tea-lab/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outbrain-inc%2Fab-tea-lab/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33686018,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-30T02:00:06.278Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-30T09:02:09.279Z","updated_at":"2026-05-30T09:02:10.590Z","avatar_url":"https://github.com/outbrain-inc.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# ABTeaLab: Bayesian A/B Testing for Estimating Heterogeneous Treatment Effects in Online Advertising\n\nCode accompanying the paper **\"Estimating Heterogeneous Treatment Effects in Online Advertising Experiments: A Hierarchical Bayesian Approach\"** by Aljoša Vodopija, Anže Alič, Tim Poštuvan, Martin Jakomin, and Blaž Škrlj (Teads), submitted to the AdKDD 2026 workshop at the ACM SIGKDD Conference on Knowledge Discovery and Data Mining.\n\n## Abstract\n\nWe present a production hierarchical Bayesian framework for estimating heterogeneous treatment effects in large-scale online advertising experiments. By jointly modeling campaign-level effects and cross-campaign heterogeneity, it addresses a limitation of inverse-variance weighting, the standard meta-analytic baseline, which becomes unstable under high heterogeneity and sparse events. Across simulations spanning a range of effect sizes and heterogeneity regimes, the approach delivers more reliable inference under high heterogeneity while matching or improving estimation accuracy in most settings. We further demonstrate its practical value in a production demand-side platform that handles over 5 million requests per second, integrating it with a sequential testing procedure and evaluating it across three experiments covering over 35,000 campaigns from more than 3,000 advertisers.\n\n## Model\n\n![Plate diagram of the hierarchical Bayesian model](assets/model_diagram.png)\n\nPlate diagram of the proposed hierarchical Bayesian model. Unshaded circles are latent variables, shaded circles are observed, the diamond is a deterministic node, and bare symbols are fixed constants.\n\n## Overview\n\nThis repository will host the implementation of the hierarchical Bayesian model and the Bayesian sequential testing pipeline (based on highest density intervals and regions of practical equivalence) described in the paper, together with the simulation studies used to compare against inverse-variance weighting baselines.\n\n## Code availability\n\n**The code will be released here only after the paper is accepted and published.**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foutbrain-inc%2Fab-tea-lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foutbrain-inc%2Fab-tea-lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foutbrain-inc%2Fab-tea-lab/lists"}