{"id":26868902,"url":"https://github.com/cintia0528/data_science-ab_testing","last_synced_at":"2025-03-31T05:35:22.560Z","repository":{"id":196548353,"uuid":"696353192","full_name":"Cintia0528/Data_Science-AB_Testing","owner":"Cintia0528","description":"Conduct a 5-way AB Test on Montana State University Library's website, comparing the original \"Interact\" button with new versions (\"Learn,\" \"Help,\" \"Connect,\" \"Services\") to boost user engagement.","archived":false,"fork":false,"pushed_at":"2023-09-25T16:01:52.000Z","size":49,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-31T05:35:18.493Z","etag":null,"topics":["abtesting","bonferroni","chisquare-test","data","data-science","datacleaning","datavisualization","hypothesis-testing","mde","statistics"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Cintia0528.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-09-25T15:19:33.000Z","updated_at":"2024-02-26T05:36:26.000Z","dependencies_parsed_at":"2023-09-26T03:24:40.201Z","dependency_job_id":null,"html_url":"https://github.com/Cintia0528/Data_Science-AB_Testing","commit_stats":null,"previous_names":["cintia0528/ab-testing","cintia0528/data_science-ab_testing"],"tags_count":null,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cintia0528%2FData_Science-AB_Testing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cintia0528%2FData_Science-AB_Testing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cintia0528%2FData_Science-AB_Testing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cintia0528%2FData_Science-AB_Testing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Cintia0528","download_url":"https://codeload.github.com/Cintia0528/Data_Science-AB_Testing/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246423503,"owners_count":20774796,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["abtesting","bonferroni","chisquare-test","data","data-science","datacleaning","datavisualization","hypothesis-testing","mde","statistics"],"created_at":"2025-03-31T05:35:21.046Z","updated_at":"2025-03-31T05:35:22.545Z","avatar_url":"https://github.com/Cintia0528.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AB-Testing\n\n## Task\nTo evaluate which version of the current \"Interact\" button performs best.\n\n## Overview\nThe Library of Montana State University has a website that students use to find books and articles.\n\nBelow the library picture, there is a search bar and three big items: “Find”, “Request” and “Interact”. All three of them contain access to important information however, the Website Analytics show that the “Interact” button has, ironically, almost no interactions.\n\nThe MSU team  wants to run a 5-way AB Test. They would like to us to compare the performance of the original button \"Interact\", to new versions: \"Learn\", \"Help\", \"Connect\", \"Services\".\n\nThe way to measure how each one of the buttons perform is by click-through-rate or CTR. \n\n## Context\nWhen assessing the effectiveness of the \"Shop Now\" button on an e-commerce website, the focus is on optimizing click-through rates due to visitors' clear intent to make a purchase. \n\nConversely, evaluating the performance of a button like \"Interact\" on a library website requires considering that visitors may not always have a specific interest in what it offers. For instance, if a visitor is seeking a book and uses the \"Find\" function, they may not find the \"Interact\" button useful or relevant.\n\nIn addition to click-through rates, it's crucial to analyze the homepage return rate and drop-off rate on the library's website. This broader evaluation helps gauge whether visitors are finding the information they expect after interacting with a specific button.\n\n## Challenge:\nPlan, structure and execute a 5 way AB Test. \n\n## Approach\nHypothesis Testing:\n\n* Explore the current data and result of UI/UX questionnaires available\n* Calculate length of the AB test based on MDE and number of visitors\n* Set up the 5 tests, length and the 5 versions (done for us)\n* Explore raw data, CTR rates of all 5 versions (explore, clean, note anomalies)\n* Form and record hypothesis (H0 \u0026 H1, metrics of interest, MDE, alpha and beta scores, test to use) \n* Categorical data, 5 versions - Chi-Square test\n* Adjust for cumulated alpha error with Bonferroni correction\n\n## Colab Files\nIn the [\"MSU Library\"](6_MSU_Library_Case_Study.ipynb) file imported the data, each button its own csv. All calculations are included. Drop-off and Homepage return rates were given by the MSU team. \n\n### Additional Notes:\nThe data was highly skewed; (10283 visits on \"Interact\" vs the others versions recieved visits in the 2000-3000 range). \nTherefore, after completing the calculations based on the requested parameters (CTR = visits / clicks). I reran the experiment at the end of the notebook under heading _\"Chi-Square Test - CTR 2\"_  where I defined CTR as elemnt of interest clicks / all clicks on the website. Ideally, the test should have been rerun, as it was *biased* from the beginning. \n\n#### I feel this adjusts for 2 factors:\n* Someone might not actually need the \"Interact\" button - or any other version of it\n* Less skewed than visits.\n\n# Skills \u0026 Tools\n1. Data Collection \u0026 Cleaning\n2. Hypothesis Testing\n3. AB Testing\n4. Chi-Square Testing\n5. Bonferroni Adjustment\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcintia0528%2Fdata_science-ab_testing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcintia0528%2Fdata_science-ab_testing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcintia0528%2Fdata_science-ab_testing/lists"}