https://github.com/samdewriter/abhypothesistesting-adcampaignperformance
The focus of the project is A/B testing. A/B testing is a user experience research methodolgy. In this project, we used the data provided by Smart Ad to carry out A/B testing to analyse how customers engaged with an advert released by the company.
https://github.com/samdewriter/abhypothesistesting-adcampaignperformance
ab-testing analytics machine-learning mlops-workflow
Last synced: 8 months ago
JSON representation
The focus of the project is A/B testing. A/B testing is a user experience research methodolgy. In this project, we used the data provided by Smart Ad to carry out A/B testing to analyse how customers engaged with an advert released by the company.
- Host: GitHub
- URL: https://github.com/samdewriter/abhypothesistesting-adcampaignperformance
- Owner: SamDewriter
- Created: 2021-07-20T18:40:50.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2022-02-21T14:13:06.000Z (over 4 years ago)
- Last Synced: 2025-10-13T05:02:39.718Z (9 months ago)
- Topics: ab-testing, analytics, machine-learning, mlops-workflow
- Language: Jupyter Notebook
- Homepage:
- Size: 844 KB
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Ad Campaign Performance: A/B Hypothesis Testing
# Introduction
SmartAd is a mobile first advertiser agency that designs intuitive touch-enabled advertising. It provides brands with an automated advertising experience via machine learning and creative excellence. Their company is based on the principle of voluntary participation which is proven to increase brand engagement and memorability 10 x more than static alternatives.
## Project Overview
In this case, the company runs an online ad for a client with the intention of increasing brand awareness and would like to know the performance of the advert and determine whether users are engaging with it or not. To do this easily, SmartAd provides an additional service called Brand Impact Optimiser (BIO), a lightweight questionnaire, served with every campaign to determine the impact of the creative ad they design, on a various upper funnel metrics, including memorability and brand sentiment. The task (and this is what this work explores) is to design a reliable hypothesis testing algorithm for the BIO service and determine whether a recent advertising campaign resulted in a significant lift in brand awareness. The goal of the analysis done here is to test if the ads that SmartAd runs resulted in a significant lift in brand awareness.
## Data Description
The BIO data for this project is a “Yes” and “No” response of online users to the following question
Q: Do you know the brand Lux?
O Yes
O No
This is a test run and the main objective is to validate the hypothesis algorithm you built. SmartAd ran this campaign from 3-10 July 2020. The users that were presented with the questionnaire above were chosen according to the following rule:
Control: users who have been shown a dummy ad
Exposed: users who have been shown a creative (ad) that was designed by SmartAd for the client.
# Objectives
The following were achieved in this project:
- Set up a A/B testing framework
- Set up a repeatable ML framework
- Performed A/B testing with classical, sequential, and machine learning methods
- Extracted valid insights in relation to the business objectives