{"id":22878548,"url":"https://github.com/sandesh300/metrics-extraction-automation-sql","last_synced_at":"2026-05-08T05:36:43.854Z","repository":{"id":265431552,"uuid":"895978554","full_name":"sandesh300/Metrics-Extraction-Automation-SQL","owner":"sandesh300","description":"Metrics Extraction and Automation from SQL","archived":false,"fork":false,"pushed_at":"2024-11-29T14:27:19.000Z","size":16119,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-06T20:39:29.598Z","etag":null,"topics":["celery-redis","fastapi","postgresql","python","supabase"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sandesh300.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-29T09:55:18.000Z","updated_at":"2024-11-29T14:47:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"ad745812-3bce-41cf-aa66-7cf7951bf3c3","html_url":"https://github.com/sandesh300/Metrics-Extraction-Automation-SQL","commit_stats":null,"previous_names":["sandesh300/metrics-extraction-automation-sql"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandesh300%2FMetrics-Extraction-Automation-SQL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandesh300%2FMetrics-Extraction-Automation-SQL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandesh300%2FMetrics-Extraction-Automation-SQL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandesh300%2FMetrics-Extraction-Automation-SQL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sandesh300","download_url":"https://codeload.github.com/sandesh300/Metrics-Extraction-Automation-SQL/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246482174,"owners_count":20784652,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["celery-redis","fastapi","postgresql","python","supabase"],"created_at":"2024-12-13T16:29:44.843Z","updated_at":"2026-05-08T05:36:38.834Z","avatar_url":"https://github.com/sandesh300.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Metrics Extraction and Automation from SQL\n\n## Objective\nUse PostgreSQL to analyze and extract insights from search and clicks data, then automate the process.\n\n## Database Schema\n\n```sql\nCREATE TABLE search_clicks (\n search_id SERIAL PRIMARY KEY,\n search_query VARCHAR(255),\n clicks INT DEFAULT 0,\n impressions INT DEFAULT 0,\n click_through_rate FLOAT,\n search_date DATE DEFAULT CURRENT_DATE\n);\n\nCREATE TABLE search_insights (\n id SERIAL PRIMARY KEY,\n insight_date DATE,\n average_ctr FLOAT,\n top_queries JSONB,\n low_performance_queries JSONB\n);\n```\n\n## Requirements\n\n### Metrics Analysis\nWrite SQL queries to:\n1. **Calculate the average click-through rate (CTR) for each day.**\n\n **Query:**\n ```sql\n SELECT\n search_date,\n AVG(click_through_rate) AS average_ctr\n FROM\n search_clicks\n GROUP BY\n search_date;\n ```\n\n **Sample Output:**\n ```json\n | search_date | average_ctr |\n | ----------- | ------------------ |\n | 2024-11-11 | 0.0800128540305011 |\n | 2024-11-04 | 0.0808993520518358 |\n | 2024-11-22 | 0.0813024774774774 |\n | 2024-11-13 | 0.0786556521739131 |\n | 2024-11-16 | 0.0789735807860262 |\n | 2024-10-31 | 0.0784418859649122 |\n | 2024-11-25 | 0.0781114035087719 |\n | 2024-11-18 | 0.0790831168831169 |\n | 2024-11-02 | 0.0772150214592275 |\n | 2024-11-27 | 0.0777504566210046 |\n | 2024-11-26 | 0.0767496659242761 |\n | 2024-11-01 | 0.0790427601809954 |\n | 2024-11-17 | 0.0774195011337869 |\n | 2024-11-10 | 0.080802277904328 |\n | 2024-11-20 | 0.0778314893617021 |\n | 2024-11-15 | 0.0789424628450107 |\n | 2024-11-24 | 0.0807866090712742 |\n | 2024-11-09 | 0.0776652360515022 |\n | 2024-11-12 | 0.0784299776286353 |\n | 2024-11-23 | 0.0767668903803131 |\n | 2024-11-21 | 0.0784713963963964 |\n | 2024-11-05 | 0.0782538461538461 |\n | 2024-11-08 | 0.0804749464668094 |\n | 2024-11-29 | 0.0780660550458716 |\n | 2024-10-30 | 0.0797660550458716 |\n | 2024-11-19 | 0.0798498929336188 |\n | 2024-11-03 | 0.079837339055794 |\n | 2024-11-14 | 0.0763444685466377 |\n | 2024-11-28 | 0.0769561926605504 |\n | 2024-11-06 | 0.0772013186813186 |\n | 2024-11-07 | 0.0774540540540541 |\n ```\n\n2. **Identify the top 5 search queries with the highest CTR over a given period.**\n\n **Query:**\n ```sql\n SELECT\n search_query,\n click_through_rate\n FROM\n search_clicks\n WHERE\n search_date BETWEEN '2024-10-30' AND '2024-11-11'\n ORDER BY\n click_through_rate DESC\n LIMIT\n 5;\n ```\n\n **Sample Output:**\n ```json\n | search_query | click_through_rate |\n | ----------------- | ------------------ |\n | javascript basics | 0.15 |\n | sql commands | 0.1499 |\n | node.js express | 0.1499 |\n | vue.js components | 0.1499 |\n | javascript basics | 0.1498 |\n ```\n\n3. **Detect queries with high impressions but low clicks (possible optimization candidates).**\n\n **Query:**\n ```sql\n SELECT\n search_query,\n impressions,\n clicks\n FROM\n search_clicks\n WHERE\n impressions \u003e 50 AND clicks \u003c 10;\n ```\n\n **Sample Output:**\n ```json\n | search_query | impressions | clicks |\n | -------------------------- | ----------- | ------ |\n | javascript basics | 149 | 8 |\n | vue.js components | 79 | 7 |\n | golang tutorial | 102 | 8 |\n | git commands | 104 | 6 |\n | jenkins pipeline | 157 | 6 |\n | nginx configuration | 81 | 7 |\n | microservices architecture | 94 | 9 |\n | design patterns | 76 | 6 |\n | django rest framework | 78 | 4 |\n | node.js express | 85 | 8 |\n | git commands | 70 | 2 |\n | kubernetes tutorial | 83 | 4 |\n | jenkins pipeline | 61 | 7 |\n | postman api testing | 82 | 6 |\n | aws cli commands | 60 | 4 |\n | ci/cd pipeline | 76 | 6 |\n | test driven development | 70 | 4 |\n | design patterns | 88 | 6 |\n | clean code principles | 68 | 1 |\n | system design basics | 52 | 3 |\n | database indexing | 73 | 1 |\n | node.js express | 86 | 8 |\n | golang tutorial | 69 | 6 |\n | vs code extensions | 64 | 5 |\n | nginx configuration | 104 | 8 |\n | postman api testing | 153 | 7 |\n | microservices architecture | 77 | 2 |\n | ci/cd pipeline | 77 | 3 |\n | test driven development | 91 | 8 |\n | agile methodology | 99 | 9 |\n | design patterns | 80 | 3 |\n | database indexing | 81 | 3 |\n | node.js express | 70 | 7 |\n | jenkins pipeline | 150 | 5 |\n | terraform basics | 199 | 7 |\n | microservices architecture | 63 | 5 |\n | ci/cd pipeline | 71 | 1 |\n | test driven development | 92 | 3 |\n | agile methodology | 84 | 5 |\n | react hooks tutorial | 57 | 7 |\n | sql commands | 103 | 9 |\n | git commands | 80 | 7 |\n ```\n\n### Pipeline Automation\nDevelop a Python script to execute these SQL queries and store the results in a summary table. Automate the script to run daily using a scheduler like Celery.\n\n## Deliverables\n\n1. Use Supabase free tier to complete the assignments.\n2. Make a dummy dataset for all the above tasks.\n3. Provide SQL queries and the summary table structure.\n4. Develop a Python script for pipeline automation.\n5. Provide sample output of the pipeline with mock data.\n\n## Setup\n\n### Prerequisites\n- Python 3.x\n- PostgreSQL\n- Supabase account\n- Redis (for Celery)\n\n### Installation\n\n1. **Clone the Repository**:\n ```bash\n git clone https://github.com/sandesh300/Metrics-Extraction-Automation-SQL\n cd metrics-extraction-automation-sql\n ```\n\n2. **Install Dependencies**:\n ```bash\n pip install -r requirements.txt\n ```\n\n3. **Set Up Environment Variables**:\n Create a `.env` file in the project root with the following content:\n ```env\n SUPABASE_URL=\u003cyour_supabase_url\u003e\n SUPABASE_KEY=\u003cyour_supabase_key\u003e\n ```\n\n4. **Start Redis Server**:\n Ensure Redis is running on your local machine.\n\n## Usage\n\n### Generate Dummy Data\nRun the `generate_dummy_data.py` script to generate mock data for testing:\n```bash\npython generate_dummy_data.py\n```\n### Python script for Pipeline automation\n```python\nimport os\nimport json\nfrom datetime import datetime, timedelta\nimport random\nimport pandas as pd\nfrom dotenv import load_dotenv\nfrom supabase import create_client\nfrom celery import Celery\nfrom celery.schedules import crontab\n\n# Load environment variables\nload_dotenv()\n\n# Initialize Supabase client\nsupabase = create_client(\n os.getenv(\"SUPABASE_URL\"),\n os.getenv(\"SUPABASE_KEY\")\n)\n\n# Initialize Celery\ncelery_app = Celery('search_analytics',\n broker='redis://localhost:6379/0',\n backend='redis://localhost:6379/0')\n\n# Configure Celery to run the task daily at midnight\ncelery_app.conf.beat_schedule = {\n 'daily-analytics': {\n 'task': 'search_analytics.process_daily_analytics',\n 'schedule': crontab(hour=0, minute=0),\n },\n}\n\ndef generate_mock_data():\n \"\"\"Generate and insert mock search data for testing.\"\"\"\n mock_queries = [\n \"python tutorial\", \"javascript basics\", \"react hooks\",\n \"sql commands\", \"docker basics\", \"aws tutorial\",\n \"machine learning\", \"data science\", \"web development\"\n ]\n \n data = []\n start_date = datetime.now() - timedelta(days=30)\n \n for day in range(31):\n for query in mock_queries:\n impressions = random.randint(50, 1000)\n clicks = random.randint(0, impressions)\n ctr = clicks / impressions if impressions \u003e 0 else 0\n \n data.append({\n \"search_query\": query,\n \"clicks\": clicks,\n \"impressions\": impressions,\n \"click_through_rate\": ctr,\n \"search_date\": (start_date + timedelta(days=day)).strftime(\"%Y-%m-%d\")\n })\n \n # Insert mock data using Supabase\n supabase.table('search_clicks').insert(data).execute()\n\ndef get_daily_ctr():\n \"\"\"Calculate average CTR for each day.\"\"\"\n query = \"\"\"\n SELECT \n search_date,\n AVG(click_through_rate) as average_ctr\n FROM search_clicks\n GROUP BY search_date\n ORDER BY search_date DESC;\n \"\"\"\n response = supabase.rpc('execute_query', {'query': query}).execute()\n return response.data\n\ndef get_top_queries():\n \"\"\"Get top 5 search queries with highest CTR.\"\"\"\n query = \"\"\"\n SELECT \n search_query,\n AVG(click_through_rate) as avg_ctr,\n SUM(impressions) as total_impressions\n FROM search_clicks\n WHERE search_date \u003e= CURRENT_DATE - INTERVAL '7 days'\n GROUP BY search_query\n HAVING SUM(impressions) \u003e= 100\n ORDER BY avg_ctr DESC\n LIMIT 5;\n \"\"\"\n response = supabase.rpc('execute_query', {'query': query}).execute()\n return response.data\n\ndef get_low_performing_queries():\n \"\"\"Identify queries with high impressions but low clicks.\"\"\"\n query = \"\"\"\n SELECT \n search_query,\n SUM(impressions) as total_impressions,\n AVG(click_through_rate) as avg_ctr\n FROM search_clicks\n WHERE search_date \u003e= CURRENT_DATE - INTERVAL '7 days'\n GROUP BY search_query\n HAVING SUM(impressions) \u003e= 500 AND AVG(click_through_rate) \u003c 0.01\n ORDER BY total_impressions DESC;\n \"\"\"\n response = supabase.rpc('execute_query', {'query': query}).execute()\n return response.data\n\n@celery_app.task\ndef process_daily_analytics():\n \"\"\"Process daily analytics and store in summary table.\"\"\"\n try:\n # Get all required metrics\n daily_ctr = get_daily_ctr()\n top_queries = get_top_queries()\n low_performing = get_low_performing_queries()\n \n # Prepare data for insertion\n insight_data = {\n \"insight_date\": datetime.now().strftime(\"%Y-%m-%d\"),\n \"average_ctr\": daily_ctr[0][\"average_ctr\"] if daily_ctr else 0,\n \"top_queries\": json.dumps(top_queries),\n \"low_performance_queries\": json.dumps(low_performing)\n }\n \n # Insert into summary table\n supabase.table('search_insights').insert(insight_data).execute()\n \n return {\"status\": \"success\", \"date\": insight_data[\"insight_date\"]}\n except Exception as e:\n return {\"status\": \"error\", \"message\": str(e)}\n\nif __name__ == \"__main__\":\n # Generate mock data if needed\n generate_mock_data()\n \n # Run analytics pipeline manually\n result = process_daily_analytics()\n print(f\"Pipeline execution result: {result}\")\n```\n\n### Run the Pipeline Manually\nRun the `metrics_script.py` script to execute the analytics pipeline manually:\n```bash\npython metrics_script.py\n```\n\n### Automate the Pipeline with Celery\n1. **Start Celery Worker**:\n ```bash\n celery -A metrics_script worker --loglevel=info\n ```\n\n2. **Start Celery Beat Scheduler**:\n ```bash\n celery -A metrics_script beat --loglevel=info\n ```\n\n## Sample Output\n\nAfter running the pipeline, you can query the `search_insights` table to see the generated insights:\n\n```sql\nSELECT * FROM search_insights;\n```\n\nSample output:\n```json\n| id | insight_date | average_ctr | top_queries | low_performance_queries |\n| -- | ------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| 9 | 2024-11-29 | 0.0781 | [{\"search_query\": \"django rest framework\", \"clicks\": 38, \"impressions\": 256, \"click_through_rate\": 0.1496}, {\"search_query\": \"golang tutorial\", \"clicks\": 39, \"impressions\": 264, \"click_through_rate\": 0.149}, {\"search_query\": \"django rest framework\", \"clicks\": 98, \"impressions\": 669, \"click_through_rate\": 0.1478}, {\"search_query\": \"vue.js components\", \"clicks\": 14, \"impressions\": 97, \"click_through_rate\": 0.147}, {\"search_query\": \"golang tutorial\", \"clicks\": 63, \"impressions\": 431, \"click_through_rate\": 0.1469}] | [{\"search_query\": \"jenkins pipeline\", \"clicks\": 19, \"impressions\": 439, \"click_through_rate\": 0.0441}, {\"search_query\": \"jenkins pipeline\", \"clicks\": 16, \"impressions\": 424, \"click_through_rate\": 0.0398}, {\"search_query\": \"postman api testing\", \"clicks\": 13, \"impressions\": 422, \"click_through_rate\": 0.0323}, {\"search_query\": \"microservices architecture\", \"clicks\": 9, \"impressions\": 409, \"click_through_rate\": 0.0233}, {\"search_query\": \"caching strategies\", \"clicks\": 10, \"impressions\": 405, \"click_through_rate\": 0.0261}, {\"search_query\": \"kubernetes tutorial\", \"clicks\": 19, \"impressions\": 391, \"click_through_rate\": 0.0493}, {\"search_query\": \"aws cli commands\", \"clicks\": 12, \"impressions\": 363, \"click_through_rate\": 0.0345}, {\"search_query\": \"git commands\", \"clicks\": 12, \"impressions\": 357, \"click_through_rate\": 0.0344}, {\"search_query\": \"ci/cd pipeline\", \"clicks\": 8, \"impressions\": 355, \"click_through_rate\": 0.0236}, {\"search_query\": \"docker compose example\", \"clicks\": 16, \"impressions\": 341, \"click_through_rate\": 0.0493}, {\"search_query\": \"agile methodology\", \"clicks\": 15, \"impressions\": 337, \"click_through_rate\": 0.0457}, {\"search_query\": \"nginx configuration\", \"clicks\": 10, \"impressions\": 337, \"click_through_rate\": 0.0302}, {\"search_query\": \"postman api testing\", \"clicks\": 14, \"impressions\": 334, \"click_through_rate\": 0.0428}, {\"search_query\": \"database indexing\", \"clicks\": 11, \"impressions\": 324, \"click_through_rate\": 0.0363}, {\"search_query\": \"test driven development\", \"clicks\": 13, \"impressions\": 323, \"click_through_rate\": 0.0428}, {\"search_query\": \"terraform basics\", \"clicks\": 15, \"impressions\": 306, \"click_through_rate\": 0.0494}, {\"search_query\": \"aws cli commands\", \"clicks\": 9, \"impressions\": 291, \"click_through_rate\": 0.0332}, {\"search_query\": \"caching strategies\", \"clicks\": 11, \"impressions\": 284, \"click_through_rate\": 0.0407}, {\"search_query\": \"design patterns\", \"clicks\": 11, \"impressions\": 259, \"click_through_rate\": 0.0441}, {\"search_query\": \"test driven development\", \"clicks\": 7, \"impressions\": 259, \"click_through_rate\": 0.0289}, {\"search_query\": \"kubernetes tutorial\", \"clicks\": 12, \"impressions\": 257, \"click_through_rate\": 0.0481}, {\"search_query\": \"aws cli commands\", \"clicks\": 10, \"impressions\": 253, \"click_through_rate\": 0.0423}, {\"search_query\": \"design patterns\", \"clicks\": 10, \"impressions\": 246, \"click_through_rate\": 0.0418}, {\"search_query\": \"clean code principles\", \"clicks\": 11, \"impressions\": 234, \"click_through_rate\": 0.0493}, {\"search_query\": \"vs code extensions\", \"clicks\": 9, \"impressions\": 230, \"click_through_rate\": 0.0395}, {\"search_query\": \"ci/cd pipeline\", \"clicks\": 7, \"impressions\": 230, \"click_through_rate\": 0.0329}, {\"search_query\": \"git commands\", \"clicks\": 11, \"impressions\": 228, \"click_through_rate\": 0.0493}, {\"search_query\": \"microservices architecture\", \"clicks\": 8, \"impressions\": 227, \"click_through_rate\": 0.0383}, {\"search_query\": \"database indexing\", \"clicks\": 10, \"impressions\": 227, \"click_through_rate\": 0.0459}, {\"search_query\": \"ci/cd pipeline\", \"clicks\": 10, \"impressions\": 220, \"click_through_rate\": 0.0457}, {\"search_query\": \"system design basics\", \"clicks\": 6, \"impressions\": 213, \"click_through_rate\": 0.0302}, {\"search_query\": \"caching strategies\", \"clicks\": 5, \"impressions\": 213, \"click_through_rate\": 0.0236}, {\"search_query\": \"caching strategies\", \"clicks\": 7, \"impressions\": 205, \"click_through_rate\": 0.0385}, {\"search_query\": \"git commands\", \"clicks\": 8, \"impressions\": 202, \"click_through_rate\": 0.044}, {\"search_query\": \"microservices architecture\", \"clicks\": 6, \"impressions\": 188, \"click_through_rate\": 0.035}, {\"search_query\": \"kubernetes tutorial\", \"clicks\": 6, \"impressions\": 184, \"click_through_rate\": 0.035}, {\"search_query\": \"agile methodology\", \"clicks\": 3, \"impressions\": 169, \"click_through_rate\": 0.0213}, {\"search_query\": \"agile methodology\", \"clicks\": 4, \"impressions\": 169, \"click_through_rate\": 0.0262}, {\"search_query\": \"microservices architecture\", \"clicks\": 7, \"impressions\": 159, \"click_through_rate\": 0.048}, {\"search_query\": \"postman api testing\", \"clicks\": 5, \"impressions\": 155, \"click_through_rate\": 0.0358}, {\"search_query\": \"vs code extensions\", \"clicks\": 6, \"impressions\": 154, \"click_through_rate\": 0.0395}, {\"search_query\": \"jenkins pipeline\", \"clicks\": 6, \"impressions\": 153, \"click_through_rate\": 0.0425}, {\"search_query\": \"vs code extensions\", \"clicks\": 6, \"impressions\": 153, \"click_through_rate\": 0.0442}, {\"search_query\": \"design patterns\", \"clicks\": 3, \"impressions\": 143, \"click_through_rate\": 0.0227}, {\"search_query\": \"caching strategies\", \"clicks\": 5, \"impressions\": 142, \"click_through_rate\": 0.036}, {\"search_query\": \"vs code extensions\", \"clicks\": 6, \"impressions\": 133, \"click_through_rate\": 0.0457}, {\"search_query\": \"ci/cd pipeline\", \"clicks\": 5, \"impressions\": 133, \"click_through_rate\": 0.0424}, {\"search_query\": \"postman api testing\", \"clicks\": 5, \"impressions\": 130, \"click_through_rate\": 0.0412}, {\"search_query\": \"caching strategies\", \"clicks\": 3, \"impressions\": 128, \"click_through_rate\": 0.0265}, {\"search_query\": \"test driven development\", \"clicks\": 2, \"impressions\": 126, \"click_through_rate\": 0.0219}, {\"search_query\": \"postman api testing\", \"clicks\": 5, \"impressions\": 126, \"click_through_rate\": 0.0427}, {\"search_query\": \"microservices architecture\", \"clicks\": 4, \"impressions\": 124, \"click_through_rate\": 0.0326}, {\"search_query\": \"agile methodology\", \"clicks\": 4, \"impressions\": 122, \"click_through_rate\": 0.0391}, {\"search_query\": \"git commands\", \"clicks\": 4, \"impressions\": 117, \"click_through_rate\": 0.0417}, {\"search_query\": \"git commands\", \"clicks\": 3, \"impressions\": 112, \"click_through_rate\": 0.0347}, {\"search_query\": \"system design basics\", \"clicks\": 4, \"impressions\": 110, \"click_through_rate\": 0.0439}, {\"search_query\": \"clean code principles\", \"clicks\": 3, \"impressions\": 106, \"click_through_rate\": 0.0293}, {\"search_query\": \"system design basics\", \"clicks\": 4, \"impressions\": 104, \"click_through_rate\": 0.0419}, {\"search_query\": \"system design basics\", \"clicks\": 4, \"impressions\": 103, \"click_through_rate\": 0.0401}, {\"search_query\": \"microservices architecture\", \"clicks\": 2, \"impressions\": 102, \"click_through_rate\": 0.0253}, {\"search_query\": \"clean code principles\", \"clicks\": 4, \"impressions\": 102, \"click_through_rate\": 0.0404}, {\"search_query\": \"jenkins pipeline\", \"clicks\": 4, \"impressions\": 100, \"click_through_rate\": 0.0462}, {\"search_query\": \"vs code extensions\", \"clicks\": 4, \"impressions\": 100, \"click_through_rate\": 0.0481}] |\n```\n\n## Conclusion\n\nThis project demonstrates how to use PostgreSQL to analyze and extract insights from search and clicks data, and automate the process using Python and Celery. The application meets the specified requirements and deliverables, including the generation of dummy data, SQL queries, pipeline automation, and sample output.\n \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsandesh300%2Fmetrics-extraction-automation-sql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsandesh300%2Fmetrics-extraction-automation-sql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsandesh300%2Fmetrics-extraction-automation-sql/lists"}