{"id":17756046,"url":"https://github.com/fabianacampanari/practicalstats-pucsp-2024","last_synced_at":"2025-05-12T00:41:40.839Z","repository":{"id":258389245,"uuid":"873842308","full_name":"FabianaCampanari/PracticalStats-PUCSP-2024","owner":"FabianaCampanari","description":"Statistical Measures in Python - Age and Salary Analysis","archived":false,"fork":false,"pushed_at":"2024-12-11T02:19:36.000Z","size":64641,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-12T00:41:30.637Z","etag":null,"topics":["excel","hipoteses-tests","hypothesis-testing","linear-regression","math","matplotlib","numpy","pandas","probabilistic-data-structures","probability-distribution","probability-statistics","pythob3","scikit-learn","scipy","scipy-stats","seaborn","statisctics","statsmodels"],"latest_commit_sha":null,"homepage":"https://github.com/FabianaCampanari/statisticalMeasures-python-","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FabianaCampanari.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":"SECURITY.md","support":"support materials/1-Integral students.pdf","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"FabianaCampanari","Custom":"https://github.com/sponsors/FabianaCampanari/card"}},"created_at":"2024-10-16T20:13:06.000Z","updated_at":"2025-04-22T20:30:12.000Z","dependencies_parsed_at":"2025-03-09T20:42:13.946Z","dependency_job_id":null,"html_url":"https://github.com/FabianaCampanari/PracticalStats-PUCSP-2024","commit_stats":null,"previous_names":["fabianacampanari/statisticalmeasures-python-","fabianacampanari/statisticalmeasures"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FabianaCampanari%2FPracticalStats-PUCSP-2024","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FabianaCampanari%2FPracticalStats-PUCSP-2024/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FabianaCampanari%2FPracticalStats-PUCSP-2024/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FabianaCampanari%2FPracticalStats-PUCSP-2024/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FabianaCampanari","download_url":"https://codeload.github.com/FabianaCampanari/PracticalStats-PUCSP-2024/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253655919,"owners_count":21943072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["excel","hipoteses-tests","hypothesis-testing","linear-regression","math","matplotlib","numpy","pandas","probabilistic-data-structures","probability-distribution","probability-statistics","pythob3","scikit-learn","scipy","scipy-stats","seaborn","statisctics","statsmodels"],"created_at":"2024-10-26T15:08:58.115Z","updated_at":"2025-05-12T00:41:40.812Z","avatar_url":"https://github.com/FabianaCampanari.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/FabianaCampanari","https://github.com/sponsors/FabianaCampanari/card"],"categories":[],"sub_categories":[],"readme":"\n**\\[[🇧🇷 Português](README.pt_BR.md)\\] \\[**[🇺🇸 English](README.md)**\\]**\n\n\n\u003cbr\u003e\u003cbr\u003e\n\n![2129d7ad-6afd-4166-9bf1-a4f9c3b9e5cc](https://github.com/user-attachments/assets/faf3e0a3-3610-4c0b-99bb-afb7a765f28d)\n\n\u003cbr\u003e\u003cbr\u003e\n\n## \u003cp align=\"center\"\u003e  Practical Statistics and Probability in Python and Excel\n#### \u003cp align=\"center\"\u003e [University of Data Science and Artificial Intelligence]() - PUC-SP - 2nd Semester/2024\n\n\u003cbr\u003e\n\n#### \u003cp align=\"center\"\u003e [![Sponsor FabianaCampanari ](https://img.shields.io/badge/Sponsor-FabianaCampanari-brightgreen?logo=GitHub)](https://github.com/sponsors/FabianaCampanari) \n\n\u003cbr\u003e\n\n\n## **Statistics and Probability**\n \nThis repository, created by [Fabiana 🚀 Campanari](https://linktr.ee/fabianacampanari) in the 2nd semester of 2024, consolidates the materials and code developed for the Statistics and Probability course within the Data Science and Artificial Intelligence program at PUC-SP, under the guidance of [Professor Eric Bacconi Gonçalves](https://www.linkedin.com/in/eric-bacconi-423137/). It is designed to support hands-on learning through exercises, scripts, and datasets.\n\n## Repository Contents:\n\n\u003cbr\u003e\n\n- **Python Scripts**: This section includes scripts for a wide range of statistical analyses, covering key topics such as distributions, population and sample concepts, and hypothesis testing. Calculations include:\n  - **Central Tendency**: Mean, median, and mode\n  - **Dispersion**: Standard deviation, variance, and range\n  - **Positional Measures**: Percentiles and quartiles (Q1, Q2, Q3)\n  - **Distribution Shape**: Skewness and kurtosis\n  - **Confidence Intervals** for estimating population parameters\n  - **Correlation and Covariance** for bivariate analysis\n \n \u003cbr\u003e   \n\n- **Practical Exercises**: Available in Python and Excel, these exercises provide hands-on practice in calculating statistical measures and applying concepts, including:\n  - **Analysis of Variance (ANOVA)**: Comparing means across multiple groups\n  - **Hypothesis Testing**: Null hypothesis (\\( H_0 \\)) tests, such as T-tests (one-sample, independent, and paired), ANOVA, and Chi-square tests\n  - **Regression Analysis**: Linear regression models for predictive analysis\n  - **Probability**: Exercises covering probability distributions, expected value, and variance\n \n \u003cbr\u003e   \n\n- **Statistical Tests**: This section includes implementations of statistical tests tailored to analyze variables like age and salary, categorized by region and educational level. Each test includes the process of setting up and testing the null hypothesis (\\( H_0 \\)) for statistical significance.\n\n\u003cbr\u003e  \n\n- **Support Materials**: Supplementary documentation on probability, relevant datasets, and homework assignments to reinforce key concepts.\n\n\u003cbr\u003e\n\n \u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/user-attachments/assets/2a7b0742-b1b1-4f4e-ba4b-50dedf6f30bb\" /\u003e\n\n\u003cbr\u003e\n\n## [Study Topics]():\n\nThis repository provides a comprehensive foundation in core topics, including Descriptive Statistics, Probability Distributions, Population and Sample, Hypothesis Testing I and II (featuring null hypothesis (\\( H_0 \\)) testing), and Regression Analysis. It serves as a practical tool for building statistical and analytical skills.\n\n\n## [Statistical Measures Analysis in Python]():\n\nThis repository contains Python scripts for descriptive statistical analysis of employee salary and age data, including analyses for the entire dataset as well as subgrouped by education level and region.\n\n## [Features]():\n\nDescriptive statistics: Mean, Median, Mode, Variance, Standard Deviation, Coefficient of Variation (CV), and Amplitude (Range).\nGrouped analysis: The same statistics calculated by grouping data based on region and education level.\nDesigned for students: Easy-to-follow code with comments and explanations for each step.\n\n\u003cbr\u003e\n\n## [Dataset]():\n\nThe dataset used in this analysis contains employee details, including their age, salary, region of origin (reg_proc), and education level (grau_instrucao).\n[Click here to get the Dataset](https://github.com/FabianaCampanari/statisticalMeasures-python-/tree/a9e92b1cbce36fa5f26edeadef937981012f0a98/Dataset)\n\n\u003cbr\u003e\n\n## [Getting Started]():\n\n### To run this script, ensure you have the following:\n\n- Python 3 installed.\n\n- Necessary libraries (pandas) installed.\n\n- An Excel file containing the dataset in the appropriate format.\n\n\n## [Codes]():\n\n### 1. Statics Measures\n\n`👇 Copy code`\n\n```python\n\nCopy code\n\n# Importing the necessary library\nimport pandas as pd\n\n# Define the file path to the dataset\nfile_path = 'add_your_dataset_path_here'\n\n# Load the dataset into a DataFrame\ndf = pd.read_excel(file_path)\n\n# Display the first few rows of the dataset\ndf.head()\n\n# --- Descriptive Statistics for 'SALARIO' (salary) ---\n\n# Generate descriptive statistics for the 'SALARIO' column\nprint(\"Descriptive statistics for 'SALARIO':\")\nprint(df.salario.describe())\n\n# Calculate the range (amplitude) of the 'SALARIO' column\nampl_salario = df['salario'].max() - df['salario'].min()\nprint(\"\\nAmplitude of 'SALARIO':\", ampl_salario)\n\n# Calculate the mode of the 'SALARIO' column\nmoda_salario = df.salario.mode()[0]\nprint(\"\\nMode of 'SALARIO':\", moda_salario)\n\n# Calculate the variance of the 'SALARIO' column\nvar_salario = df.salario.var()\nprint(\"\\nVariance of 'SALARIO':\", var_salario)\n\n# Calculate the coefficient of variation (CV) for 'SALARIO'\ncv_salario = df.salario.std() / df.salario.mean()\nprint(\"\\nCoefficient of variation (CV) of 'SALARIO':\", cv_salario)\n\n# --- Descriptive Statistics for 'SALARIO' by 'grau_instrucao' (educational level) ---\n\n# Generate descriptive statistics for 'SALARIO' grouped by 'grau_instrucao' (education level)\nprint(\"\\nDescriptive statistics for 'SALARIO' grouped by 'grau_instrucao':\")\nprint(df.groupby('grau_instrucao')['salario'].describe())\n\n# Calculate the range (amplitude) of 'SALARIO' by 'grau_instrucao'\nampl_salario_grau = df.groupby('grau_instrucao')['salario'].max() - df.groupby('grau_instrucao')['salario'].min()\nprint(\"\\nAmplitude of 'SALARIO' by 'grau_instrucao':\")\nprint(ampl_salario_grau)\n\n# Calculate the mode of 'SALARIO' by 'grau_instrucao'\nmoda_salario_grau = df.groupby('grau_instrucao')['salario'].agg(lambda x: pd.Series.mode(x)[0])\nprint(\"\\nMode of 'SALARIO' by 'grau_instrucao':\")\nprint(moda_salario_grau)\n\n# Calculate the variance of 'SALARIO' by 'grau_instrucao'\nvar_salario_grau = df.groupby('grau_instrucao')['salario'].var()\nprint(\"\\nVariance of 'SALARIO' by 'grau_instrucao':\")\nprint(var_salario_grau)\n\n# Calculate the coefficient of variation (CV) for 'SALARIO' by 'grau_instrucao'\ncv_salario_grau = df.groupby('grau_instrucao')['salario'].std() / df.groupby('grau_instrucao')['salario'].mean()\nprint(\"\\nCoefficient of variation (CV) of 'SALARIO' by 'grau_instrucao':\")\nprint(cv_salario_grau)\n\n# Summary of key descriptive statistics\nprint(\"\\nSummary for 'SALARIO' as a Whole:\")\nprint(f\"\\nAmplitude: {ampl_salario}\")\nprint(f\"\\nMode: {moda_salario}\")\nprint(f\"\\nVariance: {var_salario}\")\nprint(f\"\\nCoefficient of variation (CV): {cv_salario}\")\n\nprint(\"\\nSummary by 'grau_instrucao':\")\nprint(f\"\\nAmplitude by 'grau_instrucao': \\n{ampl_salario_grau}\")\nprint(f\"\\nMode by 'grau_instrucao': \\n{moda_salario_grau}\")\nprint(f\"\\nVariance by 'grau_instrucao': \\n{var_salario_grau}\")\nprint(f\"\\nCoefficient of variation (CV) by 'grau_instrucao': \\n{cv_salario_grau}\")\n```\n\n\u003cbr\u003e\n\n### 2. Sample Selection\n\n`👇 Copy code`\n\n```python\n# Import pandas and numpy libraries\nimport pandas as pd\nimport numpy as np\n\n# Define the file path\nfile_path = 'add_your_dataset_path_here'\n\n# Read the Excel file into a DataFrame\ndf = pd.read_excel(file_path)\n\n# Sample Selection\n\n# Simple random sample without replacement with 20 elements\nsample = df.sample(20, replace=False)\nprint(sample)\n\n# Simple random sample without replacement with 20 elements (fixing the random seed)\nsample = df.sample(n=20, replace=False, random_state=2903)\nprint(sample)\n\n# Check the classes of the variable and their proportions\nperc_est_civ = df[\"estado_civil\"].value_counts(normalize=True)\nprint(perc_est_civ)\n\n# Execute equal stratified sample by marital status\nsample_strat_equal = df.groupby(['estado_civil'], group_keys=False).apply(lambda x: x.sample(n=10, replace=False, random_state=2903))\nprint(sample_strat_equal)\n\n# Define desired total\nN = 20\n\n# Execute proportional stratified sample by marital status\nsample_strat_prop = df.groupby(['estado_civil'], group_keys=False).apply(\n    lambda x: x.sample(int(np.rint(N * len(x) / len(df))), random_state=2903)  # Proportional sample calculation\n).sample(frac=1, random_state=2903).reset_index(drop=True)  # Shuffle the sample\nprint(sample_strat_prop)\n\n# Check the proportion of each marital status\nperc_est_civ = sample_strat_prop[\"estado_civil\"].value_counts(normalize=True)\nprint(perc_est_civ)\n\n# Execute equal stratified sample by region of origin\nsample_strat_equal = df.groupby(['reg_proc'], group_keys=False).apply(lambda x: x.sample(n=10, replace=False, random_state=2903))\nprint(sample_strat_equal)\n\n# Execute equal stratified sample by education level\nsample_strat_equal = df.groupby(['grau_instrucao'], group_keys=False).apply(lambda x: x.sample(n=10, replace=False, random_state=2903))\nprint(sample_strat_equal)\n\n# Execute equal stratified sample by region of origin and education level\nsample_strat_equal = df.groupby(['reg_proc', 'grau_instrucao'], group_keys=False).apply(lambda x: x.sample(n=10, replace=False, random_state=2903))\nprint(sample_strat_equal)\n\n# Create an equal stratified sample by education level and region of origin\nsample_strat_equal = df.groupby(['grau_instrucao', 'reg_proc'], group_keys=False).apply(lambda x: x.sample(n=min(len(x), 10), replace=False, random_state=2903)).reset_index(drop=True)\nprint(sample_strat_equal)\n\n# Save the stratified sample to a new Excel file\noutput_path = 'path_to_save_sample/stratified_sample.xlsx'\nsample_strat_equal.to_excel(output_path, index=False)\n\nprint(f\"The stratified sample has been saved to {output_path}\")\n```   \n\n\u003cbr\u003e\n\n### 3. One-Sample t-Test\n\n`👇 Copy code`\n\n```python\n\n# Exercise 3 – Test the hypothesis that the salary is equal to 12. What is your conclusion?\n# Import pandas library\nimport pandas as pd\n\n# Import scipy library\nimport scipy.stats as stats\n\n# File path\nfile_path = 'add_your_dataset_path_here'\n\n# Load the file into Python\ndf = pd.read_excel(file_path)\nprint(df.head())\n\n# Bring only the age variable to perform the test\nbase_age = df['idade']\n\n# Execute the t-test testing H0: age = 32 and H1: age ≠ 32\nresult_t_test = stats.ttest_1samp(base_age, 32)\np_value = result_t_test.pvalue\nalpha = 0.05\n\nif p_value \u003c alpha:\n    print(\"We reject the null hypothesis (H0).\")\nelse:\n    print(\"We do not reject the null hypothesis (H0).\")\n\n# Remember the mean\nprint(f\"Mean age: {df.idade.mean()}\")\n\n# Execute the t-test testing H0: age = 34 and H1: age ≠ 34\nresult_t_test = stats.ttest_1samp(base_age, 34)\np_value = result_t_test.pvalue\nalpha = 0.05\n\n# Decision based on p-value\nif p_value \u003c alpha:\n    print(\"We reject the null hypothesis (H0).\")\nelse:\n    print(\"We do not reject the null hypothesis (H0).\")\n\n# With a p-value of 0.045, which is less than the significance level of 0.05, we reject the null hypothesis. This indicates that there is statistically significant evidence to suggest that the average age of employees is different from 34.\n\n# Execute the t-test testing H0: age = 35 and H1: age ≠ 35\nresult_t_test = stats.ttest_1samp(base_age, 35)\np_value = result_t_test.pvalue\nalpha = 0.05\n\n# Decision based on p-value\nif p_value \u003c alpha:\n    print(\"We reject the null hypothesis (H0).\")\nelse:\n    print(\"We do not reject the null hypothesis (H0).\")\n\n# With a p-value of 0.2234, which is greater than the significance level of 0.05, we do not reject the null hypothesis. This means that there is not enough evidence to conclude that the average age of employees is different from the hypothesized value (32, 35, or any other value being tested).\n\n# Answering question 3\n\n# Test the hypothesis that the salary is equal to 12. What is your conclusion?\nimport pandas as pd\nimport scipy.stats as stats\n\n# File path\nfile_path = 'add_your_dataset_path_here'\n\n# Load the data\ndf = pd.read_excel(file_path)\nprint(df.head())\n\n# Select the variable of interest (salary)\nsalaries = df['salario']\n\n# Execute the t-test testing H0: salary = 12 and H1: salary ≠ 12\nresult_t_test = stats.ttest_1samp(salaries, 12)\nprint(result_t_test)\n\n# Get the p-value from the test result\np_value = result_t_test.pvalue\n\n# Define the significance level\nalpha = 0.05\n\n# Interpret the result\np_value = 8.755117588192511e-06\n\nif p_value \u003c alpha:\n    print(\"We reject the null hypothesis (H0).\")\nelse:\n    print(\"We do not reject the null hypothesis (H0).\")\n\n# Define the conclusion based on the p-value and the significance level\nif p_value \u003c alpha:\n    conclusion = \"We reject the null hypothesis (H0).\"\nelse:\n    conclusion = \"We do not reject the null hypothesis (H0).\"\n\n# Display the test result and conclusion\nprint(f\"t-test statistic: {result_t_test.statistic}\")\nprint(f\"p-value: {result_t_test.pvalue}\")\nprint(conclusion)\n\n# Analysis and Conclusion with p-value and significance level\n\n# In the hypothesis test performed, we tested the null hypothesis (H0) that the average salary of employees is equal to 12 against the alternative hypothesis (H1) that the average salary of employees is different from 12.\n\n# The results of the t-test were as follows:\n# - t-test statistic: -4.500727991746298\n# - p-value: 8.755117588192511e-06\n\n# With a p-value of approximately 8.76e-06, which is significantly less than the significance level of 0.05, we reject the null hypothesis (H0). This indicates that there is statistically significant evidence to suggest that the average salary of employees is different from 12.\n\n# Therefore, the conclusion of the test is that we reject the null hypothesis (H0) and accept the alternative hypothesis (H1), indicating that the average salary of employees is not equal to 12.\n```\n\u003cbr\u003e\n\n### 4. Two-Sample t-Test\n\n`👇 Copy code`\n\n```python\n### Question 4\n\n#### To test the hypothesis that income is equal for the two marital statuses (single and married), we can use the t-test for independent samples. We will follow these steps:\n### Visualization\n\n#### 1. Extract income data for singles and married individuals.\n#### 2. Perform the t-test for independent samples.\n#### 3. Interpret the p-value to accept or reject the null hypothesis.\n\n### Steps in pseudocode:\n\n1. **Import necessary libraries** (pandas and scipy.stats).\n2. **Load data from the Excel file**.\n3. **Extract income columns for singles and married individuals**.\n4. **Perform the t-test for independent samples**.\n5. **Interpret the result based on the p-value**.\n\n# Install scipy if necessary\n# %pip install scipy pandas\nimport pandas as pd\nfrom scipy import stats\n\n# Load the data from the Excel file\nfile_path = 'add_your_dataset_path_here'\ndf = pd.read_excel(file_path)\n\n# Visualize the first rows of the DataFrame\nprint(df.head())\n\n# Check the mean salary by marital status group\nprint(df.groupby(['estado_civil'])['salario'].describe())\n\n# Extract income columns for singles and married individuals\nsingle_income = df[df['estado_civil'] == 's']['salario']\nmarried_income = df[df['estado_civil'] == 'c']['salario']\n\n# Perform the t-test for independent samples\nt_stat, p_value = stats.ttest_ind(married_income, single_income, equal_var=False)\n\n# Display the results of the t-test\nprint(\"Results of the t-Test:\")\nprint(f\"t-statistic: {t_stat}\")\nprint(f\"p-value: {p_value}\")\n\n# Interpret the result\nalpha = 0.05\nif p_value \u003c alpha:\n    print(\"Conclusion: We reject the null hypothesis. The incomes are different for the two marital statuses.\")\nelse:\n    print(\"Conclusion: We do not reject the null hypothesis. The incomes are equal for the two marital statuses.\")\n\n### Conclusion Results of the t-Test:\n\n#### Interpretation:\n - t-statistic: 4.567472731259726 \u003cbr\u003e\n - p-value: 6.527014259249644e-06\n\nThe p-value is extremely small (6.527014259249644e-06), much smaller than the common significance level (0.05). This indicates that there is a significant difference in income between the two marital statuses.\n\n**Conclusion:**\n\nWe reject the null hypothesis. The incomes are different for the two marital statuses (single and married).\n```\n\n\u003cbr\u003e\n\n### 5. One-Way ANOVA\n\n`👇 Copy code`\n\n```python\n# Import necessary libraries\nimport pandas as pd\nimport statsmodels.api as sm\nfrom statsmodels.formula.api import ols\nfrom statsmodels.stats.multicomp import pairwise_tukeyhsd\n\n# File path\nfile_path = 'add_your_dataset_path_here'\n\n# Load the data into Python\ndf = pd.read_excel(file_path)\n\n# Display the first few rows of the DataFrame\nprint(df.head())\n\n# Check the average salary by education level\nprint(\"Average salary by education level:\")\nprint(df.groupby(['grau_instrucao'])['salario'].describe())\n\n# Create a model to compare salary by education level\nmodel = ols('salario ~ grau_instrucao', data=df).fit()\n\n# Perform ANOVA\nanova_result = sm.stats.anova_lm(model)\nprint(\"ANOVA Results:\")\nprint(anova_result)\n\n# Interpret the results\nalpha = 0.05\np_value = anova_result['PR(\u003eF)'][0]\nif p_value \u003c alpha:\n    conclusion_anova = \"There is a significant difference in salaries among different education levels.\"\nelse:\n    conclusion_anova = \"There is no significant difference in salaries among different education levels.\"\nprint(f\"Conclusion from ANOVA: {conclusion_anova}\")\n\n# Post Hoc Tukey Test to evaluate specific differences\ntukey = pairwise_tukeyhsd(endog=df.salario, groups=df.grau_instrucao)\nprint(\"Post Hoc Tukey Test Results:\")\nprint(tukey.summary())\n\n# Interpret Tukey results\nprint(\"Interpreting Tukey's test results:\")\nfor result in tukey.summary().data[1:]:  # Skip header\n    group1, group2, meandiff, p_adj, lower, upper, reject = result\n    if reject:\n        print(f\"Significant difference between {group1} and {group2}: mean difference = {meandiff:.4f}, p-adj = {p_adj:.4f}\")\n    else:\n        print(f\"No significant difference between {group1} and {group2}: mean difference = {meandiff:.4f}, p-adj = {p_adj:.4f}\")\n\n# Overall conclusion\nprint(\"Overall Conclusion:\")\nprint(\"The results indicate that salary is significantly affected by education level, with higher education corresponding to higher salaries.\")\n```\n\n\u003cbr\u003e\n\n### 6.  Two-Way ANOVA\n\n`👇 Copy code`\n\n```python\n# Import necessary libraries\nimport pandas as pd\nimport statsmodels.api as sm\nfrom statsmodels.formula.api import ols\nfrom statsmodels.stats.multicomp import pairwise_tukeyhsd\n\n# File path\nfile_path = 'add_your_dataset_path_here'\n\n# Load the data into Python\ndf = pd.read_excel(file_path)\n\n# Display the first few rows of the DataFrame\nprint(\"Initial DataFrame:\")\nprint(df.head())\n\n# Check the average salary by education level and marital status\nprint(\"\\nAverage salary by education level and marital status:\")\nprint(df.groupby(['grau_instrucao', 'estado_civil'])['salario'].describe())\n\n# Generate the model to compare salary by education level and marital status\nmodel = ols('salario ~ grau_instrucao + estado_civil', data=df).fit()\n\n# Apply ANOVA\nanova_result = sm.stats.anova_lm(model)\nprint(\"\\nANOVA Results:\")\nprint(anova_result)\n\n# Interpret the results\nalpha = 0.05\np_value_instrucao = anova_result['PR(\u003eF)']['grau_instrucao']\np_value_civil = anova_result['PR(\u003eF)']['estado_civil']\n\nif p_value_instrucao \u003c alpha:\n    conclusion_instrucao = \"There is a significant difference in salaries among different education levels.\"\nelse:\n    conclusion_instrucao = \"There is no significant difference in salaries among different education levels.\"\n\nif p_value_civil \u003c alpha:\n    conclusion_civil = \"There is a significant difference in salaries among different marital statuses.\"\nelse:\n    conclusion_civil = \"There is no significant difference in salaries among different marital statuses.\"\n\nprint(f\"\\nConclusion from ANOVA for Education Level: {conclusion_instrucao}\")\nprint(f\"Conclusion from ANOVA for Marital Status: {conclusion_civil}\")\n\n# Post Hoc Tukey Test to evaluate specific differences for marital status\nprint(\"\\nPost Hoc Tukey Test Results for Marital Status:\")\ntukey_estado_civil = pairwise_tukeyhsd(endog=df.salario, groups=df.estado_civil)\nprint(tukey_estado_civil.summary())\n\n# Post Hoc Tukey Test to evaluate specific differences for education level\nprint(\"\\nPost Hoc Tukey Test Results for Education Level:\")\ntukey_instrucao = pairwise_tukeyhsd(endog=df.salario, groups=df.grau_instrucao)\nprint(tukey_instrucao.summary())\n\n# Overall conclusion\nprint(\"\\nOverall Conclusion:\")\nprint(\"The results indicate that salary is significantly affected by both education level and marital status.\")\n```\n\n\u003cbr\u003e\n\n### 7. Two-way ANOVA with Interaction\n\n`👇 Copy code`\n\n```python\n# Import necessary libraries\nimport pandas as pd\nimport statsmodels.api as sm\nfrom statsmodels.formula.api import ols\nfrom statsmodels.stats.multicomp import pairwise_tukeyhsd\n\n# File path\nfile_path = 'add_your_dataset_path_here'\n\n# Load the data into Python\ndf = pd.read_excel(file_path)\n\n# Display the first few rows of the DataFrame\nprint(\"Initial DataFrame:\")\nprint(df.head())\n\n# Check the average salary by education level and marital status\nprint(\"\\nAverage salary by education level and marital status:\")\nprint(df.groupby(['grau_instrucao', 'estado_civil'])['salario'].describe())\n\n# Generate the model to compare salary by education level and marital status, including interaction\nmodel = ols('salario ~ grau_instrucao * estado_civil', data=df).fit()\n\n# Apply ANOVA\nanova = sm.stats.anova_lm(model)\nprint(\"\\nANOVA Results:\")\nprint(anova)\n\n# Interpret the results\nalpha = 0.05\np_value_instrucao = anova['PR(\u003eF)']['grau_instrucao']\np_value_civil = anova['PR(\u003eF)']['estado_civil']\np_value_interaction = anova['PR(\u003eF)']['grau_instrucao:estado_civil']\n\n# Conclusions from ANOVA\nconclusions = {\n    \"Grau de Instrução\": \"significant\" if p_value_instrucao \u003c alpha else \"not significant\",\n    \"Estado Civil\": \"significant\" if p_value_civil \u003c alpha else \"not significant\",\n    \"Interação\": \"significant\" if p_value_interaction \u003c alpha else \"not significant\"\n}\n\nfor factor, result in conclusions.items():\n    print(f\"Conclusion from ANOVA for {factor}: There is a {result} effect.\")\n\n# Post Hoc Tukey Test for marital status\nprint(\"\\nPost Hoc Tukey Test Results for Marital Status:\")\ntukey_estado_civil = pairwise_tukeyhsd(endog=df.salario, groups=df.estado_civil)\nprint(tukey_estado_civil.summary())\n\n# Post Hoc Tukey Test for education level\nprint(\"\\nPost Hoc Tukey Test Results for Education Level:\")\ntukey_instrucao = pairwise_tukeyhsd(endog=df.salario, groups=df.grau_instrucao)\nprint(tukey_instrucao.summary())\n\n# Overall conclusion\nprint(\"\\nOverall Conclusion:\")\nprint(\"The results indicate that salary is significantly affected by both education level and marital status,\")\nprint(\"and there is also a significant interaction between these two factors.\")\n```\n\n\u003cbr\u003e\n\n### 8. Chi-Square Test for One Variable.\n\n`👇 Copy code`\n\n```python\n# Import necessary libraries\nimport pandas as pd\nimport scipy.stats as stats\n\n# File path\nfile_path = 'add_your_dataset_path_here'\n\n# Load the data into a DataFrame\ndf = pd.read_excel(file_path)\n\n# Display the first few rows of the DataFrame\nprint(\"Initial DataFrame:\")\nprint(df.head())\n\n# Frequency table for the variable 'reg_proc'\nfreq_reg_proc = df['reg_proc'].value_counts()\nprint(\"\\nFrequency of the 'reg_proc' variable:\")\nprint(freq_reg_proc)\n\n# Perform Chi-Square Test\nchi2_stat, p_val = stats.chisquare(freq_reg_proc)\nprint(\"\\nChi-Square Test Results:\")\nprint(f\"Chi-Square Statistic: {chi2_stat}\")\nprint(f\"p-value: {p_val}\")\n\n# Interpretation and Conclusion\nalpha = 0.05  # Significance level\nif p_val \u003c alpha:\n    print(\"Reject the null hypothesis. The distribution of the region of origin is not the same in the sample.\")\nelse:\n    print(\"Fail to reject the null hypothesis. The distribution of the region of origin is the same in the sample.\")\n```\n\n\u003cbr\u003e\n\n9. Chi-Square Test for Independence of Two Variables\n\n`👇 Copy code`\n\n```python\n# Import necessary libraries\nimport pandas as pd\nimport scipy.stats as stats\n\n# File path\nfile_path = 'add_your_dataset_path_here'\n\n# Load the data into a DataFrame\ndf = pd.read_excel(file_path)\n\n# Display the first few rows of the DataFrame\nprint(\"Initial DataFrame:\")\nprint(df.head())\n\n# Create a contingency table for 'grau_instrucao' and 'reg_proc'\ncontingency_table = pd.crosstab(df['grau_instrucao'], df['reg_proc'])\nprint(\"\\nContingency Table:\")\nprint(contingency_table)\n\n# Perform Chi-Square Test of Independence\nchi2_stat, p_val, dof, expected = stats.chi2_contingency(contingency_table)\nprint(\"\\nResults of the Chi-Square Test of Independence:\")\nprint(f\"Chi-Square Statistic: {chi2_stat}\")\nprint(f\"p-value: {p_val}\")\nprint(f\"Degrees of Freedom: {dof}\")\nprint(\"Expected Frequencies:\")\nprint(expected)\n\n# Interpretation and Conclusion\nalpha = 0.05  # Significance level\nif p_val \u003c alpha:\n    print(\"\\nConclusion: Reject the null hypothesis. The distribution of education levels varies according to the region of origin.\")\nelse:\n    print(\"\\nConclusion: Fail to reject the null hypothesis. The distribution of education levels does not vary according to the region of origin.\")\n```\n\n\n\n\n\u003cbr\u003e\u003cbr\u003e\n\n\u003cp align=\"center\"\u003e \u003ca href=\"#top\"\u003eBack to Top\u003c/a\u003e\n\n\n#\n###### \u003cp align=\"center\"\u003e Copyright 2024 Fabiana Campanari. Code released under the [MIT license.](https://github.com/FabianaCampanari/FabianaCampanari/blob/66325d147794b5fc4688d56e6b78e8cdf42946e4/LICENSE)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabianacampanari%2Fpracticalstats-pucsp-2024","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabianacampanari%2Fpracticalstats-pucsp-2024","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabianacampanari%2Fpracticalstats-pucsp-2024/lists"}