{"id":48434026,"url":"https://github.com/ingannamortesciencedev/ecom-ru-stat-analysis","last_synced_at":"2026-04-06T12:02:20.804Z","repository":{"id":346137559,"uuid":"1188621016","full_name":"IngannamorteScienceDev/ecom-ru-stat-analysis","owner":"IngannamorteScienceDev","description":"Statistical analysis of the Russian e-commerce market for a master's thesis in Business Informatics.","archived":false,"fork":false,"pushed_at":"2026-03-22T21:32:21.000Z","size":2354,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-23T03:25:34.831Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IngannamorteScienceDev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-22T10:54:44.000Z","updated_at":"2026-03-22T21:32:24.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/IngannamorteScienceDev/ecom-ru-stat-analysis","commit_stats":null,"previous_names":["ingannamortesciencedev/ecom-ru-stat-analysis"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/IngannamorteScienceDev/ecom-ru-stat-analysis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IngannamorteScienceDev%2Fecom-ru-stat-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IngannamorteScienceDev%2Fecom-ru-stat-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IngannamorteScienceDev%2Fecom-ru-stat-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IngannamorteScienceDev%2Fecom-ru-stat-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IngannamorteScienceDev","download_url":"https://codeload.github.com/IngannamorteScienceDev/ecom-ru-stat-analysis/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IngannamorteScienceDev%2Fecom-ru-stat-analysis/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31471469,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-06T08:36:52.050Z","status":"ssl_error","status_checked_at":"2026-04-06T08:36:51.267Z","response_time":112,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-06T12:02:18.279Z","updated_at":"2026-04-06T12:02:20.793Z","avatar_url":"https://github.com/IngannamorteScienceDev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# ecom-ru-stat-analysis\n\n### Statistical analysis of the Russian e-commerce market  \n### Master's thesis analytics project in **Business Informatics**\n\n\u003cp align=\"center\"\u003e\n  \u003cimg alt=\"Python\" src=\"https://img.shields.io/badge/Python-3.10%2B-blue.svg\"\u003e\n  \u003cimg alt=\"Status\" src=\"https://img.shields.io/badge/status-thesis%20analytics-success\"\u003e\n  \u003cimg alt=\"Focus\" src=\"https://img.shields.io/badge/focus-statistics%20%7C%20forecasting%20%7C%20regional%20analysis-purple\"\u003e\n  \u003cimg alt=\"Domain\" src=\"https://img.shields.io/badge/domain-e--commerce-orange\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  Reproducible Python pipeline for collecting, transforming, analyzing, and visualizing data on the development of the Russian e-commerce market.\n\u003c/p\u003e\n\n\u003c/div\u003e\n\n---\n\n## Overview\n\nThis repository contains a full analytical pipeline created for a master's thesis devoted to the **statistical analysis of the Russian e-commerce market**.\n\nThe project combines:\n\n- official statistical data,\n- reproducible Python-based calculations,\n- regional comparative analytics,\n- forecasting,\n- factor analysis,\n- exportable figures and tables.\n\nThe repository is designed as both:\n\n- an academic research project,\n- and a portfolio-grade analytics system.\n\n---\n\n## Project objectives\n\nThe project is designed to solve the following analytical tasks:\n\n- evaluate the dynamics of the share of online sales in Russian retail trade,\n- identify regional leaders and outsiders,\n- assess interregional differentiation,\n- detect long-term growth leaders,\n- compare trajectories of federal districts,\n- estimate factor relationships using external variables,\n- build and compare short-term forecasting models,\n- export reproducible analytical outputs.\n\n---\n\n## Repository structure\n\n```text\necom-ru-stat-analysis/\n├── config/\n│   └── config.yaml\n├── data/\n│   ├── raw/\n│   ├── external/\n│   ├── interim/\n│   └── processed/\n├── notebooks/\n│   ├── 01_collect.ipynb\n│   ├── 02_prepare.ipynb\n│   ├── 03_analysis.ipynb\n│   └── 04_modeling.ipynb\n├── outputs/\n│   ├── figures/\n│   ├── tables/\n│   ├── models/\n│   └── logs/\n├── scripts/\n│   ├── clean_project.py\n│   ├── run_collect.py\n│   ├── run_prepare.py\n│   ├── run_analysis.py\n│   ├── run_modeling.py\n│   └── run_full_pipeline.py\n├── src/\n│   ├── collect/\n│   ├── processing/\n│   ├── analysis/\n│   ├── modeling/\n│   ├── viz/\n│   └── utils/\n├── tests/\n├── .gitignore\n├── LICENSE\n├── pyproject.toml\n├── README.md\n└── requirements.txt\n````\n\n---\n\n## Core functionality\n\nThe pipeline includes the following analytical blocks.\n\n### Data preparation\n\n* parsing Rosstat source files,\n* extracting national and regional data,\n* splitting subjects of the Russian Federation and federal districts,\n* harmonizing datasets for further analysis.\n\n### Statistical metrics\n\n* absolute change,\n* growth rate,\n* growth increment,\n* CAGR,\n* quartile summaries,\n* outlier detection.\n\n### Regional analytics\n\n* top and bottom subject rankings,\n* leaders by absolute increase,\n* leaders by CAGR,\n* quartile dynamics,\n* heatmap-ready matrices,\n* federal district summaries.\n\n### Factor analysis\n\n* optional external factor merging,\n* correlation matrix generation,\n* cross-sectional regression for the latest valid year,\n* export of coefficients, model fit, and regression sample.\n\n### Forecasting\n\n* model comparison,\n* holdout-based evaluation,\n* baseline forecast generation,\n* confidence interval export.\n\n### Export\n\n* PNG graphics,\n* CSV analytical tables,\n* consolidated Excel workbook.\n\n---\n\n## Data sources\n\nThe project is based on a combination of official and auxiliary sources.\n\n### Main source\n\n**Rosstat**\n\n* share of online sales in total retail turnover,\n* Russian Federation level,\n* subject level,\n* federal district level.\n\n### External factors\n\nDepending on availability, the project can also integrate:\n\n* population by region and year,\n* per capita income,\n* ICT-related indicators,\n* payment and digital transaction indicators.\n\n### Design principle\n\nThe project architecture supports two modes:\n\n* a **core descriptive mode** based on the Rosstat block,\n* an **extended explanatory mode** with external factors.\n\n---\n\n## Analytical workflow\n\n```mermaid\nflowchart TD\n    A[Raw source files] --\u003e B[Cleaning and harmonization]\n    B --\u003e C[Processed analytical datasets]\n    C --\u003e D[Statistical metrics]\n    D --\u003e E[Regional analysis]\n    D --\u003e F[Factor analysis]\n    D --\u003e G[Forecasting]\n    E --\u003e H[Figures and tables]\n    F --\u003e H\n    G --\u003e H\n    H --\u003e I[Final analytical outputs]\n```\n\n---\n\n## Visual showcase\n\n### Share of online sales in Russian retail turnover\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/share_online_rf_dynamics.png\" alt=\"RF dynamics\" width=\"900\"\u003e\n\u003c/p\u003e\n\n### Annual growth increments of the online sales share\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/rf_growth_increment_pct.png\" alt=\"RF growth\" width=\"900\"\u003e\n\u003c/p\u003e\n\n### Top subjects of the Russian Federation by online sales share\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/top_subjects_latest_year.png\" alt=\"Top subjects\" width=\"900\"\u003e\n\u003c/p\u003e\n\n### Bottom subjects of the Russian Federation by online sales share\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/bottom_subjects_latest_year.png\" alt=\"Bottom subjects\" width=\"900\"\u003e\n\u003c/p\u003e\n\n### Leaders by absolute increase in online sales share\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/top_subjects_absolute_increase.png\" alt=\"Absolute increase\" width=\"900\"\u003e\n\u003c/p\u003e\n\n### Leaders by CAGR\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/top_subjects_cagr.png\" alt=\"CAGR leaders\" width=\"900\"\u003e\n\u003c/p\u003e\n\n### Quartile dynamics of regional distribution\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/subjects_quartiles_by_year.png\" alt=\"Quartiles\" width=\"900\"\u003e\n\u003c/p\u003e\n\n### Federal district dynamics\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/federal_districts_dynamics.png\" alt=\"Federal districts\" width=\"900\"\u003e\n\u003c/p\u003e\n\n### Historical trajectory and forecast\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./outputs/figures/rf_history_and_forecast.png\" alt=\"Forecast\" width=\"900\"\u003e\n\u003c/p\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eAdditional figures\u003c/strong\u003e\u003c/summary\u003e\n\n\u003cbr\u003e\n\n* `./outputs/figures/subjects_boxplot_latest_year.png`\n* `./outputs/figures/subjects_heatmap_top.png`\n* `./outputs/figures/subjects_first_last_scatter.png`\n* `./outputs/figures/subjects_factor_regression_coefficients.png` (if factor regression is available)\n\n\u003c/details\u003e\n\n---\n\n## Key outputs\n\n### Processed datasets\n\nLocated in `data/processed/`\n\nMain files include:\n\n* `master_rf_dataset_with_metrics.csv`\n* `master_regions_subjects_dataset_with_metrics.csv`\n* `master_federal_districts_dataset_with_metrics.csv`\n\n### Figures\n\nLocated in `outputs/figures/`\n\nMain files include:\n\n* `share_online_rf_dynamics.png`\n* `rf_growth_increment_pct.png`\n* `top_subjects_latest_year.png`\n* `bottom_subjects_latest_year.png`\n* `top_subjects_absolute_increase.png`\n* `top_subjects_cagr.png`\n* `subjects_first_last_scatter.png`\n* `subjects_quartiles_by_year.png`\n* `subjects_boxplot_latest_year.png`\n* `subjects_heatmap_top.png`\n* `federal_districts_dynamics.png`\n* `rf_history_and_forecast.png`\n\n### Tables\n\nLocated in `outputs/tables/`\n\nMain files include:\n\n* `top_subjects_latest_year.csv`\n* `bottom_subjects_latest_year.csv`\n* `subjects_first_last_comparison.csv`\n* `top_subjects_absolute_increase.csv`\n* `top_subjects_cagr.csv`\n* `subjects_quartiles_by_year.csv`\n* `subject_outliers_latest_year.csv`\n* `federal_districts_latest_year.csv`\n* `subjects_heatmap_matrix.csv`\n* `subjects_factor_correlation_matrix.csv`\n* `subjects_factor_regression_coefficients.csv`\n* `subjects_factor_regression_fit.csv`\n* `subjects_factor_regression_sample.csv`\n* `analytical_tables.xlsx`\n\n### Model outputs\n\nLocated in `outputs/models/`\n\nMain files:\n\n* `forecast_model_comparison.csv`\n* `forecast_rf_values.csv`\n\n---\n\n## Installation\n\n### Clone the repository\n\n```bash\ngit clone https://github.com/IngannamorteScienceDev/ecom-ru-stat-analysis.git\ncd ecom-ru-stat-analysis\n```\n\n### Create and activate a virtual environment\n\n#### Windows PowerShell\n\n```powershell\npython -m venv .venv\n.\\.venv\\Scripts\\Activate.ps1\n```\n\n#### macOS / Linux\n\n```bash\npython -m venv .venv\nsource .venv/bin/activate\n```\n\n### Install dependencies\n\n```bash\npip install -r requirements.txt\n```\n\n---\n\n## Usage\n\n### Input data placement\n\nSource files should be placed into:\n\n```text\ndata/raw/\ndata/external/\n```\n\nTypical raw source file:\n\n```text\ndata/raw/rosstat_share_online_regions.xlsx\n```\n\nSupported external factor filenames:\n\n* `population_by_region_year.csv`\n* `income_pc_by_region_year.csv`\n* `online_buyers_pct_by_region_year.csv`\n* `cashless_payments_index_by_region_year.csv`\n\nExpected long format for external factors:\n\n```csv\nregion,year,factor_name\nАлтайский край,2022,123.45\n...\n```\n\nThe value column must match the factor name expected in `config/config.yaml`.\n\n### Full pipeline run\n\n```bash\npython scripts/run_full_pipeline.py\n```\n\nThis command:\n\n* removes generated artifacts and cache,\n* rebuilds processed data,\n* recalculates metrics,\n* runs structural analytics,\n* runs factor analysis,\n* compares and builds forecast models,\n* generates figures,\n* exports tables.\n\n### Step-by-step execution\n\n```bash\npython scripts/run_prepare.py\npython scripts/run_modeling.py\npython scripts/run_analysis.py\n```\n\n---\n\n## Generated files by stage\n\n### Preparation\n\n* cleaned RF dataset,\n* cleaned regional dataset,\n* subject-level and federal district processed layers,\n* optional external factor merge.\n\n### Analysis\n\n* rankings,\n* quartiles,\n* outlier tables,\n* comparative plots,\n* heatmap matrix,\n* factor matrices and regression outputs.\n\n### Forecasting\n\n* model comparison table,\n* final forecast table with interval.\n\n### Export\n\n* consolidated Excel workbook,\n* figure set,\n* model outputs.\n\n---\n\n## Methodological notes\n\n### Territorial hierarchy\n\nThe project explicitly separates:\n\n* Russian Federation,\n* federal districts,\n* subjects of the Russian Federation.\n\nThis avoids methodological distortion in comparative analytics.\n\n### Forecasting approach\n\nThe forecasting block uses practical model comparison:\n\n* naive benchmark,\n* linear trend,\n* Holt linear trend.\n\nThe best model is selected using holdout-period error metrics.\n\n### Factor analysis logic\n\nThe factor block is designed to remain usable even with imperfect external coverage:\n\n* it searches for the latest year with sufficient common factor data,\n* builds a valid regional sample,\n* estimates cross-sectional regression,\n* exports interpretable outputs.\n\n### Reproducibility\n\nThe repository is structured around a reproducible workflow:\n\n* source data are preserved,\n* generated artifacts are rebuilt automatically,\n* tables and figures can be reproduced before defense or review.\n\n---\n\n## Troubleshooting\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eModuleNotFoundError: No module named 'src'\u003c/strong\u003e\u003c/summary\u003e\n\nAll project scripts inject the project root into `sys.path`.\nRun commands from the repository root.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eForecast file not found\u003c/strong\u003e\u003c/summary\u003e\n\nRun the full pipeline:\n\n```bash\npython scripts/run_full_pipeline.py\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eFactor regression sample is empty\u003c/strong\u003e\u003c/summary\u003e\n\nThis means the external factor files do not overlap sufficiently by:\n\n* region,\n* year,\n* variable availability.\n\nCheck:\n\n* filenames,\n* column names,\n* year coverage,\n* consistency of region labels.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eFigures do not render in GitHub README\u003c/strong\u003e\u003c/summary\u003e\n\nMake sure the image files are tracked by Git and pushed to the repository:\n\n```bash\ngit add -f outputs/figures/*\ngit commit -m \"docs: add generated figures for README showcase\"\ngit push\n```\n\n\u003c/details\u003e\n\n---\n\n## Future improvements\n\nPotential extensions:\n\n* richer panel factor modeling,\n* automated downloaders for public source files,\n* geospatial map-based analytics,\n* dashboard layer for interactive exploration,\n* formal report generation.\n\n---\n\n## Author\n\n**Yulia Liubushkina**\nGitHub: [@ZeroZivi](https://github.com/ZeroZivi)\n\n---\n\n## License\n\nThis repository is provided for academic and research purposes.\nSee [`LICENSE`](./LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fingannamortesciencedev%2Fecom-ru-stat-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fingannamortesciencedev%2Fecom-ru-stat-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fingannamortesciencedev%2Fecom-ru-stat-analysis/lists"}