{"id":24641435,"url":"https://github.com/pawlo77/autoprep","last_synced_at":"2025-08-23T19:17:39.088Z","repository":{"id":268724283,"uuid":"905282292","full_name":"Pawlo77/AutoPrep","owner":"Pawlo77","description":"Automatic ML library for enhanced data preprocessing and explainability ","archived":false,"fork":false,"pushed_at":"2025-01-12T08:35:22.000Z","size":17554,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-12T09:29:46.515Z","etag":null,"topics":["automl","data-science","maschine-learning","pipeline","python"],"latest_commit_sha":null,"homepage":"https://pawlo77.github.io/AutoPrep/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Pawlo77.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-18T14:12:13.000Z","updated_at":"2025-01-04T21:02:09.000Z","dependencies_parsed_at":"2024-12-18T14:51:21.115Z","dependency_job_id":"9733a9a8-46d2-4bf4-8b40-d473f306b314","html_url":"https://github.com/Pawlo77/AutoPrep","commit_stats":null,"previous_names":["pawlo77/autoprep"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pawlo77%2FAutoPrep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pawlo77%2FAutoPrep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pawlo77%2FAutoPrep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pawlo77%2FAutoPrep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Pawlo77","download_url":"https://codeload.github.com/Pawlo77/AutoPrep/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235575658,"owners_count":19012159,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automl","data-science","maschine-learning","pipeline","python"],"created_at":"2025-01-25T12:13:24.350Z","updated_at":"2025-01-25T12:13:24.947Z","avatar_url":"https://github.com/Pawlo77.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Auto-Prep\n\n**Auto-Prep** is an automated data preprocessing and analysis pipeline that generates comprehensive LaTeX reports. It handles common preprocessing tasks, creates insightful visualizations, and documents the entire process in a professional PDF report. It focuses on tabular data, supporting numerous explainable AI models. Emphasizing interpretability and ease of use, it includes subsections for each model, explaining their strengths, weaknesses, and providing usage examples.\n\nFor detailed product description [see this notebook](https://github.com/Pawlo77/AutoPrep/tree/main/examples/walkthrough/walkthrough.ipynb)\n\n## [Docs](https://pawlo77.github.io/AutoPrep/)\n\n## Features\n\n- **Automated data cleaning and preprocessing**\n- **Intelligent feature type detection**\n- **Advanced categorical encoding with rare category handling**\n- **Comprehensive exploratory data analysis (EDA)**\n- **Automated visualization generation**\n- **Professional LaTeX report generation**\n- **Modular and extensible design**\n- **Support for numerous explainable ML models**\n- **Explainability with model-specific examples**\n\n## Report Contents\n\nThe generated report includes:\n\n1. **Title page and table of contents**\n2. **Overview**\n   - Platform structure\n   - Dataset structure\n3. **Exploratory Data Analysis**\n   - Distribution plots\n   - Correlation matrix\n   - Missing value analysis\n4. **Model Performance**\n   - Accuracy metrics\n   - Model details\n\n## Installation\n\nIn order to use our tool, you need to have latex intalled on your local machine.\n\n### Using pip (Recommended)\n\n1. Install Auto-Prep directly from PyPI:\n    ```bash\n    pip install auto-prep\n    ```\n\n2. Run the example usage:\n    ```bash\n    python example_usage.py\n    ```\n\n### Using Poetry\n\n1. Ensure you have Poetry installed:\n    ```bash\n    curl -sSL https://install.python-poetry.org | python3 -\n    ```\n\n2. Clone the repository:\n    ```bash\n    git clone https://github.com/yourusername/auto-prep.git\n    cd auto-prep\n    ```\n\n3. Install dependencies:\n    ```bash\n    poetry install\n    ```\n\n4. Activate the virtual environment:\n    ```bash\n    poetry shell\n    ```\n\n5. Run the example usage:\n    ```bash\n    python example_usage.py\n    ```\n\n## Important informations\n\n- due to multiprocessing enabled, run method is recommended to be called under name __main__ check - see example in next point. Number of cores used can be set in config.\n\n- difference between `config.set` vs `config.update` - first one can be used to see default values for each setting, and it will overwritte all non-passed values to their defaults. Second option will just overwritte provied arguments without validation, can be used to create new fields in config.\n\n- `config.root_dir` if exists is cleared on call of `AutoPrep().run()`. If logs are pointed to be stored there, it will delete their file handlers causing errors.\n\n- logs returned to console might be very unreadable due to many warnings in dependencies. Please refer to stored log files for clean logs.\n\n- for changes in config to be loaded, config.update must be called before any other import from autoprep package - as example:\n\n    ```python\n    import logging\n    from auto_prep.utils import config\n\n    config.update(log_level=logging.DEBUG)\n\n    import numpy as np\n\n    from auto_prep.prep import AutoPrep\n    from sklearn.datasets import fetch_openml\n\n    # Load your dataset\n    data = fetch_openml(name=\"titanic\", version=1, as_frame=True, parser=\"auto\").frame\n    data[\"survived\"] = data[\"survived\"].astype(np.uint8)\n\n    # Create and run pipeline\n    pipeline = AutoPrep()\n\n    if __name__ == \"__main__\":\n        pipeline.run(data, target_column=\"survived\")\n    ```\n\n    For same reason AutoPrep is not exported to top-level package. It is known implementation fault.\n\n## Examples\n\nRefer to [this folder](https://github.com/Pawlo77/AutoPrep/tree/main/examples/).\n\n## Author\n\n- **Paweł Pozorski** - [GitHub](https://github.com/Pawlo77)\n- **Katarzyna Rogalska**\n- **Julia Kruk**\n- **Gaspar Sekula**\n\n## Notes for Developers\n\n1. Poetry is used for dependency management and virtual environments. The following functions are implemented:\n   - `poetry run format` - Format code\n   - `poetry run lint` - Lint code\n   - `poetry run check` - Check code\n   - `poetry run test` - Run tests\n   - `poetry run pre-commit run --all-files` - Run pre-commit hooks\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpawlo77%2Fautoprep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpawlo77%2Fautoprep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpawlo77%2Fautoprep/lists"}