{"id":19937210,"url":"https://github.com/santiviquez/ageml","last_synced_at":"2025-05-03T14:31:12.917Z","repository":{"id":193776301,"uuid":"647473679","full_name":"santiviquez/ageml","owner":"santiviquez","description":"Study the temporal performance degradation of machine learning models.","archived":false,"fork":false,"pushed_at":"2024-01-26T20:32:04.000Z","size":18570,"stargazers_count":16,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-07T16:21:14.280Z","etag":null,"topics":["machine-learning","performance-degradation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/santiviquez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-30T21:30:36.000Z","updated_at":"2025-03-30T02:17:11.000Z","dependencies_parsed_at":"2024-01-26T21:31:56.220Z","dependency_job_id":null,"html_url":"https://github.com/santiviquez/ageml","commit_stats":null,"previous_names":["santiviquez/ageml"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/santiviquez%2Fageml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/santiviquez%2Fageml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/santiviquez%2Fageml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/santiviquez%2Fageml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/santiviquez","download_url":"https://codeload.github.com/santiviquez/ageml/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252203190,"owners_count":21710904,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","performance-degradation"],"created_at":"2024-11-12T23:31:43.690Z","updated_at":"2025-05-03T14:31:11.536Z","avatar_url":"https://github.com/santiviquez.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# ageML\nageML is a framework designed to study the temporal performance degradation of machine learning models. The goal of this project is to facilitate the exploration of performance degradation by providing tools that allow users to easily test how their models would evolve over time when trained and tested on different periods of their data.\n\nDisclaimer: This project is still in its early stages, so the code interface might change in the future, and some elements might be hardcoded. However, the idea is to improve it over time, making it more user-friendly.\n\n\n\u003cp align=\"center\"\u003e\n \u003cimg src=\"figures/model_aging_plot_avocados_lr.png\" alt=\"temporal degradation plot of a linear regressor on the avocado sales dataset\" width=\"600\"/\u003e\n\u003c/p\u003e\n\n## Features\nCurrently, ageML implements one test to study the \"aging\" process that machine learning models can experience when in production due to covariate or concept shift.\n\n### Temporal Degradation Test\nExamines how various models perform when trained on different samples of the same dataset. This framework is based on the aging framework developed by [Vela et al.](https://www.nature.com/articles/s41598-022-15245-z) in 2022.\n\n\u003cp align=\"center\"\u003e\n \u003cimg src=\"figures/temporal_degradadation_test.svg\" alt=\"temporal degradation test\" width=\"600\"/\u003e\n\u003c/p\u003e\n\n### WIP: Continuous Retraining Test\nSimulates a fixed-schedule retraining process of a machine learning model in production.\n\u003cp align=\"center\"\u003e\n \u003cimg src=\"figures/continuous_retraining_test.svg\" alt=\"continuous retraining test\" width=\"600\"/\u003e\n\u003c/p\u003e\n\n\n## Installation\nThe package hasn't been published on PyPI yet, which means you cannot install it via the regular Python channels. Instead, you'll have to clone the repository and install it from your local copy.\n\n```bash\ngit clone https://github.com/santiviquez/ageml.git\ncd ageml\npip install .\n```\n\n## Quickstart\n\n```python\nfrom ageml import TemporalDegradation\nfrom ageml.datasets import load_avocado_sales\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import mean_absolute_error\n\ndata = load_avocado_sales()\n\nexperiment = TemporalDegradation(\n    timestamp_column_name='inference_time',\n    target_column_name='demand',\n    n_train_samples=52,\n    n_test_samples=12,\n    n_prod_samples=24,\n    n_simulations=10)\n\nexperiment.run(data, model=LinearRegression())\n\nexperiment.plot(\n    freq='W',\n    metric=mean_absolute_error,\n    min_test_error=1e7,\n    plot_name='Model Ageing Chart: Avocado Sales Prediction - LinearRegression')\n\nresults = experiment.get_results(\n    freq='W',\n    metric=mean_absolute_error,\n    min_test_error=1e7)\n\nprint(results)\n```\n\n## Quickstart\nCheck out the [issues page](https://github.com/santiviquez/ageml/issues) if you want to start building this with me 😊\n\n## Author\n- [santiviquez](https://www.twitter.com/santiviquez)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsantiviquez%2Fageml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsantiviquez%2Fageml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsantiviquez%2Fageml/lists"}