{"id":41166998,"url":"https://github.com/vcerqueira/modelradar","last_synced_at":"2026-01-22T19:34:59.079Z","repository":{"id":234272010,"uuid":"788504025","full_name":"vcerqueira/modelradar","owner":"vcerqueira","description":"Aspect-based Forecasting Accuracy","archived":false,"fork":false,"pushed_at":"2025-07-15T20:51:52.000Z","size":10025,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-25T13:55:32.993Z","etag":null,"topics":["deep-learning","evaluation-framework","forecasting","machine-learning","time-series"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vcerqueira.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-04-18T14:44:11.000Z","updated_at":"2025-07-15T20:51:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"772c753a-9780-429f-a0c5-f6ab65df1aa0","html_url":"https://github.com/vcerqueira/modelradar","commit_stats":null,"previous_names":["vcerqueira/modelradar"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/vcerqueira/modelradar","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vcerqueira%2Fmodelradar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vcerqueira%2Fmodelradar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vcerqueira%2Fmodelradar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vcerqueira%2Fmodelradar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vcerqueira","download_url":"https://codeload.github.com/vcerqueira/modelradar/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vcerqueira%2Fmodelradar/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28669270,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T17:07:18.858Z","status":"ssl_error","status_checked_at":"2026-01-22T17:05:02.040Z","response_time":144,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","evaluation-framework","forecasting","machine-learning","time-series"],"created_at":"2026-01-22T19:34:59.010Z","updated_at":"2026-01-22T19:34:59.070Z","avatar_url":"https://github.com/vcerqueira.png","language":"Jupyter Notebook","readme":"# Model Radar 🎯\n\nA framework for aspect-based evaluation of time series forecasting models based on Nixtla's ecosystem.\n\n[![PyPi Version](https://img.shields.io/pypi/v/modelradar)](https://pypi.org/project/modelradar/)\n[![GitHub](https://img.shields.io/github/stars/vcerqueira/modelradar?style=social)](https://github.com/vcerqueira/modelradar)\n[![Downloads](https://static.pepy.tech/badge/modelradar)](https://pepy.tech/project/modelradar)\n\n## Overview\n\nModel Radar introduces a novel aspect-based forecasting evaluation approach that goes beyond traditional aggregate metrics. Our framework enables:\n- Fine-grained performance analysis across different forecasting aspects\n- Better understanding of model behavior in varying conditions\n- More informed model selection based on specific use case requirements\n\n## 🚀 Getting Started\n\nCheck the `notebooks` folder for usage examples and tutorials.\n\nCheck **[ModelRadar-Experiments](https://github.com/vcerqueira/experiments-modelradar)** repository for a thorough\napplication of ModelRadar.\n\n## Installation\n\nYou can install modelradar using pip:\n\n```bash\npip install modelradar\n```\n\n### [Optional] Installation from source\n\nTo install modelradar from source, clone the repository and run the following command:\n\n```bash\ngit clone https://github.com/vcerqueira/modelradar\npip install -e modelradar\n```\n\n### Prerequisites\n\nRequired dependencies:\n```\nutilsforecast==0.2.11\nnumpy==1.26.4\nplotnine==0.14.5\n```\n\n⚠️ I've noticed some issues when running with more recent versions of numpy and utilsforecast. \nTry to use the versions above.\n\n### Examples\n\nBesides the examples in the `notebooks` folder, here's some outputs you can get from *modelradar*:\n\n- Spider chart with overview on several dimensions:\n\n![radar](assets/examples/radar.png)\n\n- Parallel coordinates chart with overview on several dimensions:\n\n![radar2](assets/examples/parcoords.png)\n\n- Barplot chart controlling for a given variable (in this case, anomaly status):\n\n![radar2](assets/examples/anomaly_status.png)\n\n- Grouped bar plot showing win/draw/loss ratios wrt different models:\n\n\u003cimg src=\"assets/examples/win_ratios.png\" width=\"70%\" alt=\"radar2\"\u003e\n\n\n## 📑 References\n\n\u003e Cerqueira, V., Roque, L., \u0026 Soares, C. \"Forecasting with Deep Learning: Beyond Average of Average of Average Performance.\" Discovery Science: 27th International Conference, DS 2024, Pisa, Italy, 2024, Proceedings 27. Springer International Publishing, 2024.\n\nCheck DS24 folder to reproduce the experiments published on this paper.\nThe main repository and package contains an updated framework.\n\n### **⚠️ WARNING**\n\n\u003e modelradar is in the early stages of development. \n\u003e The codebase may undergo significant changes. \n\u003e If you encounter any issues, please report\n\u003e them in [GitHub Issues](https://github.com/vcerqueira/modelradar/issues)","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvcerqueira%2Fmodelradar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvcerqueira%2Fmodelradar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvcerqueira%2Fmodelradar/lists"}