{"id":15057235,"url":"https://github.com/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl","last_synced_at":"2025-10-04T16:32:05.269Z","repository":{"id":238433178,"uuid":"796531436","full_name":"JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl","owner":"JuliaDecisionFocusedLearning","description":"Benchmarks for decision-focused learning","archived":false,"fork":false,"pushed_at":"2024-10-17T15:30:31.000Z","size":491,"stargazers_count":6,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2024-10-19T06:12:17.408Z","etag":null,"topics":["autodiff","combinatorics","graphs","julia","machine-learning","optimization"],"latest_commit_sha":null,"homepage":"","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JuliaDecisionFocusedLearning.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-06T06:10:32.000Z","updated_at":"2024-10-16T14:35:16.000Z","dependencies_parsed_at":"2024-05-14T09:49:20.174Z","dependency_job_id":null,"html_url":"https://github.com/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl","commit_stats":null,"previous_names":["juliadecisionfocusedlearning/inferoptbenchmarks.jl","juliadecisionfocusedlearning/decisionfocusedlearningbenchmarks.jl"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaDecisionFocusedLearning%2FDecisionFocusedLearningBenchmarks.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaDecisionFocusedLearning%2FDecisionFocusedLearningBenchmarks.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaDecisionFocusedLearning%2FDecisionFocusedLearningBenchmarks.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaDecisionFocusedLearning%2FDecisionFocusedLearningBenchmarks.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JuliaDecisionFocusedLearning","download_url":"https://codeload.github.com/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235285441,"owners_count":18965324,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autodiff","combinatorics","graphs","julia","machine-learning","optimization"],"created_at":"2024-09-24T22:04:16.706Z","updated_at":"2025-10-04T16:32:05.251Z","avatar_url":"https://github.com/JuliaDecisionFocusedLearning.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DecisionFocusedLearningBenchmarks.jl\n\n[![Stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://JuliaDecisionFocusedLearning.github.io/DecisionFocusedLearningBenchmarks.jl/stable/)\n[![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://JuliaDecisionFocusedLearning.github.io/DecisionFocusedLearningBenchmarks.jl/dev/)\n[![Build Status](https://github.com/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl/actions/workflows/Test.yml/badge.svg?branch=main)](https://github.com/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl/actions/workflows/Test.yml?query=branch%3Amain)\n[![Coverage](https://codecov.io/gh/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl/branch/main/graph/badge.svg)](https://app.codecov.io/gh/JuliaDecisionFocusedLearning/DecisionFocusedLearningBenchmarks.jl)\n[![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/JuliaDiff/BlueStyle)\n\n\u003e [!WARNING]  \n\u003e  This package is currently under active development. The API may change in future releases.\n\u003e  Please refer to the [documentation](https://JuliaDecisionFocusedLearning.github.io/DecisionFocusedLearningBenchmarks.jl/stable/) for the latest updates.\n\n## What is Decision-Focused Learning?\n\nDecision-Focused Learning (DFL) is a paradigm that integrates machine learning prediction with combinatorial optimization to make better decisions under uncertainty.\nUnlike traditional \"predict-then-optimize\" approaches that optimize prediction accuracy independently of downstream decision quality, DFL directly optimizes end-to-end decision performance.\n\nA typical DFL algorithm involves training a parametrized policy that combines a statistical predictor with an optimization component:\n\n```math\nx \\;\\longrightarrow\\; \\boxed{\\,\\text{Statistical model } \\varphi_w\\,} \n\\;\\xrightarrow{\\theta}\\; \\boxed{\\,\\text{CO algorithm } f\\,} \n\\;\\longrightarrow\\; y\n```\n\nWhere:\n- **Statistical model** $\\varphi_w$: machine learning predictor (e.g., neural network)\n- **CO algorithm** $f$: combinatorial optimization solver\n- **Instance** $x$: input data (e.g., features, context)\n- **Parameters** $\\theta$: predicted parameters for the optimization problem solved by `f`\n- **Solution** $y$: output decision/solution\n\n## Package Overview\n\n**DecisionFocusedLearningBenchmarks.jl** provides a comprehensive collection of benchmark problems for evaluating decision-focused learning algorithms. The package offers:\n\n- **Standardized benchmark problems** spanning diverse application domains\n- **Common interfaces** for creating datasets, statistical models, and optimization algorithms\n- **Ready-to-use DFL policies** compatible with [InferOpt.jl](https://github.com/JuliaDecisionFocusedLearning/InferOpt.jl) and the whole [JuliaDecisionFocusedLearning](https://github.com/JuliaDecisionFocusedLearning) ecosystem\n- **Evaluation tools** for comparing algorithm performance\n\n## Benchmark Categories\n\nThe package organizes benchmarks into three main categories based on their problem structure:\n\n### Static Benchmarks (`AbstractBenchmark`)\nSingle-stage optimization problems with no randomness involved:\n- [`ArgmaxBenchmark`](@ref): argmax toy problem\n- [`Argmax2DBenchmark`](@ref): 2D argmax toy problem\n- [`RankingBenchmark`](@ref): ranking problem\n- [`SubsetSelectionBenchmark`](@ref): select optimal subset of items\n- [`PortfolioOptimizationBenchmark`](@ref): portfolio optimization problem\n- [`FixedSizeShortestPathBenchmark`](@ref): find shortest path on grid graphs with fixed size\n- [`WarcraftBenchmark`](@ref): shortest path on image maps\n\n### Stochastic Benchmarks (`AbstractStochasticBenchmark`)  \nSingle-stage optimization problems under uncertainty:\n- [`StochasticVehicleSchedulingBenchmark`](@ref): stochastic vehicle scheduling under delay uncertainty\n\n### Dynamic Benchmarks (`AbstractDynamicBenchmark`)\nMulti-stage sequential decision-making problems:\n- [`DynamicVehicleSchedulingBenchmark`](@ref): multi-stage vehicle scheduling under customer uncertainty\n- [`DynamicAssortmentBenchmark`](@ref): sequential product assortment selection with endogenous uncertainty\n\n## Getting Started\n\nIn a few lines of code, you can create benchmark instances, generate datasets, initialize learning components, and evaluate performance, using the same syntax across all benchmarks:\n\n```julia\nusing DecisionFocusedLearningBenchmarks\n\n# Create a benchmark instance for the argmax problem\nbenchmark = ArgmaxBenchmark()\n\n# Generate training data\ndataset = generate_dataset(benchmark, 100)\n\n# Initialize policy components\nmodel = generate_statistical_model(benchmark)\nmaximizer = generate_maximizer(benchmark)\n\n# Training algorithm you want to use\n# ... your training code here ...\n\n# Evaluate performance\ngap = compute_gap(benchmark, dataset, model, maximizer)\n```\n\nThe only component you need to customize is the training algorithm itself.\n\n## Related Packages\n\nThis package is part of the [JuliaDecisionFocusedLearning](https://github.com/JuliaDecisionFocusedLearning) organization, and built to be compatible with other packages in the ecosystem:\n- **[InferOpt.jl](https://github.com/JuliaDecisionFocusedLearning/InferOpt.jl)**: differentiable optimization layers and losses for decision-focused learning\n- **[DecisionFocusedLearningAlgorithms.jl](https://github.com/JuliaDecisionFocusedLearning/DecisionFocusedLearningAlgorithms.jl)**: collection of generic black-box implementations of decision-focused learning algorithms\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJuliaDecisionFocusedLearning%2FDecisionFocusedLearningBenchmarks.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJuliaDecisionFocusedLearning%2FDecisionFocusedLearningBenchmarks.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJuliaDecisionFocusedLearning%2FDecisionFocusedLearningBenchmarks.jl/lists"}