{"id":31865351,"url":"https://github.com/zuoxingdong/rlbidder","last_synced_at":"2025-10-30T20:35:41.932Z","repository":{"id":318712991,"uuid":"1072651032","full_name":"zuoxingdong/rlbidder","owner":"zuoxingdong","description":"Reinforcement learning auto-bidding library for research and production.","archived":false,"fork":false,"pushed_at":"2025-10-23T13:37:35.000Z","size":1640,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-24T09:03:01.094Z","etag":null,"topics":["ai","auctions","auto-bidding","autobidder","bidding","cql","decision-transformers","imitation-learning","iql","offline-reinforcement-learning","offline-rl","online-advertising","pid","pytorch","pytorch-lightning","reinforcement-learning","rl","torch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zuoxingdong.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-09T03:01:32.000Z","updated_at":"2025-10-23T15:20:35.000Z","dependencies_parsed_at":"2025-10-10T21:19:39.135Z","dependency_job_id":"497fd352-c37a-44ba-bda5-d9d07918a341","html_url":"https://github.com/zuoxingdong/rlbidder","commit_stats":null,"previous_names":["zuoxingdong/rlbidder"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/zuoxingdong/rlbidder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zuoxingdong%2Frlbidder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zuoxingdong%2Frlbidder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zuoxingdong%2Frlbidder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zuoxingdong%2Frlbidder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zuoxingdong","download_url":"https://codeload.github.com/zuoxingdong/rlbidder/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zuoxingdong%2Frlbidder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281271365,"owners_count":26472489,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-27T02:00:05.855Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","auctions","auto-bidding","autobidder","bidding","cql","decision-transformers","imitation-learning","iql","offline-reinforcement-learning","offline-rl","online-advertising","pid","pytorch","pytorch-lightning","reinforcement-learning","rl","torch"],"created_at":"2025-10-12T19:17:18.625Z","updated_at":"2025-10-30T20:35:41.925Z","avatar_url":"https://github.com/zuoxingdong.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RLBidder\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/rlbidder.jpg\" alt=\"rlbidder\" width=\"800px\" style=\"max-width: 65%;\"/\u003e\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\u003cstrong\u003eReinforcement learning auto-bidding library \nfor research and production.\u003c/strong\u003e\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.org/project/rlbidder/\"\u003e\u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/rlbidder.svg\"\u003e\u003c/a\u003e\n  \u003cimg alt=\"Python\" src=\"https://img.shields.io/badge/Python-3.11%2B-blue.svg\"\u003e\n  \u003cimg alt=\"PyTorch\" src=\"https://img.shields.io/badge/PyTorch-2.6%2B-EE4C2C.svg\"\u003e\n  \u003cimg alt=\"Lightning\" src=\"https://img.shields.io/badge/Lightning-2.4%2B-792EE5.svg\"\u003e\n  \u003ca href=\"#quickstart\"\u003e\u003cimg alt=\"Quickstart\" src=\"https://img.shields.io/badge/Quickstart-ready-brightgreen.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#-overview\"\u003eOverview\u003c/a\u003e •\n  \u003ca href=\"#-who-should-use-rlbidder\"\u003eWho Should Use This\u003c/a\u003e •\n  \u003ca href=\"#installation\"\u003eInstallation\u003c/a\u003e •\n  \u003ca href=\"#quickstart\"\u003eQuickstart\u003c/a\u003e •\n  \u003ca href=\"#benchmarking-results\"\u003eBenchmarks\u003c/a\u003e •\n  \u003ca href=\"#-module-guide\"\u003eAPI\u003c/a\u003e •\n  \u003ca href=\"#citation\"\u003eCitation\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n## 📖 Overview\n\n`rlbidder` is a comprehensive toolkit for training and deploying reinforcement learning agents in online advertising auctions. Built for both **industrial scale** and **research agility**, it provides:\n\n- **Complete offline RL pipeline**: Rust-powered data processing (Polars) → SOTA algorithms (IQL, CQL, DT, GAVE) → parallel evaluation\n- **Modern ML infrastructure**: PyTorch Lightning multi-GPU training, experiment tracking, automated reproducibility\n- **Production insights**: Interactive dashboards for campaign monitoring, market analytics, and agent behavior analysis\n- **Research rigor**: Statistically robust benchmarking with RLiable metrics, tuned control baselines, and round-robin evaluation\n\nWhether you're deploying bidding systems at scale or researching novel RL methods, `rlbidder` bridges the gap between academic innovation and production readiness.\n\n---\n\n## 🎯 Who Should Use rlbidder?\n\n**Researchers** looking to experiment with SOTA offline RL algorithms (IQL, CQL, DT, GAVE, GAS) on realistic auction data with rigorous benchmarking.\n\n**AdTech Practitioners** comparing RL agents against classic baselines (PID, BudgetPacer) before production deployment.\n\n---\n\n## 🚀 Key Features \u0026 What Makes rlbidder Different\n\n`rlbidder` pushes beyond conventional RL libraries by integrating cutting-edge techniques from both RL research and modern LLM/transformer architectures. Here's what sets it apart:\n\n### **Rust-Powered Data Pipeline**\n- **Standardized workflow**: Scan Parquet → RL Dataset → Feature Engineering → DT Dataset with reproducible artifacts at every stage\n- **Polars Lazy API**: Streaming data processing with a blazingly fast Rust engine that handles massive datasets without memory overhead\n- **Scalable workflows**: Process 100GB+ auction logs efficiently with lazy evaluation and zero-copy operations\n- **Feature engineering**: Drop-in scikit-learn-style transformers (Symlog, Winsorizer, ReturnScaledReward) for states, actions, and rewards\n\n### **State-of-the-Art RL Algorithms**\n- **Comprehensive baselines**: Classic control (Heuristic, BudgetPacer, PID) and learning-based methods (BC, CQL, IQL, DT, GAVE, GAS)\n- **HL-Gauss Distributional RL**: Smooth Gaussian-based distributional Q-learning for improved uncertainty quantification, advancing beyond standard categorical approaches\n- **Efficient ensemble critics**: Leverage `torch.vmap` for vectorized ensemble operations—much faster than traditional loop-based implementations\n- **Numerically stable stochastic policies**: DreamerV3-style `SigmoidRangeStd` and TorchRL-style `BiasedSoftplus` to avoid numerical instabilities from exp/log operations\n\n### **Modern Transformer Stack (LLM-Grade)**\n- **FlashAttention (SDPA)**: Uses latest PyTorch scaled dot-product attention API for accelerated training\n- **RoPE positional encoding**: Rotary positional embeddings for improved sequence length generalization, adopted from modern LLMs\n- **QK-Norm**: Query-key normalization for enhanced training stability at scale\n- **SwiGLU**: Advanced feed-forward networks for superior expressiveness\n- **Efficient inference**: `DTInferenceBuffer` with deque-based temporal buffering for online Decision Transformer deployment\n\n### **Simulated Online Evaluation \u0026 Visualization**\n- **Parallel evaluation**: Multi-process evaluators with pre-loaded data per worker—much faster than sequential benchmarking\n- **Robust testing**: Round-robin agent rotation with multi-seed evaluation for statistically reliable comparisons\n- **Tuned competitors**: Classic control methods (BudgetPacer, PID) with optimized hyperparameters as baselines\n- **Interactive dashboards**: Production-ready Plotly visualizations with market structure metrics (HHI, Gini, volatility) and RLiable metrics\n- **Industrial analytics**: Campaign health monitoring, budget pacing diagnostics, auction dynamics, and score distribution analysis\n\n### **Modern ML Engineering Stack**\n- **Modular design**: Enables both production readiness and rapid prototyping\n- **PyTorch Lightning**: Reduce boilerplate code, automatic mixed precision, gradient accumulation\n- **Draccus configuration**: Type-safe dataclass-to-CLI with hierarchical configs, dot-notation overrides, and zero boilerplate\n- **Local experiment tracking**: AIM for experiment management without external cloud dependencies\n\n### **Comparison with AuctionNet**\n\n| Feature | AuctionNet | rlbidder |\n|---------|-----------|----------|\n| **Data Engine** | Pandas | **Polars Lazy (Rust)** ✨ |\n| **Configuration** | argparse | **Draccus (dataclass-to-CLI)** ✨ |\n| **Distributional RL** | ❌ | **HL-Gauss** ✨ |\n| **Ensemble Method** | ❌ | **torch.vmap** ✨ |\n| **Transformer Attention** | Standard | **SDPA/FlashAttn** ✨ |\n| **Positional Encoding** | Learned | **RoPE** ✨ |\n| **Policy Stability** | exp(log_std) | **SigmoidRangeStd/BiasedSoftplus** ✨ |\n| **Parallel Evaluation** | ❌ | **ProcessPool + Round-robin** ✨ |\n| **Visualization** | ❌ | **Production Dashboards** ✨ |\n\n---\n\n## 📊 Benchmarking Results\n\nWe evaluate all agents using rigorous statistical methods across multiple delivery periods with round-robin testing and multi-seed evaluation. The evaluation protocol follows RLiable best practices for statistically reliable algorithm comparison.\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50%\" align=\"center\"\u003e\n      \u003cimg src=\"assets/benchmark_violin.png\" alt=\"Benchmark violin plots\" width=\"100%\" /\u003e\n      \u003cp\u003e\u003cstrong\u003eScore Distribution Analysis\u003c/strong\u003e\u003cbr/\u003eViolin plots showing performance distributions across agents and seeds.\u003c/p\u003e\n    \u003c/td\u003e\n    \u003ctd width=\"50%\" align=\"center\"\u003e\n      \u003cimg src=\"assets/benchmark_bar.png\" alt=\"Benchmark bar charts\" width=\"100%\" /\u003e\n      \u003cp\u003e\u003cstrong\u003eMean Performance Comparison\u003c/strong\u003e\u003cbr/\u003eAggregated performance metrics with confidence intervals.\u003c/p\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd colspan=\"2\" align=\"center\"\u003e\n      \u003cimg src=\"assets/rliable.png\" alt=\"RLiable metrics\" width=\"65%\" /\u003e\n      \u003cp\u003e\u003cstrong\u003eRLiable Statistical Metrics\u003c/strong\u003e\u003cbr/\u003ePerformance profiles and aggregate metrics following RLiable best practices.\u003c/p\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n## 📈 Interactive Dashboards \u0026 Gallery\n\nBeyond raw performance metrics, `rlbidder` helps you understand *why* agents behave the way they do. Production-grade interactive dashboards summarize policy behavior, campaign health, and auction dynamics for both research insights and production monitoring.\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50%\" align=\"center\"\u003e\n      \u003cimg src=\"assets/market.png\" alt=\"Auction market analysis\" width=\"100%\" /\u003e\n      \u003cp\u003e\u003cstrong\u003eAuction market analysis\u003c/strong\u003e\u003cbr/\u003eMarket concentration, volatility, and competitiveness.\u003c/p\u003e\n    \u003c/td\u003e\n    \u003ctd width=\"50%\" align=\"center\"\u003e\n      \u003cimg src=\"assets/campaign_cql.png\" alt=\"Campaign analysis for CQL\" width=\"100%\" /\u003e\n      \u003cp\u003e\u003cstrong\u003eCampaign analysis (CQL)\u003c/strong\u003e\u003cbr/\u003eSegment-level delivery quality and conversion outcomes.\u003c/p\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50%\" align=\"center\"\u003e\n      \u003cimg src=\"assets/budgetpace_cql.png\" alt=\"Budget pacing for CQL\" width=\"100%\" /\u003e\n      \u003cp\u003e\u003cstrong\u003eBudget pacing (CQL)\u003c/strong\u003e\u003cbr/\u003eDaily spend pacing and CPA stabilization diagnostics.\u003c/p\u003e\n    \u003c/td\u003e\n    \u003ctd width=\"50%\" align=\"center\"\u003e\n      \u003cimg src=\"assets/scatter.png\" alt=\"Auction metrics scatterplots\" width=\"100%\" /\u003e\n      \u003cp\u003e\u003cstrong\u003eAuction metrics scatterplots\u003c/strong\u003e\u003cbr/\u003eSpend, conversion, ROI, and win-rate trade-offs.\u003c/p\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\n---\n\n## 🚀 Getting Started\n\n### Installation\n\n#### Prerequisites\n\n- Python 3.11 or newer\n- PyTorch 2.6 or newer (follow [PyTorch install guide](https://pytorch.org/get-started/locally/))\n- GPU with 8GB+ vRAM recommended for training\n\n#### Local Development\n\n```bash\ngit clone https://github.com/zuoxingdong/rlbidder.git\ncd rlbidder\npip install -e .\n```\n\n---\n\n### Quickstart\n\nFollow the steps below to reproduce the full offline RL workflow on processed campaign data. \n\n#### Step 1: Data Preparation\n\n```bash\n# Download sample competition data (periods 7-8 and trajectory 1)\nbash scripts/download_raw_data.sh -p 7-8,traj1 -d data/raw\n\n# Convert raw CSV to Parquet (faster I/O with Polars)\npython scripts/convert_csv_to_parquet.py --raw_data_dir=data/raw\n\n# Build evaluation-period parquet files\npython scripts/build_eval_dataset.py --data_dir=data\n\n# Create training transitions (trajectory format for offline RL)\npython scripts/build_transition_dataset.py --data_dir=data --mode=trajectory\n\n# Fit scalers for state, action, and reward normalization\npython scripts/scale_transitions.py --data_dir=data --output_dir=scaled_transitions\n\n# Generate Decision Transformer trajectories with return-to-go\npython scripts/build_dt_dataset.py \\\n  --build.data_dir=data \\\n  --build.reward_type=reward_dense \\\n  --build.use_scaled_reward=true\n```\n\n**What you'll have:** Preprocessed datasets in `data/processed/` and fitted scalers in `data/scaled_transitions/` ready for training.\n\n#### Step 2: Train Agents\n\n```bash\n# Train IQL (Implicit Q-Learning) - value-based offline RL\npython examples/train_iql.py \\\n  --model_cfg.lr_actor 3e-4 \\\n  --model_cfg.lr_critic 3e-4 \\\n  --model_cfg.num_q_models 5 \\\n  --model_cfg.bc_alpha 0.01 \\\n  --train_cfg.enable_aim_logger=False\n\n# Train DT (Decision Transformer) - sequence modeling for RL\npython examples/train_dt.py \\\n  --model_cfg.embedding_dim 512 \\\n  --model_cfg.num_layers 6 \\\n  --model_cfg.lr 1e-4 \\\n  --model_cfg.rtg_scale 98 \\\n  --model_cfg.target_rtg 2.0 \\\n  --train_cfg.enable_aim_logger=False\n```\n\n**What you'll have:** Trained model checkpoints in `examples/checkpoints/` with scalers and hyperparameters.\n\n**💡 Configuration powered by draccus:** All training scripts use type-safe dataclass configs with automatic CLI generation. Override any nested config with dot-notation (e.g., `--model_cfg.lr 1e-4`) or pass config files directly.\n\n**💡 Track experiments with Aim:** All training scripts automatically log metrics, hyperparameters, and model artifacts to Aim (a local experiment tracker). \nTo use Aim, first initialize your project with:\n\n```bash\naim init\n```\n\nThen launch the Aim UI to visualize training progress:\n\n```bash\naim up --port 43800\n```\n\nThen open `http://localhost:43800` in your browser to explore training curves, compare runs, and analyze hyperparameter configurations.\n\n#### Step 3: Evaluate in Simulated Auctions\n\n```bash\n# Evaluate IQL agent with parallel multi-seed evaluation\npython examples/evaluate_agents.py \\\n  --evaluation.data_dir=data \\\n  --evaluation.evaluator_type=OnlineCampaignEvaluator \\\n  --evaluation.delivery_period_indices=[7,8] \\\n  --evaluation.num_seeds=5 \\\n  --evaluation.num_workers=8 \\\n  --evaluation.output_dir=examples/eval \\\n  --agent.agent_class=IQLBiddingAgent \\\n  --agent.model_dir=examples/checkpoints/iql \\\n  --agent.checkpoint_file=best.ckpt\n```\n\n**What you'll have:** Evaluation reports, campaign summaries, and auction histories in `examples/eval/` ready for visualization.\n\n**Next steps:** Generate dashboards with `examples/performance_visualization.ipynb` or explore the evaluation results with Polars DataFrames.\n\n---\n\n## 📦 Module Guide\n\nEach module handles a specific aspect of the RL bidding pipeline:\n\n| Module | Description | Key Classes/Functions |\n| --- | --- | --- |\n| 📚 `rlbidder.agents` | Offline RL agents and control baselines | `IQLModel`, `CQLModel`, `DTModel`, `GAVEModel`, `BudgetPacerBiddingAgent` |\n| 🔧 `rlbidder.data` | Data processing, scalers, and datasets | `OfflineDataModule`, `TrajDataset`, `SymlogTransformer`, `WinsorizerTransformer` |\n| 🏪 `rlbidder.envs` | Auction simulation and value sampling | `OnlineAuctionEnv`, `ValueSampler`, `sample_conversions` |\n| 🎯 `rlbidder.evaluation` | Multi-agent evaluation and metrics | `ParallelOnlineCampaignEvaluator`, `OnlineCampaignEvaluator` |\n| 🧠 `rlbidder.models` | Neural network building blocks | `StochasticActor`, `EnsembledQNetwork`, `NormalHead`, `HLGaussLoss` |\n| 📊 `rlbidder.viz` | Interactive dashboards and analytics | `create_campaign_dashboard`, `create_market_dashboard`, `plot_rliable_metrics` |\n| 🛠️ `rlbidder.utils` | Utilities and helpers | `set_seed`, `log_distribution`, `regression_report` |\n\n---\n\n## 🏗️ Architecture\n\nThe library follows a modular design with clear separation of concerns. Data flows from raw logs through preprocessing, training, and evaluation to final visualization:\n\n```mermaid\nflowchart TD\n    subgraph Data[\"📦 Data Pipeline\"]\n        direction TB\n        raw[\"Raw Campaign Data\u003cbr/\u003e\u003ci\u003eCSV/Parquet logs\u003c/i\u003e\"]\n        scripts[\"Build Scripts\u003cbr/\u003econvert • build_eval\u003cbr/\u003ebuild_transition • scale\"]\n        artifacts[\"📁 Preprocessed Artifacts\u003cbr/\u003eprocessed/ • scaled_transitions/\u003cbr/\u003e\u003ci\u003eParquet + Scalers\u003c/i\u003e\"]\n        \n        raw --\u003e|transform| scripts\n        scripts --\u003e|generate| artifacts\n    end\n\n    subgraph Core[\"⚙️ Core Library Modules\"]\n        direction TB\n        data_mod[\"\u003cb\u003erlbidder.data\u003c/b\u003e\u003cbr/\u003eOfflineDataModule\u003cbr/\u003eTrajDataset • ReplayBuffer\u003cbr/\u003e🔧 \u003ci\u003eHandles batching \u0026 scaling\u003c/i\u003e\"]\n        models[\"\u003cb\u003erlbidder.models\u003c/b\u003e\u003cbr/\u003eStochasticActor • EnsembledQNetwork\u003cbr/\u003eValueNetwork • Losses • Optimizers\u003cbr/\u003e🧠 \u003ci\u003eAgent building blocks\u003c/i\u003e\"]\n        agents[\"\u003cb\u003erlbidder.agents\u003c/b\u003e\u003cbr/\u003eIQLModel • CQLModel • DTModel\u003cbr/\u003e📚 \u003ci\u003eLightningModule implementations\u003c/i\u003e\"]\n        \n        agents --\u003e|composes| models\n    end\n\n    subgraph Training[\"🔥 Training Pipeline\"]\n        direction TB\n        train[\"\u003cb\u003eexamples/train_iql.py\u003c/b\u003e\u003cbr/\u003e🎛️ Config + CLI\u003cbr/\u003e\u003ci\u003eOrchestration script\u003c/i\u003e\"]\n        trainer[\"⚡ Lightning Trainer\u003cbr/\u003efit() • validate()\u003cbr/\u003e\u003ci\u003eMulti-GPU support\u003c/i\u003e\"]\n        ckpt[\"💾 Model Checkpoints\u003cbr/\u003ebest.ckpt • last.ckpt\u003cbr/\u003e\u003ci\u003e+ scalers + hparams\u003c/i\u003e\"]\n        \n        train --\u003e|instantiates| data_mod\n        train --\u003e|instantiates| agents\n        train --\u003e|launches| trainer\n        trainer --\u003e|saves| ckpt\n    end\n\n    subgraph Eval[\"🎯 Online Evaluation\"]\n        direction TB\n        evaluator[\"\u003cb\u003erlbidder.evaluation\u003c/b\u003e\u003cbr/\u003eOnlineCampaignEvaluator\u003cbr/\u003eParallelEvaluator\u003cbr/\u003e🔄 \u003ci\u003eMulti-seed, round-robin\u003c/i\u003e\"]\n        env[\"\u003cb\u003erlbidder.envs\u003c/b\u003e\u003cbr/\u003eAuction Simulator\u003cbr/\u003e🏪 \u003ci\u003eMulti-agent market\u003c/i\u003e\"]\n        results[\"📈 Evaluation Results\u003cbr/\u003eCampaign Reports • Agent Summaries\u003cbr/\u003eAuction Histories\u003cbr/\u003e\u003ci\u003ePolars DataFrames\u003c/i\u003e\"]\n        \n        evaluator --\u003e|simulates| env\n        env --\u003e|produces| results\n    end\n\n    subgraph Viz[\"📊 Visualization \u0026 Analysis\"]\n        direction TB\n        viz[\"\u003cb\u003erlbidder.viz\u003c/b\u003e\u003cbr/\u003ePlotly Dashboards\u003cbr/\u003eMarket Metrics\u003cbr/\u003e🎨 \u003ci\u003eInteractive HTML\u003c/i\u003e\"]\n        plots[\"📉 Production Dashboards\u003cbr/\u003eCampaign Health • Market Structure\u003cbr/\u003eBudget Pacing • Scatter Analysis\"]\n        \n        viz --\u003e|renders| plots\n    end\n\n    artifacts ==\u003e|loads| data_mod\n    artifacts -.-\u003e|eval data| evaluator\n    ckpt ==\u003e|load_from_checkpoint| evaluator\n    results ==\u003e|consumes| viz\n\n    classDef dataStyle fill:#1565c0,stroke:#0d47a1,stroke-width:3px,color:#fff,font-weight:bold\n    classDef coreStyle fill:#ef6c00,stroke:#e65100,stroke-width:3px,color:#fff,font-weight:bold\n    classDef trainStyle fill:#6a1b9a,stroke:#4a148c,stroke-width:3px,color:#fff,font-weight:bold\n    classDef evalStyle fill:#2e7d32,stroke:#1b5e20,stroke-width:3px,color:#fff,font-weight:bold\n    classDef vizStyle fill:#c2185b,stroke:#880e4f,stroke-width:3px,color:#fff,font-weight:bold\n    \n    class Data,raw,scripts,artifacts dataStyle\n    class Core,data_mod,models,agents coreStyle\n    class Training,train,trainer,ckpt trainStyle\n    class Eval,evaluator,env,results evalStyle\n    class Viz,viz,plots vizStyle\n```\n\n**Design Principles:**\n- 🔌 **Modular** - Each component is independently usable and testable\n- ⚡ **Scalable** - Polars + Lightning enable massive datasets and efficient training\n- 🔄 **Reproducible** - Deterministic seeding, configuration management, and evaluation\n- 🚀 **Production-ready** - Type hints, error handling, logging, and monitoring built-in\n\n---\n\n## 🤝 Contributing\n\n- 🌟 Star the repo if you find it useful\n- 🔀 Fork and submit PRs for bug fixes or new features\n- 📝 Improve documentation and add examples\n- 🧪 Add tests for new functionality\n\n\n---\n\n## 🌟 Acknowledgments\n\n`rlbidder` builds upon ideas from:\n\n- **[AuctionNet](https://github.com/alimama-tech/AuctionNet)** original pioneer, for auction environment and benchmark design\n- **[PyTorch Lightning](https://github.com/Lightning-AI/pytorch-lightning)** for training infrastructure\n- **[Draccus](https://github.com/dlwh/draccus)** for elegant dataclass-to-CLI configuration management\n- **[TRL](https://github.com/huggingface/trl)** \u0026 **[Transformers](https://github.com/huggingface/transformers)** for modern transformer implementations\n- **[Polars](https://github.com/pola-rs/polars)** for high-performance data processing\n- **[PyTorch RL](https://github.com/pytorch/rl)** for RL algorithm\n\n---\n\n## 📝 Citation\n\nIf you use `rlbidder` in your work, please cite it using the BibTeX entry below.\n\n```bibtex\n@misc{zuo2025rlbidder,\n  author = {Zuo, Xingdong},\n  title = {RLBidder: Reinforcement learning auto-bidding library for research and production},\n  year = {2025},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/zuoxingdong/rlbidder}}\n}\n```\n\n---\n\n## License\n\nMIT License. See `LICENSE`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzuoxingdong%2Frlbidder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzuoxingdong%2Frlbidder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzuoxingdong%2Frlbidder/lists"}