{"id":13440215,"url":"https://github.com/LumingSun/ML4DB-paper-list","last_synced_at":"2025-03-20T09:32:29.954Z","repository":{"id":39756637,"uuid":"207538731","full_name":"LumingSun/ML4DB-paper-list","owner":"LumingSun","description":"Papers for database systems powered by artificial intelligence (machine learning for database)","archived":false,"fork":false,"pushed_at":"2024-09-09T02:22:43.000Z","size":1244,"stargazers_count":624,"open_issues_count":0,"forks_count":82,"subscribers_count":47,"default_branch":"master","last_synced_at":"2024-09-09T03:39:23.063Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LumingSun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-10T11:16:49.000Z","updated_at":"2024-09-09T02:22:47.000Z","dependencies_parsed_at":"2023-02-18T14:46:14.481Z","dependency_job_id":"61286b43-45ac-4a51-9fca-b996826ff843","html_url":"https://github.com/LumingSun/ML4DB-paper-list","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LumingSun%2FML4DB-paper-list","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LumingSun%2FML4DB-paper-list/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LumingSun%2FML4DB-paper-list/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LumingSun%2FML4DB-paper-list/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LumingSun","download_url":"https://codeload.github.com/LumingSun/ML4DB-paper-list/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221745274,"owners_count":16873743,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:20.777Z","updated_at":"2025-03-20T09:32:29.936Z","avatar_url":"https://github.com/LumingSun.png","language":null,"funding_links":[],"categories":["HarmonyOS","Others","Table of Contents"],"sub_categories":["Windows Manager"],"readme":"\n# [Paper List] AI4DB / ML4DB / Autonomous Database / Self-driving Database / 智能数据库 / 自治数据库\n\nPaper list for database systems with artificial intelligence (machine learning, deep learning, reinforcement learning)\n\nNew papers keep coming, remember to **Watch** this repo if you are interested in this topic.\n\n有关机器学习、神经网络、强化学习、自调优技术等在数据库系统中的应用的文章列表，列表持续更新中，记得按赞、分享、打开小铃铛！\n\nWelcome to PR!\n\n欢迎大家补充！\n\nThere are so many papers emerging about [Text-To-SQL](https://github.com/eosphoros-ai/Awesome-Text2SQL)! Sadly I'm not an expert with the topic and can not tell the quality of the papers.  \nLooking forward to contributions (PR, comment, discussion) about Text-To-SQL！🫶\n\nTable of Contents\n=================\n* [System \u0026 Tutorial](#system-and-tutorial)\n  * [Training data](#training-data)\n* [Data Access](#data-access)\n  * [Configuration Tuning](#configuration-tuning)\n  * [Physical Design](#physical-design)\n    * [Learned structure](#learned-structure)\n    * [Index](#index)\n      * [Index Structure](#index-structure)\n      * [LSM-tree related](#lsm-tree-related)\n      * [Index Recommendation](#index-recommendation)\n    * [Materialized View](#materialized-view)\n    * [Schema \u0026amp; Partition](#schema--partition)\n      * [Offline](#offline)\n      * [Online](#online)\n  * [Cache related](#cache-related)\n* [Workload](#workload)\n    * [Resource Estimation and Auto-scaling](#resource-estimation-and-auto-scaling)\n    * [Performance Diagnosis and Modeling](#performance-diagnosis-and-modeling)\n    * [Workload Shift Detection](#workload-shift-detection)\n    * [Metrics Prediction for Queries](#metrics-prediction-for-queries)\n    * [Workload Characterization \u0026 Forecasting](#workload-characterization-\u0026-forecasting)\n* [Query Optimization](#query-optimization)\n   * [Query Rewrite](#query-write)\n   * [Cardinality Estimation](#cardinality-estimation)\n     * [Data-based](#data-based)\n     * [Query-based](#query-based)\n  * [Cost Estimation](#cost-estimation)\n    * [Single Query](#single-query)\n    * [Concurrent](#Concurrent)\n  * [Join Optimization](#join-optimization)\n  * [Query Plan](#query-plan)\n* [Query Execution](#query-execution)\n  * [Sort](#sort)\n  * [Join](#join)\n  * [Adaptive Query Processing](#adaptive-query-processing)\n  * [Approximate Query Processing](#Approximate-query-processing)\n  * [Sheduling](#sheduling)\n* [Text-to-SQL](#text-to-sql)\n* [SQL Related](#sql-related)\n\n\n## System and Tutorial\n* ***SageDB: A Learned Database System (CIDR 2019)***\n* Database Learning: Toward a Database that Becomes Smarter Every Time (SIGMOD 2017)\n* Self-Driving Database Management Systems (CIDR 2017)\n* Self-Driving : From General Purpose to Specialized DBMSs (Phd@PVLDB 2018)  \n* Active Learning for ML Enhanced Database Systems (SIGMOD 2020)\n* Database Meets Artificial Intelligence: A Survey (TKDE 2020)\n* Self-driving database systems: a conceptual approach (Distributed and Parallel Databases 2020)\n* One Model to Rule them All: Towards Zero-Shot Learning for Databases (arXiv 2021)\n* UDO: Universal Database Optimization using Reinforcement Learning (arXiv 2021) [Source Code](https://github.com/jxiw/UDO)\n* Towards a Benchmark for Learned Systems (SMDB workshop 2021)\n* A Unified Transferable Model for ML-Enhanced DBMS [Vision] (arXiv 2021)\n* AI Meets Database: AI4DB and DB4AI (SIGMOD 2021)\n* Expand your Training Limits! Generating Training Data for ML-based Data Management (SIGMOD 2021)\n* MB2: Decomposed Behavior Modeling for Self-Driving Database Management Systems (SIGMOD 2021)\n* Towards instance-optimized data systems (VLDB 2021 from Tim Kraska)\n* Make Your Database System Dream of Electric Sheep: Towards Self-Driving Operation (VLDB 2021 from Andy Pavlo)\n* openGauss: An Autonomous Database System (VLDB 2021 from Guoliang Li)\n* Experience-Enhanced Learning: One Size Still does not Fit All in Automatic Database Management (arXiv 2021)\n* Baihe: SysML Framework for AI-driven Databases (arXiv 2022)\n* Survey on Learnable Databases: A Machine Learning Perspective (Big Data Research 2021)\n* Database Optimizers in the Era of Learning (ICDE 2022)\n* Machine Learning for Data Management: A System View (ICDE 2022)\n* Tastes Great! Less Filling! High Performance and Accurate Training Data Collection for Self-Driving Database Management Systems (SIGMOD 2022)\n* SAM: Database Generation from Query Workload with Supervised Autoregressive Model (SIGMOD 2022) [Source code](https://github.com/Jamesyang2333/SAM)\n* Detect, Distill and Update: Learned DB Systems Facing Out of Distribution Data (SIGMOD 2023) [Source code](https://github.com/meghdadk/DDUp)\n* SageDB: An Instance-Optimized Data Analytics System (VLDB 2023)\n* Towards Building Autonomous Data Services on Azure (SIGMOD-Companion ’23)\n* Database Gyms (CIDR 2023)\n* Check Out the Big Brain on BRAD: Simplifying Cloud Data Processing with Learned Automated Data Meshes (VLDB 2023)\n* Machine Unlearning in Learned Databases: An Experimental Analysis (SIGMOD 2024) [Source code](https://github.com/meghdadk/DB_unlearning)\n* PilotScope: Steering Databases with Machine Learning Drivers (VLDB 2024) [Source code](https://github.com/alibaba/pilotscope)\n* Machine Learning for Databases: Foundations, Paradigms, and Open problems (SIGMOD 2024)\n* NeurDB: An AI-powered Autonomous Data System (arXiv 2024)\n* GaussML: An End-to-End In-Database Machine Learning System (ICDE 2024)\n* NeurDB: On the Design and Implementation of an AI-powered Autonomous Database (arXiv 2024)\n* LLM for Data Management (VLDB 2024)\n* Blueprinting the Cloud: Unifying and Automatically Optimizing Cloud Data Infrastructures with BRAD (VLDB 2024)\n* The Holon Approach for Simultaneously Tuning Multiple Components in a Self-Driving Database Management System with Machine Learning via Synthesized Proto-Actions (VLDB 2024)\n### Training Data Collection\n* Expand your Training Limits! Generating Training Data for ML-based Data Management (SIGMOD 2021)\n* DataFarm: Farm Your ML-based Query Optimizer's Food! - Human-Guided Training Data Generation -. (CIDR 2022)\n* Farming Your ML-based Query Optimizer's Food. (ICDE 2022, **best demo award**)\n* Hit the Gym: Accelerating Query Execution to Efficiently Bootstrap Behavior Models for Self-Driving Database Management Systems (VLDB 2024)\n* Theoretical Analysis of Learned Database Operations under Distribution Shift through Distribution Learnability (ICML 2024)\n\n## Data Access\n### Configuration Tuning\n* SARD: A statistical approach for ranking database tuning parameters (ICDEW, 2008)\n* Regularized Cost-Model Oblivious Database Tuning with Reinforcement Learning （2016）\n* Automatic Database Management System Tuning Through Large-scale Machine Learning (SIGMOD 2017)\n* The Case for Automatic Database Administration using Deep Reinforcement Learning ( 2018 ArXiv)\n* An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning (SIGMOD 2019)\n* External vs. Internal : An Essay on Machine Learning Agents for Autonomous Database Management Systems\n* QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning (VLDB 2019)\n* Optimizing Databases by Learning Hidden Parameters of Solid State Drives (VLDB 2019)\n* iBTune: Individualized Buffer Tuning for Large-scale Cloud Databases (VLDB 2019)\n* Black or White? How to Develop an AutoTuner for Memory-based Analytics (SIGMOD 2020)\n* Learning Efficient Parameter Server Synchronization Policies for Distributed SGD (ICLR 2020)\n* Too Many Knobs to Tune? Towards Faster Database Tuning by Pre-selecting Important Knobs (HotStorage 2020)\n* Dynamic Configuration Tuning of Working Database Management Systems (LifeTech 2020)\n* Adaptive Multi-Model Reinforcement Learning for Online Database Tuning (EDBT 2021)\n* An inquiry into machine learning-based automatic configuration tuning services on real-world database management systems (VLDB 2021)\n* The Case for NLP-Enhanced Database Tuning: Towards Tuning Tools that \"Read the Manual\" (VLDB 2021)\n* CGPTuner: a Contextual Gaussian Process Bandit Approach for the Automatic Tuning of IT Configurations Under Varying Workload Conditions (VLDB 2021)\n* ResTune: Resource Oriented Tuning Boosted by Meta-Learning for Cloud Databases (SIGMOD 2021)\n* KML: Using Machine Learning to Improve Storage Systems (arXiv 2021)\n* Database Tuning using Natural Language Processing (SIGMOD Record 2021)\n* Towards Dynamic and Safe Configuration Tuning for Cloud Databases (SIGMOD 2022)\n* Automatic Performance Tuning for Distributed Data Stream Processing Systems (ICDE 2022)\n* Adaptive Code Learning for Spark Configuration Tuning (ICDE 2022)\n* DB-BERT: A Database Tuning Tool that \"Reads the Manual\" (SIGMOD 2022)\n* HUNTER: An Online Cloud Database Hybrid Tuning System for Personalized Requirements (SIGMOD 2022)\n* LOCAT: Low-Overhead Online Configuration Auto-Tuning of Spark SQL Applications (SIGMOD 2022)\n* Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation (VLDB 2022)\n* LlamaTune: Sample-Efficient DBMS Configuration Tuning (VLDB 2022)\n* BLUTune: Query-informed Multi-stage IBM Db2 Tuning via ML (CIKM 2022)\n* A Unified and Efficient Coordinating Framework for Autonomous DBMS Tuning (arXiv 2023)\n* Automatic Database Knob Tuning: A Survey (TKDE)\n* Deep learning based Auto Tuning for Database Management System (arXiv 2023)\n* KeenTune: Automated Tuning Tool for Cloud Application Performance Testing and Optimization (ISSTA 2023)\n* ContTune: Continuous Tuning by Conservative Bayesian Optimization for Distributed Stream Data Processing Systems (arXiv 2023)\n* GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization (arXiv 2023)\n* An Eficient Transfer Learning Based Configuration Adviser for Database Tuning (VLDB 2024)\n* DB‑GPT: Large Language Model Meets Database (DSE 2024)\n* A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning (arXiv 2024)\n* TIE: Fast Experiment-driven ML-based Configuration Tuning for In-memory Data Analytics (IEEE Transactions on Computers)\n* VDTuner: Automated Performance Tuning for Vector Data Management Systems (ICDE 2024) [Source code](https://github.com/tiannuo-yang/VDTuner)\n* Nautilus: A Benchmarking Platform for DBMS Knob Tuning (DEEM 2024) [Source code](https://github.com/uw-mad-dash/nautilus)\n* Is Large Language Model Good at Database Knob Tuning? A Comprehensive Experimental Evaluation (arXiv 2024)\n* CTuner: Automatic NoSQL Database Tuning with Causal Reinforcement Learning (Internetware 2024)\n* KnobTree: Intelligent Database Parameter Configuration via Explainable Reinforcement Learning (arXiv 2024)\n* KnobCF: Uncertainty-aware Knob Tuning (arXiv 2024)\n* Db2une: Tuning Under Pressure via Deep Learning (VLDB 2024)\n* {\\lambda}-Tune: Harnessing Large Language Models for Automated Database System Tuning (arXiv 2024)\n* Db2une: Tuning Under Pressure via Deep Learning (VLDB 2024)\n\n### Physical Design\n* Tiresias: Enabling Predictive Autonomous Storage and Indexing (VLDB 2022)\n#### Learned structure\n* Stacked Filters: Learning to Filter by Structure (VLDB 2021)\n* LEA: A Learned Encoding Advisor for Column Stores (aiDM 2021)\n* Learning over Sets for Databases (EDBT 2024)\n#### Index\n##### Index Structure\n* Learning to hash for indexing big data - A survey (2016)\n* The Case for Learned Index Structures (SIGMOD 2018)\n* A-Tree: A Bounded Approximate Index Structure (2017)\n* FITing-Tree: A Data-aware Index Structure (SIGMOD 2019)\n* Learned Indexes for Dynamic Workloads (2019)\n* SOSD: A Benchmark for Learned Indexes (2019)\n* Learning Multi-dimensional Indexes (2019)\n* ALEX: An Updatable Adaptive Learned Index (SIGMOD 2020)\n* Effectively Learning Spatial Indices (VLDB 2020) [GitHub Link](https://github.com/Liuguanli/RSMI)\n* Stable Learned Bloom Filters for Data Streams (VLDB 2020)\n* START — Self-Tuning Adaptive Radix Tree (ICDEW 2020)\n* Learned Data Structures (2020)\n* RadixSpline: a single-pass learned index (aiDM2020)\n* The ML-Index: A Multidimensional, Learned Index for Point, Range, and Nearest-Neighbor Queries (EDBT 2020)\n* The PGM-index: a fully-dynamic compressed learned index with provable worst-case bounds (VLDB 2020)\n* A Tutorial on Learned Multi-dimensional Indexes (SIGSPATIAL 2020)\n* Why Are Learned Indexes So Effective? (ICML 2020)\n* Learned Indexes for a Google-scale Disk-based Database (arXiv 2020)\n* SIndex: A Scalable Learned Index for String Keys （APSys 2020)\n* XIndex: A Scalable Learned Index for Multicore Data Storage （PPoPP 2020)\n* Tsunami: A Learned Multi-dimensional Index for Correlated Data and Skewed Workloads (VLDB 2021)\n* A Lazy Approach for Efficient Index Learning (2021)\n* The RLR-Tree: A Reinforcement Learning Based R-Tree for Spatial Data (arXiv 2021)\n* Spatial Interpolation-based Learned Index for Range and kNN Queries (arXiv 2021)\n* APEX: A High-Performance Learned Index on Persistent Memory (arXiv 2021)\n* RUSLI: Real-time Updatable Spline Learned Index (aiDM 2021)\n* PLEX: Towards Practical Learned Indexing (arXiv 2021)\n* SPRIG: A Learned Spatial Index for Range and kNN Queries (SSTD 2021)\n* Benchmarking Learned Indexes (VLDB 2021)\n* Updatable Learned Index with Precise Positions (VLDB 2021)\n* The Case for Learned In-Memory Joins (arXiv 2021)\n* Bounding the Last Mile: Efficient Learned String Indexing (arXiv 2021)\n* FINEdex: A Fine-grained Learned Index Scheme for Scalable and Concurrent Memory Systems (VLDB 2022)\n* The next 50 Years in Database Indexing or: The Case for Automatically Generated Index Structures (VLDB 2022)\n* The Concurrent Learned Indexes for Multicore Data Storage (Transactions on Storage 2022)\n* TONE: cutting tail-latency in learned indexes (CHEOPS 22)\n* A Learned Index for Exact Similarity Search in Metric Spaces (ArXiv 2022)\n* RW-tree: A Learned Workload-aware Framework for R-tree Construction (ICDE 2022)\n* The \"AI+R\"-tree: An Instance-optimized R-tree (MDM 2022)\n* LHI: A Learned Hamming Space Index Framework for Efficient Similarity Search (SIGMOD 2022)\n* Entropy Learned Hashing: 10X Faster Hashing with Controllable Uniformity (SIGMOD 2022)\n* Tuning Hierarchical Learned Indexes on Disk and Beyond (SIGMOD 2022)\n* FLIRT: A Fast Learned Index for Rolling Time frames (EDBT 2022)\n* Testing the Robustness of Learned Index Structures (arXiv 2022)\n* The Case for ML-Enhanced High-Dimensional Indexes (2022)\n* A Learned Index for Exact Similarity Search in Metric Spaces (arxiv 2022)\n* PLIN: A Persistent Learned Index for Non-Volatile Memory with High Performance and Instant Recovery (VLDB 2023)\n* A Data-aware Learned Index Scheme for Efficient Writes (ICPP 2022)\n* Frequency Estimation in Data Streams: Learning the Optimal Hashing Scheme (TKDE)\n* FILM: A Fully Learned Index for Larger-Than-Memory Databases (VLDB 2023)\n* WISK: A Workload-aware Learned Index for Spatial Keyword Queries (arXiv 2023)\n* Efficiently Learning Spatial Indices (ICDE 2023)\n* Cutting Learned Index into Pieces: An In-depth Inquiry into Updatable Learned Indexes (ICDE 2023)\n* DILI: A Distribution-Driven Learned Index (arXiv 2023)\n* Learned Index: A Comprehensive Experimental Evaluation (VLDB 2023)\n* LMSFC: A Novel Multidimensional Index based on Learned Monotonic Space Filling Curves (Extended Version) (arXiv 2023)\n* One stone, two birds: A lightweight multidimensional learned index with cardinality support (arXiv 2023)\n* A Simple Yet High-Performing On-disk Learned Index: Can We Have Our Cake and Eat it Too? (aiXiv 2023)\n* Fast Partitioned Learned Bloom Filter (arXiv 2023)\n* Efficient Index Learning via Model Reuse and Fine-tuning (ICDEW 2023)\n* COAX: Correlation-Aware Indexing (ICDEW 2023)\n* Learned Index with Dynamic e (openreview 2023)\n* Learning to Optimize LSM-trees: Towards A Reinforcement Learning based Key-Value Store for Dynamic Workloads (arXiv 2023)\n* SALI: A Scalable Adaptive Learned Index Framework based on Probability Models (SIGMODE 2024)\n* Sieve: A Learned Data-Skipping Index for Data Analytics (VLDB 2023)\n* Demonstrating Waffle: A Self-driving Grid Index (VLDB Demo 2023)\n* Can LSH (Locality-Sensitive Hashing) Be Replaced by Neural Network? (arXiv 2023)\n* Workload-aware and Learned Z-Indexes (arXiv 2023)\n* AirIndex: Versatile Index Tuning Through Data and Storage (SIGMOD 2024)\n* A Fast Learned Key-Value Store for Concurrent and Distributed Systems (TKDE 2023)\n* When Learned Indexes Meet Persistent Memory: The Analysis and the Optimization (TKDE 2023)\n* PLATON: Top-down R-tree Packing with Learned Partition Policy (PACMMOD 2023)\n* A Learned Cuckoo Filter for Approximate Membership Queries over Variable-sized Sliding Windows on Data Streams (PACMMOD 2023)\n* WIPE: a Write-Optimized Learned Index for Persistent Memory (TACO 2023)\n* Algorithmic Complexity Attacks on Dynamic Learned Indexes (VLDB 2024)\n* A Fully On-disk Updatable Learned Index (ICDE 2024)\n* Limousine: Blending Learned and Classical Indexes to Self-Design Larger-than-Memory Cloud Storage Engines (SIGMOD 2024)\n* AStore: Uniformed Adaptive Learned Index and Cache for RDMA-enabled Key-Value Store (TKDE 2024)\n* Cabin: A Compressed Adaptive Binned Scan Index (SIGMOD 2024)\n* SWIX: A Memory-efficient Sliding Window Learned Index (SIGMOD 2024)\n* Limousine: Blending Learned and Classical Indexes to Self-Design Larger-than-Memory Cloud Storage Engines (SIGMOD 2024)\n* A Survey of Learned Indexes for the Multi-dimensional Space (arXiv 2024)\n* Hyper: A High-Performance and Memory-Efficient Learned Index via Hybrid Construction (Proceedings of the ACM on Management of Data 2024)\n* Predicate caching: Query-driven secondary indexing for cloud data warehouses (SIGMOD 2024)\n* AStore: Uniformed Adaptive Learned Index and Cache for RDMA-Enabled Key-Value Store (TKDE 2024)\n* Can Learned Indexes be Built Efficiently? A Deep Dive into Sampling Trade-offs (SIGMOD 2024)\n* Making In-Memory Learned Indexes Efficient on Disk (SIGMOD 2024)\n* LeaderKV: Improving Read Performance of KV Stores via Learned Index and Decoupled KV Table (ICDE 2024)\n* Chameleon: Towards Update-Efficient Learned Indexing for Locally Skewed Data (ICDE 2024)\n* Revisiting Learned Index with Byte-addressable Persistent Storage (ICPP 2024)\n* UpLIF: An Updatable Self-Tuning Learned Index Framework (arXiv 2024)\n* LITS: An Optimized Learned Index for Strings (VLDB 2024)\n* Evaluating Learned Indexes for External-Memory Joins (arXiv 2024)\n* Learned Indexes with Distribution Smoothing via Virtual Points (arXiv 2024)\n* VEGA: An Active-tuning Learned Index with Group-Wise Learning Granularity (SIGMOD 2025)\n##### LSM-tree related\n* Leaper: A Learned Prefetcher for Cache Invalidation in LSM-tree based Storage Engines （VLDB 2020）\n* From WiscKey to Bourbon: A Learned Index for Log-Structured Merge Trees (OSDI 2020)\n* TridentKV: A Read-Optimized LSM-Tree Based KV Store via Adaptive Indexing and Space-Efficient Partitioning (TPDS 2022)\n* LearnedKV: Integrating LSM and Learned Index for Superior Performance on SSD (arXiv 2024)\n* CAMAL: Optimizing LSM-trees via Active Learning (arXiv 2024)\n* DobLIX: A Dual-Objective Learned Index for Log-Structured Merge Trees (arXiv 2025)\n##### Index Recommendation\n* Index Selection in a Self- Adaptive Data Base Management System （SIGMOD 1976）\n* AutoAdmin 'What-if' Index Analysis Utility (SIGMOD 1998)\n* Self-Tuning Database Systems: A Decade of Progress (VLDB 2007)\n* AI Meets AI: Leveraging Query Executions to Improve Index Recommendations (SIGMOD 2019) \n* Automated Database Indexing using Model-free Reinforcement Learning (ICAPS 2020)\n* DRLindex: deep reinforcement learning index advisor for a cluster database (2020 Symposium on International Database Engineering \u0026 Applications)\n* Magic mirror in my hand, which is the best in the land? An Experimental Evaluation of Index Selection Algorithms (VLDB 2020) [GitHub Link](https://github.com/hyrise/index_selection_evaluation)\n* An Index Advisor Using Deep Reinforcement Learning (CIKM 2020) [GitHub Link](https://github.com/rmitbggroup/IndexAdvisor)\n* DBA bandits: Self-driving index tuning under ad-hoc, analytical workloads with safety guarantees (ICDE 2021)\n* MANTIS: Multiple Type and Attribute Index Selection using Deep Reinforcement Learning (IDEAS 2021)\n* AutoIndex: An Incremental Index Management System for Dynamic Workloads (ICDE 2022) [GitHub Link](https://github.com/zhouxh19/AutoIndex)\n* SWIRL: Selection of Workload-aware Indexes using Reinforcement Learning (EDBT 2022) [GitHub Link](https://github.com/hyrise/rl_index_selection)\n* Indexer++: workload-aware online index tuning with transformers and reinforcement learning (ACM SIGAPP SAC, 2022)\n* Budget-aware Index Tuning with Reinforcement Learning (SIGMOD 2022)\n* ISUM: Efficiently Compressing Large and Complex Workloads for Scalable Index Tuning (SIGMOD 2022)\n* DISTILL: Low-Overhead Data-Driven Techniques for Filtering and Costing Indexes for Scalable Index Tuning (VLDB 2022)\n* SmartIndex: An Index Advisor with Learned Cost Estimator (CIKM 2022)\n* HMAB: self-driving hierarchy of bandits for integrated physical database design tuning (VLDB 2022)\n* Learned Index Benefits: Machine Learning Based Index Performance Estimation (VLDB 2023) [GitHub Link](https://github.com/JC-Shi/Learned-Index-Benefits)\n* AIM: A practical approach to automated index management for SQL databases (ICDE 2023)\n* Updatable Learned Indexes Meet Disk-Resident DBMS - From Evaluations to Design Choices (SIGMOD 2023)\n* Index Tuning with Machine Learning on Quantum Computers for Large-Scale Database Applications (AIDB@VLDB 2023)\n* A Data-Driven Index Recommendation System for Slow Queries (CIKM 2023)\n* ML-Powered Index Tuning: An Overview of Recent Progress and Open Challenges (arXiv 2023)\n* Robustness of Updatable Learning-based Index Advisors against Poisoning Attack (SIGMOD 2024)\n* Refactoring Index Tuning Process with Benefit Estimation (VLDB 2024) [GitHub Link](https://github.com/HIT-DB-Group/RIBE)\n* Leveraging Dynamic and Heterogeneous Workload Knowledge to Boost the Performance of Index Advisors (VLDB 2024) [GitHub Link](https://github.com/XMUDM/BALANCE)\n* MFIX: An Efficient and Reliable Index Advisor via Multi-Fidelity Bayesian Optimization (ICDE 2024)\n* TRAP: Tailored Robustness Assessment for Index Advisors via Adversarial Perturbation (ICDE 2024)\n* Online Index Recommendation for Slow Queries (ICDE 2024)\n* Automatic Index Tuning: A Survey (TKDE)\n* Breaking It Down: An In-Depth Study of Index Advisors (VLDB 2024)\n* Can Uncertainty Quantification Enable Better Learning-based Index Tuning? (arXiv 2024)\n* Hybrid Cost Modeling for Reducing Query Performance Regression in Index Tuning (TKDE 2024)\n* A New Paradigm in Tuning Learned Indexes: A Reinforcement Learning Enhanced Approach (arXiv 2025)\n\n### Materialized View\n* Automatic View Generation with Deep Learning and Reinforcement Learning (ICDE 2020)\n* An Autonomous Materialized View Management System with Deep Reinforcement Learning (ICDE 2021)\n* A Technical Report on Dynamic Materialized View Management using Graph Neural Network\n* HMAB: self-driving hierarchy of bandits for integrated physical database design tuning (VLDB 2022)\n* AutoView: An Autonomous Materialized View Management System with Encoder-Reducer (TKDE 2022)\n* Dynamic Materialized View Management using Graph Neural Network (ICDE 2023)\n#### Schema \u0026 Partition\n* Schism: a Workload-Driven Approach to Database Replication and Partitioning (VLDB 2010)\n* Skew-Aware Automatic Database Partitioning in Shared-Nothing, Parallel OLTP Systems (SIGMOD 2012)\n* Automated Data Partitioning for Highly Scalable and Strongly Consistent Transactions (2016 Transactions on Parallel and distributed systems)\n* GridFormation : Towards Self-Driven Online Data Partitioning using Reinforcement Learning (aiDM@SIGMOD 2018)\n* Learning a Partitioning Advisor with Deep Reinforcement Learning (2019)\n* Qd-tree: Learning Data Layouts for Big Data Analytics (SIGMOD 2020)\n* A Genetic Optimization Physical Planner for Big Data Warehouses (2020)\n* Lachesis: Automated Partitioning for UDF-Centric Analytics (VLDB 2021)\n* Instance-Optimized Data Layouts for Cloud Analytics Workloads (SIGMOD 2021)\n* Jigsaw: A Data Storage and Query Processing Engine for Irregular Table Partitioning (SIGMOD 2021)\n* Dalton: Learned Partitioning for Distributed Data Streams (VLDB 2023)\n* Grep: A Graph Learning Based Database Partitioning System (Management of Data 2023)\n* Learned spatial data partitioning （arXiv 2023)\n* Relax and Let the Database Do the Partitioning Online (BIRTE 2011)\n* SWORD: Scalable Workload-Aware Data Placement for Transactional Workloads (EDBT 2013)\n* Online Data Partitioning in Distributed Database Systems (EDBT 2015)\n* A Robust Partitioning Scheme for Ad-Hoc Query Workloads (SOCC 2017)\n* Automated multidimensional data layouts in Amazon Redshift (SIGMOD 2024)\n* Oasis: An Optimal Disjoint Segmented Learned Range Filter (VLDB 2024)\n\n### Cache related\n* A Learned Cache Eviction Framework with Minimal Overhead (arXiv 2023)\n\n## Workload\n\n### Resource Management and Auto-scaling\n\n* Automated Demand-driven Resource Scaling in Relational Database-as-a-Service (SIGMOD 2016)\n* Database Workload Capacity Planning using Time Series Analysis and Machine Learning (SIGMOD 2020)\n* Seagull: An Infrastructure for Load Prediction and Optimized Resource Allocation (VLDB 2020)\n* FIRM: An Intelligent Fine-grained Resource Management Framework for SLO-Oriented Microservices (OSDI 2020)\n* Optimal Resource Allocation for Serverless Queries (arXiv 2021)\n* sinan: ml-based and qos-aware resource management for cloud microservices (ASPLOS 2021)\n* Towards Optimal Resource Allocation for Big Data Analytics (EDBT 2022)\n* Tenant Placement in Over-subscribed Database-as-a-Service Clusters (VLDB 2022)\n* Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing (arXiv 2022)\n* SIMPPO: a scalable and incremental online learning framework for serverless resource management (SoCC 2022)\n* SUFS: A Generic Storage Usage Forecasting Service Through Adaptive Ensemble Learning (ICDE 2023)\n* Auto-WLM: Machine Learning Enhanced Workload Management in Amazon Redshift (SIGMOD-Companion ’23)\n* SeLeP: Learning Based Semantic Prefetching for Exploratory Database Workloads (arXiv 2023)\n* Intelligent scaling in Amazon Redshift (SIGMOD 2024)\n* Forecasting Algorithms for Intelligent Resource Scaling: An Experimental Analysis (Socc 2024)\n\n### Performance Diagnosis and Modeling\n\n- Performance and resource modeling in highly-concurrent OLTP workloads (SIGMOD 2013)\n- DBSherlock: A Performance Diagnostic Tool for Transactional Databases (SIGMOD 2016)\n- A Top-Down Approach to Achieving Performance Predictability in Database Systems (SIGMOD 2017)\n- Diagnosing Root Causes of Intermittent Slow Queries in Cloud Databases (VLDB 2020)\n- Workload-Aware Performance Tuning for Autonomous DBMSs (ICDE 2021)\n- Sage: Practical and Scalable ML-Driven Performance Debugging in Microservices (ASPLOS 2021)\n- D-Bot: Database Diagnosis System using Large Language Models (arXiv 2023)\n- Modeling Shifting Workloads for Learned Database Systems (SIGMOD 2024)\n\n### Workload Shift Detection\n\n- Towards workload shift detection and prediction for autonomic databases (CIKM 2007)\n- Consistent on-line classification of dbs workload events (CIKM 2009)\n- On predictive modeling for optimizing transaction execution in parallel OLTP systems (VLDB 2011)\n\n### Workload Characterization \u0026 Forecasting\n\n* On Workload Characterization of Relational Database Environments (TSE 1992)\n* Workload Models for Autonomic Database Management Systems (International Conference on Autonomic and Autonomous Systems 2006)\n* Workload characterization and prediction in the cloud: A multiple time series approach (APNOMS 2012）\n* Query-based Workload Forecasting for Self-Driving Database Management Systems (SIGMOD 2018）\n* Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics (Arxiv 2018)\n* Database Workload Characterization with Query Plan Encoders (arXiv 2021)\n* Explaining Inference Queries with Bayesian Optimization (VLDB 2021)\n* Statistical Schema Learning with Occam's Razor (SIGMOD 2022)\n* Intelligent Automated Workload Analysis for Database Replatforming (SIGMOD 2022)\n* Stitcher: Learned Workload Synthesis from Historical Performance Footprints (EDBT 2022)\n* DBAugur: An Adversarial-based Trend Forecasting System for Diversified Workloads (ICDE 2023)\n* An Efficient Online Prediction of Host Workloads Using Pruned GRU Neural Nets (arXiv 2023)\n* Uncertainty-Aware Workload Prediction in Cloud Computing (arXiv 2023)\n* Real-Time Workload Pattern Analysis for Large-Scale Cloud Databases (VLDB 2023)\n* Robust Auto-Scaling with Probabilistic Workload Forecasting for Cloud Databases (ICDE 2024)\n* QPSEncoder: A Database Workload Encoder with Deep Learning (DEXA 2024)\n\n## Query Optimization\n* Learned Query Optimizer: What is New and What is Next (SIGMOD 2024)\n* GLO: Towards Generalized Learned Query Optimization (ICDE 2024)\n* Robust Query Optimization in the Era of Machine Learning: State-of-the-Art and Future Directions (ICDE 2024)\n* Presto’s History-based Query Optimizer (VLDB 2024)\n* Spatial Query Optimization With Learning (VLDB 2024)\n* DBG-PT: A Large Language Model Assisted Query Performance Regression Debugger (VLDB 2024)\n* How Good are Learned Cost Models, Really? Insights from Query Optimization Tasks (SIGMOD 2025) [GitHub Link](https://github.com/DataManagementLab/lcm-eval)\n\n### Query Rewrite\n* Sia: Optimizing Queries using Learned Predicates (SIGMOD 2021)\n* A Learned Query Rewrite System using Monte Carlo Tree Search (VLDB 2022)\n* WeTune: Automatic Discovery and Verification of Query Rewrite Rules (SIGMOD 2022)\n* A Learned Query Rewrite System (VLDB 2023)\n* Query Rewriting via Large Language Models (arXiv 2024)\n* LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency (arXiv 2024) [GitHub](https://github.com/DAMO-NLP-SG/LLM-R2)\n* R-Bot: An LLM-based Query Rewrite System (arXiv 2024)\n\n### Cardinality Estimation\n* Are We Ready For Learned Cardinality Estimation? (VLDB 2021) [GitHub Link](https://github.com/sfu-db/AreCELearnedYet)\n* A Unified Deep Model of Learning from both Data and Queries for Cardinality Estimation (SIGMOD 2021)\n* LATEST: Learning-Assisted Selectivity Estimation Over Spatio-Textual Streams (ICDE 2021)\n* Fauce: Fast and Accurate Deep Ensembles with Uncertainty for Cardinality Estimation (VLDB 2021)\n* Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluation (arXiv 2021) [GitHub Link](https://github.com/Nathaniel-Han/End-to-End-CardEst-Benchmark)\n* Learned Cardinality Estimation: A Design Space Exploration and A Comparative Evaluation (VLDB 2022)\n* Glue: Adaptively Merging Single Table Cardinality to Estimate Join Query Size (aiXiv 2021)\n* Unsupervised Selectivity Estimation by Integrating Gaussian Mixture Models and an Autoregressive Model (EDBT 2022)\n* Selectivity Functions of Range Queries are Learnable (SIGMOD 2022)\n* Prediction Intervals for Learned Cardinality Estimation: An Experimental Evaluation (ICDE 2022)\n* Learned Cardinality Estimation: An In-depth Study (SIGMOD 2022)\n* FactorJoin: A New Cardinality Estimation Framework for Join Queries (SIGMOD 2023)\n* AutoCE: An Accurate and Efficient Model Advisor for Learned Cardinality Estimation (ICDE 2023)\n* Couper: Memory-Efficient Cardinality Estimation under Unbalanced Distribution (ICDE 2023)\n* ALECE: An Attention-based Learned Cardinality Estimator for SPJ Queries on Dynamic Workloads (VLDB 2023)\n* Advanced Dataset Discovery: When Multi-Query-Dataset Cardinality Estimation Matters (aiXiv 2024)\n* Sample-Efficient Cardinality Estimation Using Geometric Deep Learning (VLDB 2024)\n* PRICE: A Pretrained Model for Cross-Database Cardinality Estimation (arXiv 2024) [GitHub Lint](https://github.com/StCarmen/PRICE)\n* ByteCard: Enhancing ByteDance's Data Warehouse with Learned Cardinality Estimation (SIGMOD 2024)\n* ASM in Action: Fast and Practical Learned Cardinality Estimation (SIGMOD 2024)\n* CardBench: A Benchmark for Learned Cardinality Estimation in Relational Database (arXiv 2024)\n* Duet: efficient and scalable hybriD neUral rElation undersTanding. (ICDE 2024)\n* Cardinality Estimation of LIKE Predicate Queries using Deep Learning (SIGMOD 2025)\n#### Data-based\n* Self-Tuning, GPU-Accelerated Kernel Density Models for Multidimensional Selectivity Estimation (SIGMOD 2015)\n* Estimating Join Selectivities using Bandwidth-Optimized Kernel Density Models (VLDB 2017)\n* DeepDB: Learn from Data, not from Queries! (VLDB 2020) [GitHub Link](https://github.com/DataManagementLab/deepdb-public)\n* Deep Unsupervised Cardinality Estimation (VLDB 2019) \n* Multi-Attribute Selectivity Estimation Using Deep Learning (arXiv 2019)\n* Deep Learning Models for Selectivity Estimation of Multi-Attribute Queries (SIGMOD 2020)\n* NeuroCard: One Cardinality Estimator for All Tables (VLDB 2020) [GitHub Link](https://github.com/neurocard/neurocard)\n* Learning to Sample: Counting with Complex Queries (VLDB 2020)\n* Selectivity estimation using probabilistic models (SIGMOD 2001)\n* Lightweight graphical models for selectivity estimation without independence assumptions (VLDB 2011)\n* Efficiently adapting graphical models for selectivity estimation (VLDB 2013)\n* An Approach Based on Bayesian Networks for Query Selectivity Estimation (DASFAA 2019)\n* BayesCard: A Unified Bayesian Framework for Cardinality Estimation (arXiv 2020) [GitHub Link](https://github.com/wuziniu/BayesCard)\n* Online Sketch-based Query Optimization (arXiv 2021)\n* LMKG: Learned Models for Cardinality Estimation in Knowledge Graphs (arXiv 2021)\n* LHist: Towards Learning Multi-dimensional Histogram for Massive Spatial Data (ICDE 2021)\n* FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation (VLDB 2021) [GitHub Link](https://github.com/wuziniu/FSPN)\n* Astrid: Accurate Selectivity Estimation for String Predicates using Deep Learning (VLDB 2021)\n* FACE: A Normalizing Flow based Cardinality Estimator (VLDB 2022)\n* Pre-training Summarization Models of Structured Datasets for Cardinality Estimation (VLDB 2022)\n* Cardinality Estimation of Approximate Substring Queries using Deep Learning (VLDB 2022)\n* Speeding Up End-to-end Query Execution via Learning-based Progressive Cardinality Estimation (Proceedings of the ACM on Management of Data)\n* Cardinality estimation with smoothing autoregressive models (WWW 2023)\n* Cardinality estimation using normalizing flow (VLDBJ 2023)\n* LPLM: A Neural Language Model for Cardinality Estimation of LIKE-Queries (SIGMOD 2024)\n* ASM: Harmonizing Autoregressive Model, Sampling, and Multi-dimensional Statistics Merging for Cardinality Estimation (SIGMOD 2024)\n* ASM in Action: Fast and Practical Learned Cardinality Estimation (SIGMOD 2024)\n* SAFE: Sampling-Assisted Fast Learned Cardinality Estimation for Dynamic Spatial Data (DEXA 2024)\n* Updateable Data-Driven Cardinality Estimator with Bounded Q-error (arXiv 2024)\n* Grid-AR: A Grid–based Booster for Learned Cardinality Estimation and Range Joins (arXiv 2024)\n#### Query-based\n* Adaptive selectivity estimation using query feedback (SIGMOD 1994)\n* Selectivity Estimation in Extensible Databases -A Neural Network Approach （VLDB 1998）\n* Effective query size estimation using neural networks.  (Applied Intelligence 2002)\n* LEO - DB2's LEarning optimizer （VLDB 2011)\n* A Black-Box Approach to Query Cardinality Estimation (CIDR 07)\n* Cardinality Estimation Using Neural Networks (2015)\n* Towards a learning optimizer for shared clouds (VLDB 2018)\n* Learning State Representations for Query Optimization with Deep Reinforcement Learning  (DEEM@SIGMOD2018)\n* Learned Cardinalities: Estimating Correlated Joins with Deep Learning （CIDR2019）[GitHub Link](https://github.com/andreaskipf/learnedcardinalities)\n* Estimating Cardinalities with Deep Sketches (SIGMOD 2019) [GitHub Link](https://github.com/andreaskipf/learnedcardinalities)\n* Selectivity estimation for range predicates using lightweight models (VLDB 2019)\n* (Review) An Empirical Analysis of Deep Learning for Cardinality Estimation (arXiv 2019)\n* Flexible Operator Embeddings via Deep Learning (arXiv 2019)\n* Improved Cardinality Estimation by Learning Queries Containment Rates (EDBT 2020)\n* NN-based Transformation of Any SQL Cardinality Estimator for Handling DISTINCT, AND, OR and NOT (2020)\n* QuickSel: Quick Selectivity Learning with Mixture Models (SIGMOD 2020)\n* Efficiently Approximating Selectivity Functions using Low Overhead Regression Models (VLDB 2020)\n* Learned Cardinality Estimation for Similarity Queries (SIGMOD 2021)\n* Uncertainty-aware Cardinality Estimation by Neural Network Gaussian Process (arXiv 2021)\n* Flow-Loss: Learning Cardinality Estimates That Matter (VLDB 2021)\n* Warper: Efficiently Adapting Learned Cardinality Estimators to Data and Workload Drifts (SIGMOD 2022)\n* Lightweight and Accurate Cardinality Estimation by Neural Network Gaussian Process for Approximate Complex Event Processing (SIGMOD 2022)\n* Enhanced Featurization of Queries with Mixed Combinations of Predicates for ML-based Cardinality Estimation (EDBT 2023)\n* Speeding Up End-to-end Query Execution via Learning-based Progressive Cardinality Estimation (SIGMOD 2023)\n* Robust Query Driven Cardinality Estimation under Changing Workloads (VLDB 2023)\n* Learned Probing Cardinality Estimation for High-Dimensional Approximate NN Search (ICDE 2023)\n* CEDA: Learned Cardinality Estimation with Domain Adaptation (VLDB 2023)\n* Efficient Cardinality and Cost Estimation with Bidirectional Compressor-based Ensemble Learning (arXiv 2023)\n* Adding Domain Knowledge to Query-Driven Learned Databases (arXiv 2023)\n* PACE: Poisoning Attacks on Learned Cardinality Estimation (SIGMOD 2024)\n* Sample-Efficient Cardinality Estimation Using Geometric Deep Learning (VLDB 2024)\n* Automating localized learning for cardinality estimation based on XGBoost (Knowledge and Information Systems)\n### Cost Estimation\n#### Single Query\n* Statistical learning techniques for costing XML queries (VLDB 2005)\n* Predicting multiple metrics for queries: Better decisions enabled by machine learning （icde 2009)\n* The Case for Predictive Database Systems : Opportunities and Challenges （CIDR 2011)\n* Learning-based query performance modeling and prediction (ICDE 2012)\n* Robust estimation of resource consumption for SQL queries using statistical techniques (VLDB 2012)\n* Learning-based SPARQL query performance modeling and prediction (WWW 2017)\n* Plan-Structured Deep Neural Network Models for Query Performance Prediction (arXiv 2019)\n* An End-to-End Learning-based Cost Estimator (arXiv 2019)(VLDB 2019)\n* Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings (2020)\n* DBMS Fitting: Why should we learn what we already know? (CIDR 2020)\n* A Note On Operator-Level Query Execution Cost Modeling (2020)\n* ML-based Cross-Platform Query Optimization (ICDE 2020)\n* Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction (VLDB 2022)\n* Efficient Learning with Pseudo Labels for Query Cost Estimation (CIKM 2022)\n* gCBO: A Cost-based Optimizer for Graph Databases (CIKM 2022)\n* QueryFormer: A Tree Transformer Model for Query Plan Representation (VLDB 2022)\n* BASE: Bridging the Gap between Cost and Latency for Query Optimization (VLDB 2023)\n* Rethinking Learned Cost Models: Why Start from Scratch? (PACMMOD 2023)\n* Budget-aware Query Tuning: An AutoML Perspective (arXiv 2024)\n* OS Pre-trained Transformer: Predicting Query Latencies across Changing System Contexts [GitHub Link](https://github.com/parimarjan/LatencyPredictor)\n* Precision Meets Resilience: Cross-Database Generalization with Uncertainty Quantification for Robust Cost Estimation (CIKM 2024)\n* DACE: A Database-Agnostic Cost Estimator (ICDE 2024)\n* QCFE: An Efficient Feature Engineering for Query Cost Estimation(ICDE 2024)\n* T3: Accurate and Fast Performance Prediction for Relational Database Systems With Compiled Decision Trees (arXiv 2025)\n\n\n#### Concurrent\n* PQR: Predicting query execution times for autonomous workload management （ICAC 2008）\n* Performance Prediction for Concurrent Database Workloads (SIGMOD 2011)\n* Predicting completion times of batch query workloads using interaction-aware models and simulation(EDBT 2011)\n* Interaction-aware scheduling of report-generation workloads (VLDB 2011) （有调度策略）\n* Towards predicting query execution time for concurrent and dynamic database workloads (not machine learning) （VLDB 2014）\n* Contender: A Resource Modeling Approach for Concurrent Query Performance Prediction （EDBT 2014）\n* Query Performance Prediction for Concurrent Queries using Graph Embedding (VLDB 2020)\n* Efficient Deep Learning Pipelines for Accurate Cost Estimations Over Large Scale Query Workload (SIGMOD 2021)\n* A Resource-Aware Deep Cost Model for Big Data Query Processing (ICDE 2022)\n* Stage: Query Execution Time Prediction in Amazon Redshif (SIGMOD 2024)\n### Join Optimization\n* Adaptive Optimization of Very Large Join Queries (SIGMOD 2018) (Not machine learning\n* Deep Reinforcement Learning for Join Order Enumeration (aiDM@SIGMOD 2018)\n* Learning to Optimize Join Queries With Deep Reinforcement Learning (ArXiv)\n* Reinforcement Learning with Tree-LSTM for Join Order Selection (ICDE 2020)\n* Research Challenges in Deep Reinforcement Learning-based Join Query Optimization (aiDM 2020)\n* Efficient Join Order Selection Learning with Graph-based Representation (KDD 2022)\n* SOAR:A Learned Join Order Selector with Graph Attention Mechanism （IJCNN 2022）\n* Query Join Order Optimization Method Based on Dynamic Double Deep Q-Network (Electronics 2023)\n* Coral: federated query join order optimization based on deep reinforcement learning (WWW 2023)\n* JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning (arXiv 2023)\n* Join Order Selection with Deep Reinforcement Learning: Fundamentals, Techniques, and Challenges (VLDB 2023)\n* Sub-optimal Join Order Identification with L1-error (SIGMOD 2024)\n* TESSM: Tree-based Selective State Space Models for Efficient Join Order Selection Learning (CIKM 2024)\n### Query Plan\n* Plan Selection Based on Query Clustering （VLDB 2002)\n* Cost-Based Query Optimization via AI Planning (AAAI 2014)\n* Sampling-Based Query Re-Optimization (SIGMOD 2016)\n* Learning State Representations for Query Optimization with Deep Reinforcement Learning  (DEEM@SIGMOD2018)\n* Towards a Hands-Free Query Optimizer through Deep Learning （CIDR 2019)\n* Neo: A Learned Query Optimizer (VLDB 2019)\n* Bao: Learning to Steer Query Optimizers (2020)\n* ML-based Cross-Platform Query Optimization (ICDE 2020)\n* Learning-based Declarative Query Optimization (2021)\n* **Bao: Making Learned Query Optimization Practical** (SIGMOD 2021 **Best Paper**!) [Doc](https://rmarcus.info/bao_docs/introduction.html) [GitHub Link](https://github.com/learnedsystems/BaoForPostgreSQL)\n* Microlearner: A fine-grained Learning Optimizer for Big Data Workloads at Microsoft (2021)\n* Steering Query Optimizers: A Practical Take on Big Data Workloads (SIGMOD 2021)\n* A Unified Transferable Model for ML-Enhanced DBMS (CIDR 2021)\n* Balsa: Learning a Query Optimizer Without Expert Demonstrations (SIGMOD 2022)\n* Leveraging Query Logs and Machine Learning for Parametric Query Optimization (VLDB 2022)\n* Deploying a Steered Query Optimizer in Production at Microsoft (SIGMOD 2022)\n* Building Learned Federated Query Optimizers (VLDB 2022 PhD Workshop)\n* Cost-based or Learning-based? A Hybrid Query Optimizer for Query Plan Selection (VLDB 2022)\n* Learn What Really Matters: A Learning-to-Rank Approach for ML-based Query Optimization (BTW 2023)\n* Lero: A Learning-to-Rank Query Optimizer (VLDB 2023) [GitHub Link](https://github.com/AlibabaIncubator/Lero-on-PostgreSQL)\n* Learned Query Superoptimization (arXiv 2023)\n* Kepler: Robust Learning for Faster Parametric Query Optimization (SIGMOD 2023)\n* LOGER: A Learned Optimizer towards Generating Efficient and Robust Query Execution Plans (VLDB 2023)\n* BitE : Accelerating Learned Query Optimization in a Mixed-Workload Environment (arXiv 2023)\n* Reinforcement Learning-based SPARQL Join Ordering Optimizer\n* LEON: A New Framework for ML-Aided Query Optimization (VLDB 2023)\n* AutoSteer: Learned Query Optimization for Any SQL Database (VLDB 2023)\n* FASTgres: Making Learned Query Optimizer Hinting Effective (VLDB 2023)\n* Simple Adaptive Query Processing vs. Learned Query Optimizers: Observations and Analysis (VLDB 2023)\n* QO-Insight: Inspecting Steered Query Optimizer (VLDB Demo 2023)\n* QPSeeker: An Efficient Neural Planner combining both data and queries through Variational Inference (EDBT 2024)\n* FOSS: A Self-Learned Doctor for Query Optimizer (ICDE 2024)\n* Lemo: A Cache-Enhanced Learned Optimizer for Concurrent Queries (PACMMOD 2023)\n* A Comparative Study and Component Analysis of Query Plan Representation Techniques in ML4DB Studies (VLDB 2024)\n* Learned Optimizer for Online Approximate Query Processing in Data Exploration (TKDE 2024)\n* A learning-based framework for spatial join processing: estimation, optimization and tuning (VLDB 2024)\n* Roq: Robust Query Optimization Based on a Risk-aware Learned Cost Model (arXiv 2024)\n* PLAQUE: Automated Predicate Learning at Query Time (SIGMOD 2024)\n* GLO: Towards Generalized Learned Query Optimization (ICDE 2024)\n* Eraser: Eliminating Performance Regression on Learned Query Optimizer (VLDB 2024)\n* Low Rank Approximation for Learned Query Optimization (aiDM 2024)\n* Lero: applying learning-to-rank in query optimizer (VLDB 2024)\n* RobOpt: A Tool for Robust Workload Optimization Based on Uncertainty-Aware Machine Learning (SIGMOD 2024)\n* A Novel Technique for Query Plan Representation Based on Graph Neural (Big Data Analytics and Knowledge Discovery)\n* An Exploratory Case Study of Query Plan Representations (aiXiv 2024)\n* JAPO: learning join and pushdown order for cloud-native join optimization (Frontiers of Computer Science 2024)\n* Steering the PostgreSQL query optimizer using hinting: State-Of-The-Art and open challenges (35th GI-Workshop on Foundations of Databases)\n* PARQO: Penalty-Aware Robust Plan Selection in Query Optimization (arXiv 2024)\n* HERO: Hint-Based Efficient and Reliable Query Optimizer (arXiv 2024)\n* Can Large Language Models Be Query Optimizer for Relational Databases? (arXiv 2025)\n* Learned Offline Query Planning via Bayesian Optimization (arXiv 2025)\n\n## Query Execution\n### Sort\n* The Case for a Learned Sorting Algorithm (SIGMOD 2020)\n* Defeating duplicates: A re-design of the LearnedSort algorithm (aiXiv 2021)\n* Towards Parallel Learned Sorting (arXiv 2022)\n### Join\n* SkinnerDB : Regret-Bounded Query Evaluation via Reinforcement Learning (VLDB 2018)\n* The Case for Learned In-Memory Joins (arXiv 2021)\n### Adaptive Query Processing\n* Eddies: Continuously adaptive query processing. (SIGMOD 2000)\n* Micro adaptivity in Vectorwise (SIGMOD 2013)\n* Cuttlefish: A Lightweight Primitive for Adaptive Query Processing (2018)\n* Scalable Multi-Query Execution using Reinforcement Learning (SIGMOD 2021)\n### Approximate Query Processing\n* DBEST: Revisiting approximate query processing engines with machine learning models (SIGMOD 2019)\n* LAQP: Learning-based Approximate Query Processing (2020)\n* Approximate Query Processing for Data Exploration using Deep Generative Models (ICDE 2020)\n* ML-AQP: Query-Driven Approximate Query Processing based on Machine Learning (2020)\n* Approximate Query Processing for Group-By Queries based on Conditional Generative Models (2021)\n* Learned Approximate Query Processing: Make it Light, Accurate and Fast (CIDR 2021)\n* NeuroSketch: Fast and Approximate Evaluation of Range Aggregate Queries with Neural Networks (SIGMOD 2023)\n* Exploiting Machine Learning Models for Approximate Query Processing (Big Data 2022)\n* Tuple Bubbles: Learned Tuple Representations for Tunable Approximate Query Processing (aiDM 2023)\n* Learning-Based Sample Tuning for Approximate Query Processing in Interactive Data Exploration (TKDE 2024)\n### Sheduling\n* Workload management for cloud databases via machine learning (ICDE 2016 WiseDB)\n* A learning-based service for cost and performance management of cloud databases （ICDEW 2017）(short version for WiSeDB)\n* WiSeDB: A Learning-based Workload Management Advisor for Cloud Databases (2016 VLDB)\n* Learning Scheduling Algorithms for Data Processing Clusters (SIGCOMM 2019)\n* CrocodileDB: Efficient Database Execution through Intelligent Deferment (CIDT 2020)\n* Buffer Pool Aware Query Scheduling via Deep Reinforcement Learning (2020)\n* Self-Tuning Query Scheduling for Analytical Workloads (SIGMOD 2021)\n* LSched: A Workload-Aware Learned Query Scheduler for Analytical Database Systems (SIGMOD 2022)\n* DBMLSched: Scheduling In-database Machine Learning Jobs (AIDB@VLDB 2023)\n* Learning Interpretable Scheduling Algorithms for Data Processing Clusters (arXiv 2024)\n\n(transaction 👇)\n\n* Scheduling OLTP transactions via learned abort prediction (aiDM@SIGMOD 2019)\n* Scheduling OLTP Transactions via Machine Learning （2019）\n* Polyjuice: High-Performance Transactions via Learned Concurrency Control (OSDI 2021)\n\n## Text-to-SQL\n* SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning (arXiv 2017)\n* An End-to-end Neural Natural Language Interface for Databases (arXiv 2018)\n* SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-Domain Text-to-SQL Task (EMNLP 2018)\n* Robust Text-to-SQL Generation with Execution-Guided Decoding (arXiv 2018)\n* Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation (ACL 2019)\n* Global Reasoning over Database Structures for Text-to-SQL Parsing (EMNLP 2019)\n* Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing (ACL 2019)\n* Natural language to SQL: Where are we today? (VLDB 2020)\n* Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing (EMNLP Findings 2020)\n* RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers (ACL 2020)\n* Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing (ACL 2020)\n* TAPAS: Weakly Supervised Table Parsing via Pre-training (ACL 2020)\n* TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data (ACL 2020)\n* Semantic Evaluation for Text-to-SQL with Distilled Test Suites (EMNLP 2020)\n* SMBOP: Semi-autoregressive Bottom-up Semantic Parsing (NAACL-HLT 2021)\n* Natural SQL: Making SQL Easier to Infer from Natural Language Specifications (EMNLP Findings 2021)\n* LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations (ACL 2021)\n* Structure-Grounded Pretraining for Text-to-SQL (NAACL-HLT 2021)\n* GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing (ICLR 2021)\n* SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQL (NeurIPS 2021)\n* GP: Context-free Grammar Pre-training for Text-to-SQL Parsers (arXiv 2021)\n* Relation Aware Semi-autoregressive Semantic Parsing for NL2SQL (arXiv 2021)\n* On Robustness of Neural Semantic Parsers (EACL 2021)\n* MT-Teql: Evaluating and Augmenting Neural NLIDB on Real-world Linguistic and Schema Variations (VLDB 2021)\n* PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models (EMNLP 2021)\n* Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training (AAAI 2021)\n* Towards robustness of text-to-sql models against synonym substitution (ACL 2021)\n* Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization (EMNLP 2021)\n* CodexDB: Generating Code for Processing SQL Queries using GPT-3 Codex (arXiv 2022)\n* S2SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers (arXiv 2022)\n* UNIFIEDSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (EMNLP 2022)\n* RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL (EMNLP 2022)\n* UNISAR: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQL (arXiv 2022)\n* N-Best Hypotheses Reranking for Text-To-SQL Systems (SLT 2022)\n* Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graph (KDD 2022)\n* SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising (NAACL-HLT Findings 2022)\n* STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing (EMNLP Findings 2022)\n* Towards Generalizable and Robust Text-to-SQL Parsing (EMNLP Findings 2022)\n* SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers (COLING 2022)\n* Towards robustness of text-to-sql models against natural and realistic adversarial table perturbation (ACL 2022)\n* Evaluating the Text-to-SQL Capabilities of Large Language Models (arXiv 2022)\n* A survey on deep learning approaches for text-to-SQL (VLDBJ 2023)\n* GAR: A Generate-and-Rank Approach for Natural Language to SQL Translation (ICDE 2023)\n* Exploring the Compositional Generalization in Context Dependent Text-to-SQL Parsing (arXiv 2023)\n* Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques (arXiv 2023)\n* Exploring Chain-of-Thought Style Prompting for Text-to-SQL (arXiv 2023)\n* Few-shot Text-to-SQL Translation using Structure and Content Prompt Learning (SIGMOD 2023)\n* Multitask pretraining with structured knowledge for text-to-SQL generation (ACL 2023)\n* Demonstrating GPT-DB: Generating Query-Specific and Customizable Code for SQL Processing with GPT-4 (VLDB Demo 2023)\n* Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing (AAAI 2023)\n* SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (arXiv 2023)\n* Teaching Large Language Models to Self-Debug (arXiv 2023)\n* A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability (arXiv 2023)\n* DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction (arXiv 2023)\n* C3: Zero-shot Text-to-SQL with ChatGPT (arXiv 2023)\n* RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL (AAAI 2023)\n* Dr.spider: A Diagnostic Evaluation Benchmark Towards Text-To-Sql Robustness (ICLR 2023)\n* Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL (arXiv 2024)\n* Natural language to SQL [Resource repo](https://github.com/yechens/NL2SQL)\n* Combining Small Language Models and Large Language Models for Zero-Shot NL2SQL (VLDB 2024)\n* Awesome-Text2SQL [Resource repo](https://github.com/eosphoros-ai/Awesome-Text2SQL)\n* Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows [Resource repo](https://github.com/xlang-ai/Spider2) (arXiv 2024)\n\n## SQL Related\n* Query2Vec (ArXiv)\n* Facilitating SQL Query Composition and Analysis (ArXiv 2020)\n* From Natural Language Processing to Neural Databases (VLDB 2021)\n* BERT Meets Relational DB: Contextual Representations of Relational Databases\n* LearnedSQLGen: Constraint-aware SQL Generation using Reinforcement Learning (SIGMOD 2022)\n* PreQR: Pre-training Representation for SQL Understanding (SIGMDO 2022)\n* From BERT to GPT-3 Codex: Harnessing the Potential of Very Large Language Models for Data Management (VLDB 2022)\n* Query Generation based on Generative Adversarial Networks (arXiv 2023)\n\n## Stargazers over time\n\n[![Stargazers over time](https://starchart.cc/LumingSun/ML4DB-paper-list.svg)](https://starchart.cc/LumingSun/ML4DB-paper-list)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLumingSun%2FML4DB-paper-list","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FLumingSun%2FML4DB-paper-list","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLumingSun%2FML4DB-paper-list/lists"}