{"id":13468451,"url":"https://github.com/dlt-hub/dlt","last_synced_at":"2026-04-01T18:37:38.550Z","repository":{"id":37415123,"uuid":"452221115","full_name":"dlt-hub/dlt","owner":"dlt-hub","description":"data load tool (dlt) is an open source Python library that makes data loading easy 🛠️ ","archived":false,"fork":false,"pushed_at":"2026-03-28T00:34:00.000Z","size":106788,"stargazers_count":5130,"open_issues_count":313,"forks_count":481,"subscribers_count":27,"default_branch":"devel","last_synced_at":"2026-03-28T00:48:44.951Z","etag":null,"topics":["data","data-engineering","data-lake","data-loading","data-warehouse","elt","extract","load","python","transform"],"latest_commit_sha":null,"homepage":"https://dlthub.com/docs","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dlt-hub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2022-01-26T09:51:04.000Z","updated_at":"2026-03-27T18:05:59.000Z","dependencies_parsed_at":"2024-01-09T11:31:34.378Z","dependency_job_id":"23e86537-00a0-4a71-9af4-f6fdd9940b1a","html_url":"https://github.com/dlt-hub/dlt","commit_stats":null,"previous_names":[],"tags_count":123,"template":false,"template_full_name":null,"purl":"pkg:github/dlt-hub/dlt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlt-hub%2Fdlt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlt-hub%2Fdlt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlt-hub%2Fdlt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlt-hub%2Fdlt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dlt-hub","download_url":"https://codeload.github.com/dlt-hub/dlt/tar.gz/refs/heads/devel","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dlt-hub%2Fdlt/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290912,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","data-engineering","data-lake","data-loading","data-warehouse","elt","extract","load","python","transform"],"created_at":"2024-07-31T15:01:11.082Z","updated_at":"2026-04-01T18:37:38.539Z","avatar_url":"https://github.com/dlt-hub.png","language":"Python","readme":"\u003ch1 align=\"center\"\u003e\n    \u003cstrong\u003edata load tool (dlt) — the open-source Python library that automates all your tedious data loading tasks\u003c/strong\u003e\n\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\nBe it a Google Colab notebook, AWS Lambda function, an Airflow DAG, your local laptop,\u003cbr/\u003eor a GPT-4 assisted development playground—\u003cstrong\u003edlt\u003c/strong\u003e can be dropped in anywhere.\n\u003c/p\u003e\n\n\n\u003ch3 align=\"center\"\u003e\n\n🚀 Join our thriving community of likeminded developers and build the future together!\n\n\u003c/h3\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca target=\"_blank\" href=\"https://dlthub.com/community\" style=\"background:none\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/slack-join-dlt.svg?labelColor=191937\u0026color=6F6FF7\u0026logo=slack\" style=\"width: 260px;\"  /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003ca target=\"_blank\" href=\"https://pypi.org/project/dlt/\" style=\"background:none\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/v/dlt?labelColor=191937\u0026color=6F6FF7\"\u003e\n  \u003c/a\u003e\n  \u003ca target=\"_blank\" href=\"https://pypi.org/project/dlt/\" style=\"background:none\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/pyversions/dlt?labelColor=191937\u0026color=6F6FF7\"\u003e\n  \u003c/a\u003e\n  \u003ca target=\"_blank\" href=\"https://pypi.org/project/dlt/\" style=\"background:none\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/dm/dlt?labelColor=191937\u0026color=6F6FF7\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n## Installation\n\ndlt supports Python 3.9 through Python 3.14. Note that some optional extras are not yet available for Python 3.14, so support for this version is considered experimental.\n\n```sh\npip install dlt\n```\n\n## Quick Start\n\nLoad chess game data from chess.com API and save it in DuckDB:\n\n```python\nimport dlt\nfrom dlt.sources.helpers import requests\n\n# Create a dlt pipeline that will load\n# chess player data to the DuckDB destination\npipeline = dlt.pipeline(\n    pipeline_name='chess_pipeline',\n    destination='duckdb',\n    dataset_name='player_data'\n)\n\n# Grab some player data from Chess.com API\ndata = []\nfor player in ['magnuscarlsen', 'rpragchess']:\n    response = requests.get(f'https://api.chess.com/pub/player/{player}')\n    response.raise_for_status()\n    data.append(response.json())\n\n# Extract, normalize, and load the data\npipeline.run(data, table_name='player')\n```\n\n\nTry it out in our **[Colab Demo](https://colab.research.google.com/drive/1NfSB1DpwbbHX9_t5vlalBTf13utwpMGx?usp=sharing)** or directly on our wasm-based [playground](https://dlthub.com/docs/tutorial/playground) in our docs.\n\n## Features\n\ndlt is an open-source Python library that loads data from various, often messy data sources into well-structured datasets. It provides lightweight Python interfaces to extract, load, inspect, and transform data. dlt and dlt docs are built from the ground up to be used with LLMs: the [LLM-native workflow](https://dlthub.com/docs/dlt-ecosystem/llm-tooling/llm-native-workflow) will take your pipeline code to data in a notebook for over [5000 sources](https://dlthub.com/workspace).\n\ndlt is designed to be easy to use, flexible, and scalable:\n\n- dlt extracts data from [REST APIs](https://dlthub.com/docs/tutorial/rest-api), [SQL databases](https://dlthub.com/docs/tutorial/sql-database), [cloud storage](https://dlthub.com/docs/tutorial/filesystem), [Python data structures](https://dlthub.com/docs/tutorial/load-data-from-an-api), and [many more](https://dlthub.com/docs/dlt-ecosystem/verified-sources).\n- dlt infers [schemas](https://dlthub.com/docs/general-usage/schema) and [data types](https://dlthub.com/docs/general-usage/schema/#data-types), [normalizes the data](https://dlthub.com/docs/general-usage/schema/#data-normalizer), and handles nested data structures.\n- dlt supports a variety of [popular destinations](https://dlthub.com/docs/dlt-ecosystem/destinations/) and has an interface to add [custom destinations](https://dlthub.com/docs/dlt-ecosystem/destinations/destination) to create reverse ETL pipelines.\n- dlt automates pipeline maintenance with [incremental loading](https://dlthub.com/docs/general-usage/incremental-loading), [schema evolution](https://dlthub.com/docs/general-usage/schema-evolution), and [schema and data contracts](https://dlthub.com/docs/general-usage/schema-contracts).\n- dlt supports [Python and SQL data access](https://dlthub.com/docs/general-usage/dataset-access/), [transformations](https://dlthub.com/docs/dlt-ecosystem/transformations), [pipeline inspection](https://dlthub.com/docs/general-usage/dashboard.md), and [visualizing data in Marimo Notebooks](https://dlthub.com/docs/general-usage/dataset-access/marimo).\n- dlt can be deployed anywhere Python runs, be it on [Airflow](https://dlthub.com/docs/walkthroughs/deploy-a-pipeline/deploy-with-airflow-composer), [serverless functions](https://dlthub.com/docs/walkthroughs/deploy-a-pipeline/deploy-with-google-cloud-functions), or any other cloud deployment of your choice.\n\n## Documentation\n\nFor detailed usage and configuration, please refer to the [official documentation](https://dlthub.com/docs).\n\n## Examples\n\nYou can find examples for various use cases in the [examples](docs/examples) folder, or in the [code examples section](https://dlthub.com/docs/examples) of our docs page.\n\n## Adding as dependency\n\n`dlt` follows the semantic versioning with the [`MAJOR.MINOR.PATCH`](https://peps.python.org/pep-0440/#semantic-versioning) pattern.\n\n* `major` means breaking changes and removed deprecations\n* `minor` new features, sometimes automatic migrations\n* `patch` bug fixes\n\nWe suggest that you allow only `patch` level updates automatically using the [Compatible Release Specifier](https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release). For example **dlt~=1.23.0** allows only versions **\u003e=1.23.0** and less than **\u003c1.24.0**\n\nPlease also see our [release notes](https://github.com/dlt-hub/dlt/releases) for notable changes between versions.\n\n## Get Involved\n\nThe dlt project is quickly growing, and we're excited to have you join our community! Here's how you can get involved:\n\n- **Connect with the Community**: Join other dlt users and contributors on our [Slack](https://dlthub.com/community)\n- **Report issues and suggest features**: Please use the [GitHub Issues](https://github.com/dlt-hub/dlt/issues) to report bugs or suggest new features. Before creating a new issue, make sure to search the tracker for possible duplicates and add a comment if you find one.\n- **Track progress of our work and our plans**: Please check out our [public Github project](https://github.com/orgs/dlt-hub/projects/9)\n- **Improve documentation**: Help us enhance the dlt documentation.\n\n## Contribute code\nPlease read [CONTRIBUTING](CONTRIBUTING.md) before you make a PR.\n\n- 📣 **New destinations are unlikely to be merged** due to high maintenance cost (but we are happy to improve SQLAlchemy destination to handle more dialects)\n- Significant changes require tests and docs and in many cases writing tests will be more laborious than writing code\n- Bugfixes and improvements are welcome! You'll get help with writing tests and docs + a decent review.\n\n## License\n\n`dlt` is released under the [Apache 2.0 License](LICENSE.txt).\n","funding_links":[],"categories":["Python","其他_机器学习与深度学习","data","🔄 Data Plattform Tools"],"sub_categories":["🧠 Prompt Engineering \u0026 Memory Bank"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdlt-hub%2Fdlt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdlt-hub%2Fdlt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdlt-hub%2Fdlt/lists"}