Entry
Appears in 4 awesome lists
data load tool is an open-source Python library for building data pipelines with schema inference and incremental loading. Includes an Airflow helper to convert dlt pipelines into Airflow DAGs/TaskGroups.
Section: Libraries, Hooks, Utilities · data load tool is an open-source Python library for building data pipelines with schema inference and incremental loading. Includes an Airflow helper to convert dlt pipelines into Airflow DAGs/TaskGroups.
Section: Data Integration · Open-source Python library for building data pipelines as code, with schema evolution and incremental loading out of the box.
Section: Data Science and Analytics · data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
Section: Data Ingestion / ETL · A Python library for building data pipelines with automatic schema inference, incremental loading, and support for multiple sources and destinations.
Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
Cloud-native orchestration platform for developing and maintaining data assets including ML models. Declarative programming model with integrated lineage and observability. Apache 2.0 licensed.
Build and share delightful machine learning apps, all in Python. The de facto standard for creating interactive ML demos with automatic UI generation from function signatures. Powers thousands of Hugging Face Spaces.
Workflow management system that makes it easy to take your data pipelines and add semantics like retries, logging, dynamic mapping, caching, failure notifications, and more.
🆓 Connect and query your data sources, build dashboards to visualize data and share them with your company. Owned by Databricks but the hosted SaaS shut down in 2021, so the project is now community-maintained under the Databricks org with no paid Redash product.