Orchestrating TFX Pipelines
Stay organized with collections
Save and categorize content based on your preferences.
Apache Airflow
Apache Airflow is a platform to
programmatically author, schedule and monitor workflows. TFX uses Airflow to
author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow
scheduler executes tasks on an array of workers while following the specified
dependencies. Rich command line utilities make performing complex surgeries on
DAGs a snap. The rich user interface makes it easy to visualize pipelines
running in production, monitor progress, and troubleshoot issues when needed.
When workflows are defined as code, they become more maintainable, versionable,
testable, and collaborative.
See the Apache Airflow for details on installing
and using Apache Airflow.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2021-01-28 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2021-01-28 UTC."],[],[],null,["# Orchestrating TFX Pipelines\n\n\u003cbr /\u003e\n\nApache Airflow\n--------------\n\n[Apache Airflow](https://airflow.apache.org/) is a platform to\nprogrammatically author, schedule and monitor workflows. TFX uses Airflow to\nauthor workflows as directed acyclic graphs (DAGs) of tasks. The Airflow\nscheduler executes tasks on an array of workers while following the specified\ndependencies. Rich command line utilities make performing complex surgeries on\nDAGs a snap. The rich user interface makes it easy to visualize pipelines\nrunning in production, monitor progress, and troubleshoot issues when needed.\nWhen workflows are defined as code, they become more maintainable, versionable,\ntestable, and collaborative.\n\nSee the [Apache Airflow](https://airflow.apache.org/) for details on installing\nand using Apache Airflow."]]
