Command Line InterfaceΒΆ

Airflow has a very rich command line interface that allows for many types of operation on a DAG, starting services, and supporting development and testing.

usage: airflow [-h]
               {resetdb,render,variables,pause,version,initdb,test,unpause,dag_state,run,list_tasks,backfill,list_dags,kerberos,worker,webserver,flower,scheduler,task_state,trigger_dag,serve_logs,clear,upgradedb}
               ...
Sub-commands:
resetdb

Burn down and rebuild the metadata database

usage: airflow resetdb [-h] [-y]
Options:
-y=False, --yes=False
 Do not prompt to confirm reset. Use with care!
render

Render a task instance’s template(s)

usage: airflow render [-h] [-sd SUBDIR] dag_id task_id execution_date
Positional arguments:
dag_id The id of the dag
task_id The id of the task
execution_date The execution date of the DAG
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
variables

List all variables

usage: airflow variables [-h] [-s KEY VAL] [-g KEY] [-j] [-d VAL]
                         [-i FILEPATH] [-e FILEPATH] [-x KEY]
Options:
-s, --set Set a variable
-g, --get Get value of a variable
-j=False, --json=False
 Deserialize JSON variable
-d, --default Default value returned if variable does not exist
-i, --import Import variables from JSON file
-e, --export Export variables to JSON file
-x, --delete Delete a variable
pause

Pause a DAG

usage: airflow pause [-h] [-sd SUBDIR] dag_id
Positional arguments:
dag_id The id of the dag
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
version

Show the version

usage: airflow version [-h]
initdb

Initialize the metadata database

usage: airflow initdb [-h]
test

Test a task instance. This will run a task without checking for dependencies or recording it’s state in the database.

usage: airflow test [-h] [-sd SUBDIR] [-dr] [-tp TASK_PARAMS]
                    dag_id task_id execution_date
Positional arguments:
dag_id The id of the dag
task_id The id of the task
execution_date The execution date of the DAG
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
-dr=False, --dry_run=False
 Perform a dry run
-tp, --task_params
 Sends a JSON params dict to the task
unpause

Resume a paused DAG

usage: airflow unpause [-h] [-sd SUBDIR] dag_id
Positional arguments:
dag_id The id of the dag
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
dag_state

Get the status of a dag run

usage: airflow dag_state [-h] [-sd SUBDIR] dag_id execution_date
Positional arguments:
dag_id The id of the dag
execution_date The execution date of the DAG
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
run

Run a single task instance

usage: airflow run [-h] [-sd SUBDIR] [-m] [-f] [--pool POOL] [-l] [-i] [-I]
                   [--ship_dag] [-p PICKLE]
                   dag_id task_id execution_date
Positional arguments:
dag_id The id of the dag
task_id The id of the task
execution_date The execution date of the DAG
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
-m=False, --mark_success=False
 Mark jobs as succeeded without running them
-f=False, --force=False
 Force a run regardless of previous success
--pool Resource pool to use
-l=False, --local=False
 Run the task using the LocalExecutor
-i=False, --ignore_dependencies=False
 Ignore upstream and depends_on_past dependencies
-I=False, --ignore_depends_on_past=False
 Ignore depends_on_past dependencies (but respect upstream dependencies)
--ship_dag=False
 Pickles (serializes) the DAG and ships it to the worker
-p, --pickle Serialized pickle object of the entire dag (used internally)
list_tasks

List the tasks within a DAG

usage: airflow list_tasks [-h] [-t] [-sd SUBDIR] dag_id
Positional arguments:
dag_id The id of the dag
Options:
-t=False, --tree=False
 Tree view
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
backfill

Run subsections of a DAG for a specified date range

usage: airflow backfill [-h] [-t TASK_REGEX] [-s START_DATE] [-e END_DATE]
                        [-m] [-l] [-x] [-a] [-i] [-I] [-sd SUBDIR]
                        [--pool POOL] [-dr]
                        dag_id
Positional arguments:
dag_id The id of the dag
Options:
-t, --task_regex
 The regex to filter specific task_ids to backfill (optional)
-s, --start_date
 Override start_date YYYY-MM-DD
-e, --end_date Override end_date YYYY-MM-DD
-m=False, --mark_success=False
 Mark jobs as succeeded without running them
-l=False, --local=False
 Run the task using the LocalExecutor
-x=False, --donot_pickle=False
 Do not attempt to pickle the DAG object to send over to the workers, just tell the workers to run their version of the code.
-a=False, --include_adhoc=False
 Include dags with the adhoc parameter.
-i=False, --ignore_dependencies=False
 Skip upstream tasks, run only the tasks matching the regexp. Only works in conjunction with task_regex
-I=False, --ignore_first_depends_on_past=False
 Ignores depends_on_past dependencies for the first set of tasks only (subsequent executions in the backfill DO respect depends_on_past).
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
--pool Resource pool to use
-dr=False, --dry_run=False
 Perform a dry run
list_dags

List all the DAGs

usage: airflow list_dags [-h] [-sd SUBDIR] [-r]
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
-r=False, --report=False
 Show DagBag loading report
kerberos

Start a kerberos ticket renewer

usage: airflow kerberos [-h] [-kt [KEYTAB]] [--pid [PID]] [-D]
                        [--stdout STDOUT] [--stderr STDERR] [-l LOG_FILE]
                        [principal]
Positional arguments:
principal kerberos principal
Options:
-kt=airflow.keytab, --keytab=airflow.keytab
 keytab
--pid PID file location
-D=False, --daemon=False
 Daemonize instead of running in the foreground
--stdout Redirect stdout to this file
--stderr Redirect stderr to this file
-l, --log-file Location of the log file
worker

Start a Celery worker node

usage: airflow worker [-h] [-p] [-q QUEUES] [-c CONCURRENCY] [--pid [PID]]
                      [-D] [--stdout STDOUT] [--stderr STDERR] [-l LOG_FILE]
Options:
-p=False, --do_pickle=False
 Attempt to pickle the DAG object to send over to the workers, instead of letting workers run their version of the code.
-q=default, --queues=default
 Comma delimited list of queues to serve
-c=16, --concurrency=16
 The number of worker processes
--pid PID file location
-D=False, --daemon=False
 Daemonize instead of running in the foreground
--stdout Redirect stdout to this file
--stderr Redirect stderr to this file
-l, --log-file Location of the log file
webserver

Start a Airflow webserver instance

usage: airflow webserver [-h] [-p PORT] [-w WORKERS]
                         [-k {sync,eventlet,gevent,tornado}]
                         [-t WORKER_TIMEOUT] [-hn HOSTNAME] [--pid [PID]] [-D]
                         [--stdout STDOUT] [--stderr STDERR]
                         [-A ACCESS_LOGFILE] [-E ERROR_LOGFILE] [-l LOG_FILE]
                         [-d]
Options:
-p=8080, --port=8080
 The port on which to run the server
-w=4, --workers=4
 Number of workers to run the webserver on
-k=sync, --workerclass=sync
 

The worker class to use for Gunicorn

Possible choices: sync, eventlet, gevent, tornado

-t=120, --worker_timeout=120
 The timeout for waiting on webserver workers
-hn=0.0.0.0, --hostname=0.0.0.0
 Set the hostname on which to run the web server
--pid PID file location
-D=False, --daemon=False
 Daemonize instead of running in the foreground
--stdout Redirect stdout to this file
--stderr Redirect stderr to this file
-A, --access_logfile
 The logfile to store the webserver access log. Use ‘-‘ to print to stderr.
-E, --error_logfile
 The logfile to store the webserver error log. Use ‘-‘ to print to stderr.
-l, --log-file Location of the log file
-d=False, --debug=False
 Use the server that ships with Flask in debug mode
flower

Start a Celery Flower

usage: airflow flower [-h] [-hn HOSTNAME] [-p PORT] [-fc FLOWER_CONF]
                      [-a BROKER_API] [--pid [PID]] [-D] [--stdout STDOUT]
                      [--stderr STDERR] [-l LOG_FILE]
Options:
-hn=0.0.0.0, --hostname=0.0.0.0
 Set the hostname on which to run the server
-p=5555, --port=5555
 The port on which to run the server
-fc, --flower_conf
 Configuration file for flower
-a, --broker_api
 Broker api
--pid PID file location
-D=False, --daemon=False
 Daemonize instead of running in the foreground
--stdout Redirect stdout to this file
--stderr Redirect stderr to this file
-l, --log-file Location of the log file
scheduler

Start a scheduler instance

usage: airflow scheduler [-h] [-d DAG_ID] [-sd SUBDIR] [-r RUN_DURATION]
                         [-n NUM_RUNS] [-p] [--pid [PID]] [-D]
                         [--stdout STDOUT] [--stderr STDERR] [-l LOG_FILE]
Options:
-d, --dag_id The id of the dag to run
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
-r, --run-duration
 Set number of seconds to execute before exiting
-n, --num_runs Set the number of runs to execute before exiting
-p=False, --do_pickle=False
 Attempt to pickle the DAG object to send over to the workers, instead of letting workers run their version of the code.
--pid PID file location
-D=False, --daemon=False
 Daemonize instead of running in the foreground
--stdout Redirect stdout to this file
--stderr Redirect stderr to this file
-l, --log-file Location of the log file
task_state

Get the status of a task instance

usage: airflow task_state [-h] [-sd SUBDIR] dag_id task_id execution_date
Positional arguments:
dag_id The id of the dag
task_id The id of the task
execution_date The execution date of the DAG
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
trigger_dag

Trigger a DAG run

usage: airflow trigger_dag [-h] [-sd SUBDIR] [-r RUN_ID] [-c CONF] dag_id
Positional arguments:
dag_id The id of the dag
Options:
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
-r, --run_id Helps to identify this run
-c, --conf JSON string that gets pickled into the DagRun’s conf attribute
serve_logs

Serve logs generate by worker

usage: airflow serve_logs [-h]
clear

Clear a set of task instance, as if they never ran

usage: airflow clear [-h] [-t TASK_REGEX] [-s START_DATE] [-e END_DATE]
                     [-sd SUBDIR] [-u] [-d] [-c] [-f] [-r] [-x]
                     dag_id
Positional arguments:
dag_id The id of the dag
Options:
-t, --task_regex
 The regex to filter specific task_ids to backfill (optional)
-s, --start_date
 Override start_date YYYY-MM-DD
-e, --end_date Override end_date YYYY-MM-DD
-sd=/Users/maxime_beauchemin/dags, --subdir=/Users/maxime_beauchemin/dags
 File location or directory from which to look for the dag
-u=False, --upstream=False
 Include upstream tasks
-d=False, --downstream=False
 Include downstream tasks
-c=False, --no_confirm=False
 Do not request confirmation
-f=False, --only_failed=False
 Only failed jobs
-r=False, --only_running=False
 Only running jobs
-x=False, --exclude_subdags=False
 Exclude subdags
upgradedb

Upgrade the metadata database to latest version

usage: airflow upgradedb [-h]