Skip to content

Commit

Permalink
Edit FAQ to reference Airflow 2 as a solution to latency problem, closes
Browse files Browse the repository at this point in the history
  • Loading branch information
ryw authored Nov 18, 2020
1 parent 0080354 commit 0da73e3
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ How to reduce airflow dag scheduling latency in production?
-----------------------------------------------------------

- ``max_threads``: Scheduler will spawn multiple threads in parallel to schedule dags. This is controlled by ``max_threads`` with default value of 2. User should increase this value to a larger value (e.g numbers of cpus where scheduler runs - 1) in production.
- ``scheduler_heartbeat_sec``: User should consider to increase ``scheduler_heartbeat_sec`` config to a higher value (e.g 60 secs) which controls how frequent the airflow scheduler gets the heartbeat and updates the job's entry in database.
- If you're using Airflow 1.10.x, consider moving to Airflow 2, which has reduced dag scheduling latency dramatically, and allows for running multiple schedulers.

Why next_ds or prev_ds might not contain expected values?
---------------------------------------------------------
Expand Down

0 comments on commit 0da73e3

Please sign in to comment.