#55020 gantt view is not getting updatedin realtime #55130

yimingpeng · 2025-09-01T09:39:38Z

Hello there,

This PR is an attempt to fix #55020 where the gantt chart wasn't showing real-time progress for running tasks.

After applying the fix:

Screen.Recording.2025-09-01.at.9.18.13.PM.mov

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

boring-cyborg · 2025-09-01T09:39:41Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

…edin-realtime

guan404ming

Thanks for fix, left some comments~

guan404ming · 2025-09-01T10:28:59Z

airflow-core/src/airflow/ui/src/layouts/Details/Gantt/Gantt.tsx

@@ -123,6 +124,20 @@ export const Gantt = ({ limit }: Props) => {

  const isLoading = runsLoading || structureLoading || summariesLoading || tiLoading;

+  useEffect(() => {


taskInstancesData would auto-refresh here so I think it could work without this useEffect.

airflow/airflow-core/src/airflow/ui/src/layouts/Details/Gantt/Gantt.tsx

Lines 109 to 120 in 2b91be8

const { data: taskInstancesData, isLoading: tiLoading } = useTaskInstanceServiceGetTaskInstances(

{

dagId,

dagRunId: runId ?? "~",

},

undefined,

{

enabled: Boolean(dagId),

refetchInterval: (query) =>

query.state.data?.task_instances.some((ti) => isStatePending(ti.state)) ? refetchInterval : false,

},

);

guan404ming · 2025-09-01T10:37:55Z

airflow-core/src/airflow/ui/src/layouts/Details/Gantt/Gantt.tsx

+
+    if (hasRunningTasks) {
+      const interval = setInterval(() => {
+        setCurrentTime(new Date().toISOString());


We could get current time with selectedTimezone easily with dayjs

const { selectedTimezone } = useTimezone(); // Make the value timezone-aware setCurrentTime(dayjs().tz(selectedTimezone).format("YYYY-MM-DD HH:mm:ss"))

guan404ming · 2025-09-01T10:53:18Z

airflow-core/src/airflow/ui/src/layouts/Details/Gantt/utils.ts

-    const target = event.native?.target as HTMLElement | undefined;
+}: ChartOptionsParams) => {
+  const isRunning = selectedRun?.state === "running";
+  const effectiveEndDate = isRunning ? new Date().toISOString() : selectedRun?.end_date;


…g-updatedin-realtime

…essed review comments

yimingpeng · 2025-09-02T11:16:06Z

Hello @guan404ming,

Thanks for all the great suggestions, I believe I have addressed them, also made a few more changes, e..g, expanding to other pending tasks (isStatePending) which was checking running only. Please help review again, thanks.

Here is the expanded test dag from the original issue post:

from datetime import datetime, timedelta

from airflow import DAG
from airflow.providers.standard.operators.bash import BashOperator
from airflow.providers.standard.operators.python import PythonOperator
from airflow.providers.standard.sensors.filesystem import FileSensor
from airflow.utils.task_group import TaskGroup

def task_that_fails():
    raise Exception("This task always fails for testing")

def task_that_succeeds():
    print("This task succeeds")
    return "success"

dag = DAG(
    'test_gantt_chart_dag',
    start_date=datetime(2025, 7, 1),
    schedule=None,
    is_paused_upon_creation=False,
    catchup=False,
    max_active_runs=1,
    default_args={
        'retries': 2,
        'retry_delay': timedelta(seconds=10),
    }
)

long_running_task = BashOperator(
    task_id="long_running_task",
    bash_command="echo 'Starting long task'; sleep 180; echo 'Long task completed'",
    dag=dag
)

queued_task = BashOperator(
    task_id="queued_task", 
    bash_command="echo 'Queued task executed'; sleep 30",
    dag=dag,
    pool='default_pool',
    pool_slots=10
)

failing_task = PythonOperator(
    task_id="failing_task",
    python_callable=task_that_fails,
    dag=dag
)

waiting_sensor = FileSensor(
    task_id="waiting_sensor",
    filepath="/tmp/nonexistent_file_for_testing.txt",
    timeout=300,
    poke_interval=30,
    dag=dag
)

scheduled_task = BashOperator(
    task_id="scheduled_task",
    bash_command="echo 'Scheduled task executed'; sleep 15",
    dag=dag
)

quick_success_task = PythonOperator(
    task_id="quick_success_task",
    python_callable=task_that_succeeds,
    dag=dag
)

upstream_failed_task = BashOperator(
    task_id="upstream_failed_task",
    bash_command="echo 'This should not run due to upstream failure'",
    dag=dag
)

with TaskGroup("processing_group", dag=dag) as processing_group:
    
    group_start = BashOperator(
        task_id="group_start",
        bash_command="echo 'Group starting'; sleep 5",
        dag=dag,
    )
    
    parallel_task_1 = BashOperator(
        task_id="parallel_task_1", 
        bash_command="echo 'Parallel task 1 running'; sleep 60",
        dag=dag,
    )
    
    parallel_task_2 = PythonOperator(
        task_id="parallel_task_2",
        python_callable=task_that_fails,
        dag=dag,
    )
    
    parallel_task_3 = BashOperator(
        task_id="parallel_task_3",
        bash_command="echo 'Parallel task 3 running'; sleep 45",
        dag=dag,
    )
    
    group_end = BashOperator(
        task_id="group_end",
        bash_command="echo 'Group completed'; sleep 10",
        trigger_rule="none_failed_min_one_success",
        dag=dag,
    )
    
    group_start >> [parallel_task_1, parallel_task_2, parallel_task_3] >> group_end

final_task = BashOperator(
    task_id="final_task",
    bash_command="echo 'All processing completed'",
    dag=dag
)

long_running_task >> scheduled_task
quick_success_task >> queued_task
failing_task >> upstream_failed_task
waiting_sensor >> scheduled_task
processing_group >> final_task

Here is the new recording after the fix is applied:

Screen.Recording.2025-09-02.at.11.13.34.PM.mov

guan404ming

Overall looks good, one nit

guan404ming · 2025-09-03T15:45:51Z

airflow-core/src/airflow/ui/src/layouts/Details/Gantt/Gantt.tsx

@@ -33,6 +33,9 @@ import {
 import "chart.js/auto";
 import "chartjs-adapter-dayjs-4/dist/chartjs-adapter-dayjs-4.esm";
 import annotationPlugin from "chartjs-plugin-annotation";
+import dayjs from "dayjs";
+import timezone from "dayjs/plugin/timezone";
+import utc from "dayjs/plugin/utc";


Is there any special reason for using this?

oh, good catch, those imports are redundant, as well as L58-L60 down below. Sorry, not very familiar with dayjs so missed that. Fixing it now, thanks for the review!

…g-updatedin-realtime

yimingpeng added 4 commits August 31, 2025 21:17

Fix real-time updates for running tasks in Gantt chart view

501b39f

Fix trailing whitespaces

4f63d89

Prek auto-fix

d9d88c3

Fix prek ui-linting

172aec5

yimingpeng requested review from bbovenzi, pierrejeambrun, ryanahamilton, jscheffl and shubhamraj-git as code owners September 1, 2025 09:39

boring-cyborg bot added the area:UI Related to UI/UX. For Frontend Developers. label Sep 1, 2025

Merge branch 'main' into apache#55020-gantt-view-is-not-getting-updat…

2baf818

…edin-realtime

guan404ming reviewed Sep 1, 2025

View reviewed changes

Yiming Peng and others added 2 commits September 2, 2025 23:06

Merge branch 'apache:main' into apache#55020-gantt-view-is-not-gettin…

54e8321

…g-updatedin-realtime

fixed real-time updates for all pending states in gantt view and addr…

a54f920

…essed review comments

guan404ming approved these changes Sep 3, 2025

View reviewed changes

Yiming Peng and others added 3 commits September 4, 2025 14:21

Merge branch 'apache:main' into apache#55020-gantt-view-is-not-gettin…

0eddd62

…g-updatedin-realtime

Merge branch 'apache:main' into apache#55020-gantt-view-is-not-gettin…

bb07e42

…g-updatedin-realtime

Remove redundant imports

bd88e6b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

#55020 gantt view is not getting updatedin realtime #55130

#55020 gantt view is not getting updatedin realtime #55130

Uh oh!

yimingpeng commented Sep 1, 2025

Uh oh!

boring-cyborg bot commented Sep 1, 2025

Uh oh!

guan404ming left a comment

Uh oh!

guan404ming Sep 1, 2025

Uh oh!

guan404ming Sep 1, 2025

Uh oh!

guan404ming Sep 1, 2025

Uh oh!

yimingpeng commented Sep 2, 2025

Uh oh!

guan404ming left a comment

Uh oh!

guan404ming Sep 3, 2025

Uh oh!

yimingpeng Sep 4, 2025

Uh oh!

Uh oh!

		@@ -123,6 +124,20 @@ export const Gantt = ({ limit }: Props) => {

		const isLoading = runsLoading \|\| structureLoading \|\| summariesLoading \|\| tiLoading;

		useEffect(() => {

	const { data: taskInstancesData, isLoading: tiLoading } = useTaskInstanceServiceGetTaskInstances(
	{
	dagId,
	dagRunId: runId ?? "~",
	},
	undefined,
	{
	enabled: Boolean(dagId),
	refetchInterval: (query) =>
	query.state.data?.task_instances.some((ti) => isStatePending(ti.state)) ? refetchInterval : false,
	},
	);

#55020 gantt view is not getting updatedin realtime #55130

Are you sure you want to change the base?

#55020 gantt view is not getting updatedin realtime #55130

Uh oh!

Conversation

yimingpeng commented Sep 1, 2025

Uh oh!

boring-cyborg bot commented Sep 1, 2025

Uh oh!

guan404ming left a comment

Choose a reason for hiding this comment

Uh oh!

guan404ming Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

guan404ming Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

guan404ming Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

yimingpeng commented Sep 2, 2025

Uh oh!

guan404ming left a comment

Choose a reason for hiding this comment

Uh oh!

guan404ming Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

yimingpeng Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!