Stay organized with collections
Save and categorize content based on your preferences.
This page describes how to resolve a known issue in Cloud Data Fusion 6.8.0 and
6.8.1 where a data pipeline fails with an unsupported program type error in
Cloud Data Fusion. This issue is resolved in version 6.8.2.
To reduce the start time for pipelines, Cloud Data Fusion version 6.8.0 and
6.8.1 instances cache the artifacts that are required to start a pipeline in a
Dataproc cluster inside a Cloud Storage bucket.
One of these cached artifacts is application.jar. Depending on the order in
which you run your pipelines, some pipelines might fail with the following
error:
Unsupportedprogramtype:Spark
For example, after you create a new 6.8.1 instance (or upgrade to 6.8.1), the
first time that you run a pipeline that only contains actions, it succeeds.
However, the next pipeline runs, which include sources or sinks, might fail with
this error.
Recommendation
To resolve this issue, do either of the following:
Click
expand_more
Expand next to Run and set the runtime argument
system.profile.properties.gcsCacheEnabled to false.
Repeat for any other failing pipelines.
REST API
Cloud Storage caching can be disabled when starting a pipeline through REST
API and also by optionally specifying runtime arguments as a JSON map in the
request body. For more information, see
Start a program.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-29 UTC."],[[["\u003cp\u003eCloud Data Fusion versions 6.8.0 and 6.8.1 have a known issue where pipelines may fail with an "Unsupported program type: Spark" error due to artifact caching.\u003c/p\u003e\n"],["\u003cp\u003eThe recommended solution is to upgrade the Cloud Data Fusion instance to version 6.8.2 or later to resolve this pipeline error.\u003c/p\u003e\n"],["\u003cp\u003eAs an alternative solution, you can disable Cloud Storage caching, which can be done for all pipelines in an instance, for a specific namespace, for a Dataproc profile, or for individual failing pipelines.\u003c/p\u003e\n"],["\u003cp\u003eDisabling Cloud Storage caching will result in slightly longer pipeline start times, as fewer artifacts are cached.\u003c/p\u003e\n"],["\u003cp\u003eCaching can be disabled via the system admin console for all options, or via the REST API.\u003c/p\u003e\n"]]],[],null,["# Troubleshoot unsupported program type errors\n\nThis page describes how to resolve a known issue in Cloud Data Fusion 6.8.0 and 6.8.1 where a data pipeline fails with an unsupported program type error in Cloud Data Fusion. This issue is resolved in version 6.8.2.\n\n\u003cbr /\u003e\n\nTo reduce the start time for pipelines, Cloud Data Fusion version 6.8.0 and\n6.8.1 instances cache the artifacts that are required to start a pipeline in a\nDataproc cluster inside a Cloud Storage bucket.\nOne of these cached artifacts is `application.jar`. Depending on the order in\nwhich you run your pipelines, some pipelines might fail with the following\nerror: \n\n Unsupported program type: Spark\n\nFor example, after you create a new 6.8.1 instance (or upgrade to 6.8.1), the\nfirst time that you run a pipeline that only contains actions, it succeeds.\nHowever, the next pipeline runs, which include sources or sinks, might fail with\nthis error.\n\nRecommendation\n--------------\n\nTo resolve this issue, do either of the following:\n\n- Recommended: [Upgrade the instance](/data-fusion/docs/how-to/upgrading#upgrade-instances) to Cloud Data Fusion version 6.8.2 or later.\n- Disable Cloud Storage caching by a [preference or runtime argument](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/480314690/Preferences+and+Runtime+Arguments).\n\n| **Note:** Disabling Cloud Storage caching results in a pipeline taking slightly longer to start running, as fewer artifacts are cached.\n\nYou can disable caching for any of the following:\n\n- For all pipelines in an instance.\n- For a given namespace.\n- For the specific Dataproc profiles that contain the failing pipelines.\n- For only the failing pipelines.\n\n### Disable Cloud Storage caching for all pipelines in an instance\n\nTo disable Cloud Storage caching for all pipelines in an instance,\nfollow these steps: \n\n### Console\n\n1. Go to your instance:\n 1. In the Google Cloud console, go to the Cloud Data Fusion page.\n\n 2. To open the instance in the Cloud Data Fusion Studio,\n click **Instances** , and then click **View instance**.\n\n [Go to Instances](https://console.cloud.google.com/data-fusion/locations/-/instances)\n2. Click **System Admin** \\\u003e **System Preferences** and set the value for\n `system.profile.properties.gcsCacheEnabled` to `false.`\n\n### REST API\n\n\nTo set `system.profile.properties.gcsCacheEnabled` to `false`, see\n[Set preferences](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/477561058/Preferences+Microservices#Set-Preferences).\n| **Note:** This change impacts start time for all pipelines in the instance.\n\n### Disable Cloud Storage caching for a given namespace\n\nTo disable Cloud Storage caching for a given namespace, follow these\nsteps: \n\n### Console\n\n1. Go to your instance:\n 1. In the Google Cloud console, go to the Cloud Data Fusion page.\n\n 2. To open the instance in the Cloud Data Fusion Studio,\n click **Instances** , and then click **View instance**.\n\n [Go to Instances](https://console.cloud.google.com/data-fusion/locations/-/instances)\n2. Click **System Admin** \\\u003e **Namespaces** and select your namespace.\n3. Click **Preferences** \\\u003e **Edit** and set the value for\n `system.profile.properties.gcsCacheEnabled` to `false`.\n\n### REST API\n\n\nTo set this through the REST API, see\n[Set preferences](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/477561058/Preferences+Microservices#Set-Preferences).\n| **Note:** This change impacts start time for all pipelines in the namespace.\n\n### Disable Cloud Storage caching for a Dataproc profile\n\nTo disable Cloud Storage caching for the specific\nDataproc profiles that contain the failing pipelines, follow\nthese steps: \n\n### Console\n\n- Set `gcsCacheEnabled` to `false` in the Dataproc profile.\n| **Note:** This change impacts the start time for all pipelines using this Dataproc profile.\n\n### Disable Cloud Storage caching for only the failing pipelines\n\nTo disable Cloud Storage caching for only the failing pipelines, follow\nthese steps: \n\n### Console\n\n1. Go to your instance:\n 1. In the Google Cloud console, go to the Cloud Data Fusion page.\n\n 2. To open the instance in the Cloud Data Fusion Studio,\n click **Instances** , and then click **View instance**.\n\n [Go to Instances](https://console.cloud.google.com/data-fusion/locations/-/instances)\n2. Click **List** and select the failing pipeline.\n3. Click expand_more **Expand** next to **Run** and set the runtime argument `system.profile.properties.gcsCacheEnabled` to `false`.\n4. Repeat for any other failing pipelines.\n\n### REST API\n\n\nCloud Storage caching can be disabled when starting a pipeline through REST\nAPI and also by optionally specifying runtime arguments as a JSON map in the\nrequest body. For more information, see\n[Start a program](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/477560983/Lifecycle+Microservices#Start-a-Program).\n| **Note:** This change impacts the start time of only specific pipelines."]]
