Stay organized with collections
Save and categorize content based on your preferences.
Mainframe Connector lets you publish messages to Pub/Sub
from your mainframe. You can use this feature, for example, to integrate your
mainframe with BigQuery or Cloud Storage, augment message data with
Dataflow, or trigger Cloud Run functions.
To run Pub/Sub commands, verify that you assign the
Pub/Sub Publisher
role to your user account. For information about assigning roles to a user
account, see Managing access using IAM.
Replace TOPIC with the name of the Pub/Sub
topic. The required format is projects/{project}/topics/{topic}.
Publish messages to a Pub/Sub topic
Mainframe Connector lets you publish structured messages from a
mainframe to a Pub/Sub topic. For example, you can use this
capability to transfer Queued Sequential Access Method (QSAM) records from your
mainframe to BigQuery using Pub/Sub. The following are
some of the advantages of using this approach:
You can transform data or augment data with AI through Dataflow or
through Pub/Sub simultaneous multithreading (SMTs).
You can move records from a mainframe to Cloud Storage.
You can synchronize mainframe application updates to cloud-based systems
similar to change data capture (CDC).
To publish structured messages from your mainframe to a Pub/Sub
topic, use the following steps:
Create a Pub/Sub topic to capture information about records
that fail to be transcoded. This topic is called a spillover topic. For
information on creating a Pub/Sub topic, see
Create a topic.
(Optional) To convert the records to JSON before sending them to
Pub/Sub, configure the JSON configuration file.
If you don't provide a configuration file, the default JSON configuration is
used.
(Optional) By default, Pub/Sub messages don't contain ordering
keys, and attributes. Spillover messages are sent to the spillover topic
with the _spillover suffix. To change the defaults, you can
configure the Pub/Sub publish configuration file. For example, you can use the following configuration:
Replace SPILLOVER_TOPIC_NAME with the name of the
spillover topic you created in step 2. The required format is
projects/{project}/topics/{topic}. Information about messages
that are not decoded properly is saved to this spillover topic.
To publish mainframe records to Pub/Sub using the
qsam decode command,
run the following JCL:
OUTPUT: the
data path
of the Pub/Sub topic you created in step 1. The required
format is pubsub:projects/{project}/topics/{topic}.
TRANSCODE-CONFIG: the
data path of
the file containing the transcoding configuration. For more information on
the transcoder configuration format, see Transcoder configuration.
JSON_CONFIGURATION: (optional) the
data path of the
JSON configuration file. If
you don't provide a configuration file, the default JSON configuration is
used.
PUBSUB_CONFIGURATION: (optional) the
data path of the
Pub/Sub publish configuration file.
If you don't specify a configuration file, the default Pub/Sub
configuration is used.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-29 UTC."],[],[],null,["# Publish messages to Pub/Sub from your mainframe\n\nMainframe Connector lets you publish messages to Pub/Sub\nfrom your mainframe. You can use this feature, for example, to integrate your\nmainframe with BigQuery or Cloud Storage, augment message data with\nDataflow, or trigger Cloud Run functions.\n\nBefore you begin\n----------------\n\n- If you haven't already, then [install Mainframe Connector on your mainframe](/mainframe-connector/docs/installation).\n- To run Pub/Sub commands, verify that you assign the [Pub/Sub Publisher](/iam/docs/roles-permissions/pubsub#pubsub.publisher) role to your user account. For information about assigning roles to a user account, see [Managing access using IAM](/run/docs/securing/managing-access).\n- Set up [Application Default Credentials](/docs/authentication/provide-credentials-adc).\n\nPublish a message to a Pub/Sub topic\n------------------------------------\n\nTo publish an unstructured Pub/Sub message from your mainframe,\nuse the [`pubsub topics publish` command](/mainframe-connector/docs/api-command-reference#pubsub_topics_publish) as follows: \n\n```\n//STEP01 EXEC BQSH\n//STDIN DD *\npubsub topics publish TOPIC --data=\"Hello,World\"\n/*\n```\n\nReplace \u003cvar translate=\"no\"\u003eTOPIC\u003c/var\u003e with the name of the Pub/Sub\ntopic. The required format is `projects/{project}/topics/{topic}`.\n\nPublish messages to a Pub/Sub topic\n-----------------------------------\n\nMainframe Connector lets you publish structured messages from a\nmainframe to a Pub/Sub topic. For example, you can use this\ncapability to transfer Queued Sequential Access Method (QSAM) records from your\nmainframe to BigQuery using Pub/Sub. The following are\nsome of the advantages of using this approach:\n\n- You can transform data or augment data with AI through Dataflow or through Pub/Sub simultaneous multithreading (SMTs).\n- You can move records from a mainframe to Cloud Storage.\n- You can synchronize mainframe application updates to cloud-based systems similar to change data capture (CDC).\n\nTo publish structured messages from your mainframe to a Pub/Sub\ntopic, use the following steps:\n\n1. Create a Pub/Sub topic to publish messages. Optionally, you can create a topic with a schema for more restrictions. For more information, see [Create a topic](/pubsub/docs/create-topic) or [Create and associate a schema when you create a topic](/pubsub/docs/associate-schema-topic).\n2. Create a Pub/Sub topic to capture information about records that fail to be transcoded. This topic is called a spillover topic. For information on creating a Pub/Sub topic, see [Create a topic](/pubsub/docs/create-topic).\n3. (Optional) To convert the records to JSON before sending them to Pub/Sub, configure the [JSON configuration file](/mainframe-connector/docs/json-configuration). If you don't provide a configuration file, the default JSON configuration is used.\n4. (Optional) By default, Pub/Sub messages don't contain ordering\n keys, and attributes. Spillover messages are sent to the spillover topic\n with the `_spillover` suffix. To change the defaults, you can\n configure the [Pub/Sub publish configuration file](/mainframe-connector/docs/pubsub-publish-configuration). For example, you can use the following configuration:\n\n ```\n {\n \"spillover-configuration\": {\n \"name\": SPILLOVER_TOPIC_NAME\n }\n }\n ```\n\n Replace \u003cvar translate=\"no\"\u003eSPILLOVER_TOPIC_NAME\u003c/var\u003e with the name of the\n spillover topic you created in step 2. The required format is\n `projects/{project}/topics/{topic}`. Information about messages\n that are not decoded properly is saved to this spillover topic.\n5. To publish mainframe records to Pub/Sub using the\n [`qsam decode` command](/mainframe-connector/docs/api-command-reference#qsam_decode),\n run the following JCL:\n\n //STEP01 EXEC BQSH\n //STDIN DD *\n qsam decode \u003cvar translate=\"no\"\u003eINPUT\u003c/var\u003e \u003cvar translate=\"no\"\u003eOUTPUT\u003c/var\u003e \\\n --copybook \u003cvar translate=\"no\"\u003eCOPYBOOK\u003c/var\u003e\n --transcode-configuration \u003cvar translate=\"no\"\u003eTRANSCODE-CONFIG\u003c/var\u003e\n --output-format=JSONL\n --input-parameter=json-dialect=\u003cvar translate=\"no\"\u003eJSON_CONFIGURATION\u003c/var\u003e\n --input-parameter=pubsub-publish-configuration=\u003cvar translate=\"no\"\u003ePUBSUB_CONFIGURATION\u003c/var\u003e\n /*\n\n Replace the following:\n - \u003cvar translate=\"no\"\u003eINPUT\u003c/var\u003e: the [data path](/mainframe-connector/docs/api-command-reference#DataPath) of the QSAM file.\n - \u003cvar translate=\"no\"\u003eOUTPUT\u003c/var\u003e: the [data path](/mainframe-connector/docs/api-command-reference#DataPath) of the Pub/Sub topic you created in step 1. The required format is `pubsub:projects/{project}/topics/{topic}`.\n - \u003cvar translate=\"no\"\u003eTRANSCODE-CONFIG\u003c/var\u003e: the [data path](/mainframe-connector/docs/api-command-reference#DataPath) of the file containing the transcoding configuration. For more information on the transcoder configuration format, see [Transcoder configuration](/mainframe-connector/docs/transcoder-configuration).\n - \u003cvar translate=\"no\"\u003eJSON_CONFIGURATION\u003c/var\u003e: (optional) the [data path](/mainframe-connector/docs/api-command-reference#DataPath) of the [JSON configuration file](/mainframe-connector/docs/json-configuration). If you don't provide a configuration file, the default JSON configuration is used.\n - \u003cvar translate=\"no\"\u003ePUBSUB_CONFIGURATION\u003c/var\u003e: (optional) the [data path](/mainframe-connector/docs/api-command-reference#DataPath) of the [Pub/Sub publish configuration file](/mainframe-connector/docs/pubsub-publish-configuration). If you don't specify a configuration file, the default Pub/Sub configuration is used."]]