Mainframe Connector moves data from IBM mainframes to Google Cloud and can be controlled directly from existing JCL batch jobs. Reporting workloads that currently tie up mainframe CPU time can be offloaded to the cloud without giving up familiar job scheduling.
What is Mainframe Connector?
Per Google Cloud documentation, IBM mainframes are used by organizations to perform critical computing tasks; many of these organizations have been working toward migrating to the cloud in recent years. Mainframe Connector lets you move mainframe data to Google Cloud so that you can offload CPU-intensive report workloads to Google Cloud.
How it works
Mainframe Connector lets you move data located on your mainframe in and out of Cloud Storage, and submit BigQuery jobs from mainframe-based batch jobs defined in Job Control Language (JCL). Mainframe Connector transcodes mainframe datasets directly to Optimized Row Columnar (ORC) format, an open-source, column-oriented data format widely used in the Apache Hadoop ecosystem and supported by BigQuery.
The service provides a subset of the Google Cloud SDK command-line utilities. JVM-based implementations of gsutil and the bq command-line utility make it possible to manage a complete extract, load, transform (ELT) pipeline entirely from IBM z/OS while retaining your existing job scheduler.
Per Google, one of the main challenges in transferring mainframe data to and from the cloud is that it’s normally a multi-step process: copying data to a file server, copying it from there to another location for processing, using a data processing stack to convert it to a modern format, and finally loading the processed data into a database or data warehouse. Mainframe Connector lets you perform all these steps with a single command, using Cloud Storage as an intermediate storage location, which reduces the time taken for mainframe data to become available in a database or data warehouse.
Reducing mainframe load (MIPS)
For most of its processing, Mainframe Connector uses a Java Virtual Machine (JVM) to minimize mainframe processor workload during data transfer, thereby reducing million instructions per second (MIPS) and lowering costs. Most of the processor-intensive work is offloaded to auxiliary processors. If those auxiliary processors are stressed, Mainframe Connector can also be configured to perform transcoding and conversion using Compute Engine.
Supported formats and character sets
Mainframe Connector supports transcoding of Queued Sequential Access Method (QSAM) or Virtual Storage Access Method (VSAM) mainframe datasets associated with COBOL copybooks in Extended Binary Coded Decimal Interchange Code (EBCDIC), as well as files in ASCII UTF-8, into the ORC, JSON, or CSV target formats. By default, Mainframe Connector transcodes datasets from the US EBCDIC Cp037 character set; it also supports the regional EBCDIC character sets French (Cp297), German (Cp1141), and Spanish (Cp1145). A custom character set can be implemented if an appropriate one is not included in the IBM JVM.
Core Features
- Simplified data transfer: Moves mainframe data directly to Cloud Storage and BigQuery.
- Batch job integration: BigQuery jobs can be submitted from existing JCL batch jobs.
- Familiar monitoring: Mainframe operations teams continue to monitor via familiar JCL job scheduling.
- Reduced MIPS: JVM-based processing and offloading to auxiliary processors lower mainframe processor load.
- Streaming transcoding: Direct conversion of mainframe datasets to ORC, JSON, or CSV, including regional EBCDIC character sets.
Typical Use Cases
Offloading reporting workloads
A company moves compute-intensive report batch jobs from the mainframe to BigQuery to reduce MIPS costs and free up mainframe capacity.
ELT pipelines directly from z/OS
A team runs a complete extract-load-transform pipeline using the JVM-based gsutil and bq implementations directly from z/OS, without replacing the existing job scheduler.
Processing COBOL copybook datasets
QSAM or VSAM datasets with COBOL copybooks are automatically transcoded to ORC and become available for analysis in BigQuery.
Multilingual mainframe environments
For datasets in regional EBCDIC character sets, such as German (Cp1141), a European company uses Mainframe Connector’s corresponding character set support.
Benefits
- Lower migration effort: A single command replaces a multi-step copy, process, and load chain.
- Cost control: Reduced MIPS usage through JVM-based processing lowers mainframe operating costs.
- Familiar operations: Integrates with existing JCL job scheduling without a new operating environment.
- Direct BigQuery connection: Mainframe data becomes available for analysis promptly.
- Multilingual character set support: Regional EBCDIC character sets for German, French, and Spanish are already built in.
Integration with innFactory
As a certified Google Cloud partner, innFactory helps you migrate mainframe workloads: assessing which report and batch processes are suited for offloading, designing the target architecture in Cloud Storage and BigQuery, and integrating Mainframe Connector into your existing JCL job scheduling.
Contact us for a consultation on Mainframe Connector.
Typical Use Cases
Technical Specifications
Frequently Asked Questions
What is Mainframe Connector?
Per Google Cloud documentation, Mainframe Connector lets you move mainframe data to Google Cloud so you can offload CPU-intensive report workloads to the cloud. It is aimed at organizations migrating from IBM mainframes toward the cloud.
How does Mainframe Connector reduce mainframe utilization (MIPS)?
Per the documentation, Mainframe Connector uses a Java Virtual Machine (JVM) for most processing to minimize mainframe processor workload during data transfer, thereby reducing MIPS and lowering costs. Most processor-intensive work is offloaded to auxiliary processors; if these are stressed, Mainframe Connector can also be configured to perform transcoding and conversion using Compute Engine.
Which data formats does the transcoding support?
Per the documentation, Mainframe Connector transcodes QSAM or VSAM mainframe datasets associated with COBOL copybooks in EBCDIC, as well as files in ASCII UTF-8, into ORC, JSON, or CSV formats. By default it assumes the US EBCDIC Cp037 character set; it also supports the regional EBCDIC character sets for French, German, and Spanish. A custom character set can be implemented if needed.
What does a typical data transfer with Mainframe Connector look like?
Instead of the usual multi-step chain of copying to a file server, further processing, format conversion, and loading into a database, Mainframe Connector performs all these steps with a single command, using Cloud Storage as an intermediate location. This reduces the time until mainframe data is available in a database or data warehouse.
Note: All product information on this page has been compiled with care, but is provided without guarantee and may be outdated or incomplete. Cloud services evolve rapidly — features, pricing, SLAs, and availability change frequently. Authoritative and up-to-date information can only be found on the official product page of Google Cloud (official documentation). This page does not represent an offer by Google Cloud.
