Storage Transfer Service transfers data from AWS S3, Azure Blob, HTTP sources, and on-premises systems to Google Cloud Storage with automatic scheduling and incremental transfers.
What is Storage Transfer Service?
Storage Transfer Service is a fully managed service for migrating and synchronizing data to Google Cloud Storage. The service supports various sources: Amazon S3, Azure Blob Storage, HTTP/HTTPS endpoints, other Cloud Storage buckets, and on-premises file systems. Transfers can be executed one-time, on schedule, or event-driven.
A core feature is incremental transfer: After the initial full copy, only changed or new files are transferred. This significantly reduces transfer times and costs for recurring synchronizations. File filters enable selection of specific file types, directories, or modification periods. Metadata preservation maintains file attributes during transfer.
For on-premises data, the Transfer Service Agent provides a Docker-based agent running on local servers. The agent scans file systems, transfers data encrypted, and supports parallel uploads for high throughput. Multiple agents can be deployed for horizontal scaling. Bandwidth limits prevent impact on other workloads.
Common Use Cases
AWS S3 to Cloud Storage Migration
A company migrates from AWS to Google Cloud. Storage Transfer Service copies all S3 buckets to Cloud Storage with metadata preservation. Incremental transfers synchronize changes during the migration phase, cutover occurs with minimal downtime.
Multi-Cloud Backup Strategy
A SaaS provider uses Storage Transfer Service for cloud backup. Production data in AWS S3 is regularly replicated to Cloud Storage. During AWS outages, the data serves as disaster recovery. Costs are lower than dedicated backup solutions.
On-Premises Archival
A media company archives local video archives to Cloud Storage. Transfer Service Agents run on local servers, night transfers use unused bandwidth. Archive Class Storage reduces long-term storage costs by 80%.
Daily Data Synchronization
An analytics team synchronizes data sources to BigQuery daily. Storage Transfer Service copies CSV files from external FTP servers to Cloud Storage. Data Transfer Service automatically loads them into BigQuery for analysis.
Compliance Data Residency
A European company consolidates data in EU regions. Data from global S3 buckets is transferred to Cloud Storage in europe-west1. Storage Transfer Service ensures all copies land in the EU.
Integration with innFactory
As a Google Cloud partner, innFactory supports you with Storage Transfer Service: migration planning, agent setup, transfer optimization, and cost analysis.
Contact us for a consultation on Storage Transfer Service and Google Cloud migration.
Available Tiers & Options
Online Transfer
- Automatic scheduling
- Incremental transfers
- Bandwidth control
- Depends on network bandwidth
On-Premises Agent
- For local file systems
- Encrypted transfer
- Parallel uploads
- Agent installation required
Typical Use Cases
Technical Specifications
Frequently Asked Questions
What is Storage Transfer Service?
Storage Transfer Service is a fully managed service for transferring data to Google Cloud Storage. The service supports sources like AWS S3, Azure Blob Storage, HTTP/HTTPS endpoints, and on-premises file systems with automatic scheduling and incremental transfers.
From which sources can I transfer data?
Supported sources are Amazon S3, Azure Blob Storage, HTTP/HTTPS URLs, other Cloud Storage buckets, and POSIX-compliant file systems (NFS, local disks) via the Transfer Service Agent. For very large local data volumes, Transfer Appliance is an alternative.
Can I schedule transfers?
Yes, Storage Transfer Service supports one-time transfers, recurring schedules (daily, weekly), and event-driven transfers. Incremental transfers copy only changed files, saving time and costs.
How are costs calculated?
Transfers between Cloud Storage buckets are free. For external sources (S3, Azure, HTTP), fees apply per GB of transferred data. Egress costs from the source provider are additional. On-premises transfers via agents are also charged.
Can I limit bandwidth?
Yes, Storage Transfer Service offers bandwidth limits to avoid impacting other workloads. You can set limits in MB/s and schedule transfers for night hours or weekends.
How does the on-premises agent work?
The Transfer Service Agent is a Docker container running on local servers. It scans file systems, transfers data encrypted to Cloud Storage, and supports parallel uploads for high throughput. Multiple agents can be used for scaling.
