Phases of the Zero Downtime Migration process
With Zero Downtime Migration, your applications can continue to run while you migrate data from one Cassandra-based cluster to another, resulting in little or no downtime and minimal service interruptions.
Why migrate?
There are many reasons that you might need to migrate data and applications. For example:
-
You want to move to a different database provider. For example, you might move from self-managed clusters to a cloud-based Database-as-a-Service (DBaaS), such as Astra DB.
-
You need to upgrade a cluster to a newer version or infrastructure.
-
You want to move client applications from shared clusters to dedicated clusters for greater control over individual configurations.
-
You want to consolidate client applications running on separate clusters onto one shared cluster to minimize sprawl and maintenance.
ZDM is comprised of ZDM Proxy, ZDM Utility, and ZDM Proxy Automation, which orchestrate activity-in-transition on your clusters. To move and validate data, you use Astra DB Sideloader, Cassandra Data Migrator, or DSBulk Migrator. ZDM Proxy keeps your clusters in sync at all times by a dual-write logic configuration, which means you can stop the migration or roll back at any point. For more information about these tools, see Compare DataStax migration tools.
When the migration is complete, the data is present in the new database, and you can update your client applications to connect exclusively to the new database. The old database becomes obsolete and can be removed.
Requirements for zero downtime
True zero downtime migration is only possible if your database meets the minimum requirements, including cluster compatibility, described in Feasibility checks If your database doesn’t meet these requirements, you can still complete the migration, but downtime might be necessary to finish the migration.
For more information, see Feasibility checks
Migration phases
A migration project includes preparation for the migration and five migration phases.
The following sections describe the major events in each phase and how your client applications perform read and write operations on your origin and target clusters during each phase.
The origin is is your existing Cassandra-based environment, which can be Apache Cassandra, DSE, or Astra DB. The target is your new Cassandra-based environment where you want to migrate your data and client applications.
Pre-migration client application operations
Here’s a look at a pre-migration from a high-level view. At this point, your client applications are performing read/write operations with an existing CQL-compatible database such as Apache Cassandra, DSE, or Astra DB.
|
For the migration to succeed, the origin and target clusters must have matching schemas. A CQL statement that your client application sends to ZDM Proxy must be able to succeed on both the origin and target clusters. This means that any keyspace that your client application uses must exist on both the origin and target clusters with the same name. The table names, column names, and data types must also match. For more information, see Schema/keyspace compatibility. |
Migration planning
Before you begin the migration, plan and prepare for the migration:
Phase 1: Deploy ZDM Proxy and connect client applications
In this first phase, deploy the ZDM Proxy instances and connect client applications to the proxies. This phase activates the dual-write logic. Writes are bifurcated (sent to both the origin and target), while reads are executed on the origin only.
Phase 2: Migrate data
In this phase, migrate existing data using Astra DB Sideloader, Cassandra Data Migrator, or DSBulk Migrator. For information about these tools, see Compare DataStax migration tools.
ZDM Proxy will continue to perform dual writes while you move data and validate that the migrated data is correct.
Phase 3: Enable asynchronous dual reads
In this optional phase, you can enable the asynchronous dual reads feature to test the target cluster’s ability to handle a production workload before you permanently switch your applications to the target cluster at the end of the migration process.
When enabled, ZDM Proxy sends asynchronous read requests to the secondary cluster in addition to the synchronous read requests that are sent to the primary cluster by default.
For more information, see Enable asynchronous dual reads and How ZDM Proxy handles reads and writes.
Phase 4: Route reads to the target cluster
In this phase, read routing on ZDM Proxy is switched to the target cluster so that all reads are executed on the target. Writes are still sent to both clusters.
At this point, the target becomes the primary cluster.
Phase 5: Connect directly to the target cluster
In this phase, you move your client applications off ZDM Proxy and connect them directly to the target cluster.
Once this happens, the migration is complete, and you now exclusively use the target cluster.
Zero Downtime Migration interactive lab
As a companion to the ZDM documentation, you can use the Zero Downtime Migration interactive lab to try the entire migration process in a demo environment.
The lab only requires a GitHub account and a supported browser. All browsers are supported except Safari.
You don’t need to install anything because the lab uses a pre-configured GitPod environment.
This lab provides an interactive, detailed walkthrough of the migration process, including pre-migration preparation and each of the five migration phases. The lab describes and demonstrates all steps and automation required to prepare for and complete a migration from any supported origin database to any supported target database.
