Manage failovers

If a Bigtable cluster becomes unresponsive, replication makes it possible for incoming traffic to fail over to another cluster in the same instance. Failovers can be either manual or automatic, depending on the app profile an application is using and how the app profile is configured.

This page describes the steps for performing a failover between clusters.

If an app profile routes all requests to a single cluster, you can perform a manual failover.
If an app profile uses multi-cluster routing, failovers are automatic and you don't need to take any action.

Before you read this page, you should be familiar with the overview of Bigtable replication. You should also be familiar with the routing options that are available for Bigtable.

Perform a manual failover

Use a manual failover if an app profile routes all requests to a single cluster and that cluster becomes unresponsive. For examples of the criteria you could use to determine that a cluster is unresponsive, see Manual failovers. Check your instance's replication latency before you decide to fail over. For more information, see Charts for replication.

To perform a manual failover, update your app profile so that it routes requests to a responsive cluster instead of the unresponsive cluster:

Console

Open the list of Bigtable instances in the Google Cloud console.

Open the instance list
In the Application profiles column, click the app profile that is routing traffic to the unresponsive cluster.

If you don't see the app profile you want to edit, you can view a complete list by clicking the name of the instance, then clicking Application profiles in the left pane.
Under Cluster routing, select a responsive cluster in your instance.

Important: If Any cluster (multi-cluster) is selected, the app profile handles failovers automatically. Click Cancel to exit without saving.
Click Save. A confirmation dialog appears.
Carefully review the warnings in the confirmation dialog, then follow the instructions in the dialog and click Continue.

gcloud

If you don't know the instance ID, use the bigtable instances list command to view a list of your project's instances:
```
gcloud bigtable instances list
```
If you don't know the instance's cluster IDs, use the bigtable clusters list command to view a list of clusters in the instance:
```
gcloud bigtable clusters list --instances=INSTANCE_ID
```
Replace INSTANCE_ID with the permanent identifier for the instance.
If you don't know the app profile's ID, use the bigtable app-profiles list command to view a list of the instance's app profiles:
```
gcloud bigtable app-profiles list --instance=INSTANCE_ID
```
Replace INSTANCE_ID with the permanent identifier for the instance.
Use the bigtable app-profiles update command to change the cluster that the app profile uses:
```
gcloud bigtable app-profiles update APP_PROFILE_ID \
    --instance=INSTANCE_ID \
    --route-to=CLUSTER_ID
```
Provide the following:
- APP_PROFILE_ID: The permanent identifier for the app profile.
- INSTANCE_ID: The permanent identifier for the instance.
- CLUSTER_ID: The cluster ID that all requests should be routed to. This flag enables single-cluster routing.
If you receive an error message, carefully review any warnings in the error message. If you want to override the error, run the command again with the --force flag.

Shortly after you update the app profile, any applications that use the app profile will start to direct all of their requests to the healthy cluster that you selected. The unhealthy cluster will continue using CPU to handle replication and other maintenance tasks.

After the unhealthy cluster recovers, you can follow the same steps to update your app profile so that it routes all requests to the recovered cluster.

Perform an automatic failover

With Bigtable, automatic failovers truly are automatic. If an app profile uses multi-cluster routing, and the nearest cluster to the application server becomes unhealthy, you don't need to take any action. Bigtable automatically fails over, even if the cluster is only briefly unhealthy, and uses the nearest healthy cluster to handle requests until the unhealthy cluster has recovered.

To view the number of requests that were automatically rerouted over a given period of time, look at the Automatic Failovers graph in the Google Cloud console: open the list of instances, click the instance name, then click Monitoring.

What's next

Learn how to monitor a Bigtable instance.