AGS Logo AGS Logo

Firestore Data Migration

Close up of a hard disk with platters and head exposed.

Photo by Jandira Sonnendeck on Unsplash

Do you need to backup and restore your Firebase data, or perhaps migrate data to another project? This blog post will get you started with some simple tricks.

Cloud Console

To manage your Firestore database you'll need to use the Google Cloud console instead of through the Firebase console as you're probably used to.

As seen in Figure 1, on the console you'll need to select your database by clicking on the Database ID, which is typically (default) for Firebase projects.

Google Cloud UI for the Firestore Databases screen, showing a list of databases including the Firebase (default) databases.
Firestore Database in Google Cloud Console

Once the database has selected, choose Import/Export in the side nav. As shown in Figure 2, this page provides both Import and Export functions and a list of recent operations.

Google Cloud UI for Firestore showing the Import/Export screen once a specific database has been selected. There are Import and Export buttons to begin operations and an empty list of recent operations.
Database Import/Export console

Export

Exporting your Firestore database from here is really easy. Select the Export button shown in Figure 2 and then select the Export Source and Destination as shown in Figure 3. The source can either be your entire database, or you can select one or more collection groups if you want to export a subset of the database. The destination will be a cloud storage bucket - I usually create a bucket specifically for backups

Google Cloud UI for Firestore showing the Export screen. You can choose between Export entire database or Export one or more collection groups. Additionally you'll select a Destination which is a cloud storage bucket.
Firestore Export

The export process is usually fast, and after initiating the export you'll be returned to the Import/Export UI where it will show the new export activity in the list and will let you know when it is completed.

Import

When you want to restore an export you select the Import button shown in Figure 2 and then browse to the correct cloud storage bucket and download folder as shown in Figure 4. You'll select a file that starts with a timestamp and ends with "overall_export_metadata".

Google Cloud UI for Firestore showing the Import screen screen. You'll choose a file to import by browsing through cloud storage buckets. Once you've identified the export folder you'll need to select the overall_export_metadata file to begin the import.
Firestore Import

Note that everything contained within the selected export will REPLACE conflicting records in your database so be extremely sure you know what you're importing. If your export contains only a partial export (one or more collection groups) then only those collection groups should be affected but TAKE A FULL DATABASE BACKUP BEFOREHAND ANYWAY.

Migration

To migrate data between two Firestore databases simply export from one and import to the other.

It may not be immediately obvious on how to download the entire backup folder, but Figure 5 shows that selecting the folder containing backups and selecting the row with the specific backup enables a download option for the entire backup folder.

Google Cloud UI for the Cloud Storage bucket browser. Selecting a bucket shows the directory tree and files contained within. From here files (and folders) may be downloaded and uploaded.

Cloud Storage bucket download

There is also an upload folder option to make it easy to move the data on the other end.

Command Line

You can use the gsutil command line tool to move files and folders around between Cloud Storage buckets including downloading to your local filesystem or uploading local files up to the cloud.

To copy files between cloud buckets:

gsutil -m cp -R gs://bucket/folder gs://bucket/folder

This is by FAR the fastest way to transfer files, such as when migrating data between Firebase accounts as the transfer is performed within Google's network. In some cases you may want a local copy of the files.

To download cloud files:

gsutil -m cp -R gs://bucket/folder ./folder

You may alternatively use the gcloud CLI tool which offers similar copy capabilities. Additionally, gcloud can be used to schedule regular Firestore backups, which every production project should have.

For more details on exactly how to use the CLI tools to schedule backups or move files around, please see the resource links below.

Resources

Firebase flame logo

Andromeda is a Google Cloud partner specializing in Firebase, building applications for our clients alongside our own portfolio of products.

License: CC BY-NC-ND 4.0 (Creative Commons)