Google BigQuery destination reference

This reference explains in detail how to configure data transfer to Google BigQuery.

Prerequisites

Before you complete the procedure in this guide, perform all of the following actions:

• Create a Google BigQuery destination. For more information, see Transferring data to Google BigQuery.

Configuring data transfer to Google BigQuery

To configure data transfer to Google BigQuery, follow these steps:

1. Select the workspace you work with in Adverity and then, in the platform navigation menu, click Destinations.

2. Click the Google BigQuery destination in the list.

3. In the left navigation panel, click Schema Options.

4. Fill in the following fields:

   Data Mapping

   By default, Adverity applies the Data Mapping conventions to the data extracts when transferring them to the destination. For more information on Data Mapping, see Harmonizing data.

   To transfer data to the destination without applying the Data Mapping conventions, clear this checkbox.

   Force String Type

   By default, Adverity automatically recognizes the data type of each field. Select this checkbox to convert all fields to the String data type.

5. Click Save.

Troubleshooting: Data transfer to Google BigQuery fails after changing the table structure of the data extract

A data transfer to Google BigQuery may fail because of a change in the table structure of the data extract. To ensure the frictionless operation of production systems, Adverity only modifies existing tables in Google BigQuery if the changes are considered neutral, such as adding a new field to the table. Changes to the table structure of the data extract are not considered neutral. As a consequence, Adverity does not update the existing table in Google BigQuery resulting in failed data transfer.

The following actions change the table structure of the data extract:

• You change the key columns in the datastream's Data Mapping. For more information on key columns, see Setting key columns.

• You change the data type of a column in the datastream's Data Mapping. For more information on the data types used in Adverity, see Data types used in data harmonization.

• You change the maximum character length of values in a column which contains String values.

To resolve this problem, follow these steps:

1. In Google BigQuery, modify or drop the table that corresponds to the datastream.

2. In Adverity, transfer data from the datastream to Google BigQuery.