Configuring advanced Data Mapping

This guide explains how to configure advanced Data Mapping for your datastreams.

Introduction

Apply Data Mapping to a datastream to map source fields to target fields that conform to Adverity’s unified naming and formatting conventions.

This guide explains how to configure advanced Data Mapping for your datastreams. For more information on configuring basic Data Mapping, see Harmonizing data.

Changes to the Data Mapping may change the table structure of the data extracts. The changed table structure may disrupt the data structure in your destinations.

Changes to the Data Mapping only affect future data extracts and future data transfers to your destinations.

Prerequisites

Before you complete the procedure in this guide, perform all of the following actions:

Mapping a source field to a target field

To harmonize your data, map the source fields in your data extract to target fields provided in Adverity. Using the same target fields in multiple datastreams makes it easier to harmonize your data. To map a source field to a target field, follow these steps:

  1. Select the workspace you work with in Adverity and then, in the platform navigation menu, click Datastreams.

  2. Select a datastream.

  1. In the top navigation panel, click Data Mapping.

  1. In the row for the source field you want to map, in the Target fields column, type in the target field that you want to use and select it from the drop-down menu.

The Data Mapping changes are saved automatically.

Mapping a source field to a new target field

If the target field that you want to use does not appear in the drop-down menu, you can create a new target field with a name of your choice. This creates a new field in the data transferred to the destination. To create a new target field and map a source field in your datastream to your new field, follow these steps:

  1. Select the workspace you work with in Adverity and then, in the platform navigation menu, click Datastreams.

  2. Select a datastream.

  1. In the top navigation panel, click Data Mapping.

  1. In the row for the source field you want to map, in the Target fields column, type in the name of the target field that you want to create and click + Create new.

  2. In the Create new target field pop-up that opens, in the Type field, select the data type. For more information on data types, see Data types used in data harmonization.

  3. (Optional) In the Length field, specify the maximum character length of the values in the new field. This option is only available if you select String in the Type field.

  4. (Optional) In the Measure field, specify the mathematical function underlying the values in the new field. This option is only available if you select a numerical data type in the Type field, and it only has an effect if you transfer data to Explore & Present. For more information, see Measures used in data harmonization.

  5. (Optional) In the Description field, enter a description for the new target field.

  6. Click Create.

As a result, you have mapped your chosen source field to your newly created target field. This target field will now appear on the Data Schema page.

Mapping source fields to target fields in the data extract preview

You can also map a source field to a new or existing target field in a specific data extract in the data extract preview. To do this, follow these steps:

  1. Preview the data extract containing the field you want to map.

  2. In the table header, under the name of the source field that you want to map, perform one of the following actions:

    • To map an unmapped source field to an existing target field, click on Not mapped and type in the name of the target field that you want to map.

    • To change the target field to which a source field is already mapped, click on the existing target field and type in the name of the target field that you want to map.

    • To map a source field to a new target field, click on Not mapped or the existing target field, then type in the name of the target field you want to create and click + Create new.

      In the Create new target field window, follow steps 5-9 above.

Creating a new target field without mapping

To create a new target field on the Data Schema page without mapping a source field to your new field, follow these steps:

  1. Select the workspace you work with in Adverity and then, in the platform navigation menu, click Data Schema.

  2. In the top right corner of the page, click Add target column.

  3. In the Name field, write the name of the new field. Use only lower-case letters, numbers, and underscores.

  4. In the Type field, select the data type. For more information on data types, see Data types used in data harmonization.

  5. (Optional) In the Length field, specify the maximum character length of the values in the new field. This option is only available if you select String in the Type field.

  6. (Optional) In the Measure field, specify the mathematical function underlying the values in the new field. This option is only available if you select a numerical data type in the Type field, and it only has an effect if you transfer data to Explore & Present. For more information, see Measures used in data harmonization.

  7. (Optional) In the Description field, enter a description for the new target field.

  8. Click Save.

As a result, you have created a target field. This target field will now appear in the Data Mapping tab when configuring the Data Mapping for a datastream.

Editing a target field

Certain data types are not compatible with some destinations. For example, Explore & Present does not support the data type DateTime. Edit the data type of a target field to harmonize your data extract so the data types are compatible with the destination to which you want to transfer data.

To edit a target field, follow these steps:

  1. Select the workspace you work with in Adverity and then, in the platform navigation menu, click Data Schema.

  2. Hold the pointer over the target field to edit, and then click Edit.

  3. In the Type field, change the data type. For more information on data types, see Data types used in data harmonization.

  4. (Optional) In the Length field, change the maximum character length of the values in the new field. This option is only available if you select String in the Type field.

  5. (Optional) In the Measure field, specify the mathematical function underlying the values in the new field. This option is only available if you select a numerical data type in the Type field, and it only has an effect if you transfer data to Explore & Present. For more information, see Measures used in data harmonization.

  6. Click Save.

As a result, you have updated the target field.

Setting key columns

Key columns uniquely identify a data set. Use key columns to ensure that data is correctly overwritten when you transfer data to the destination that you assigned to the datastream. For more information, see Configuring transfer settings.

You can only set dimension fields as key columns.

To set a field as a key column, follow these steps:

  1. Select the workspace you work with in Adverity and then, in the platform navigation menu, click Datastreams.

  1. Select the chosen datastream.

  1. In the top navigation panel, click Data Mapping.

  1. Find the field in the list.

  2. In the Key Column column, enable the toggle.

The Data Mapping changes are saved automatically.

After changing the key columns, you need to perform additional actions to transfer data to some destinations. For an example, see SQL Database destination reference.

To overwrite data in a destination based on key columns, select Key Columns in the destination configuration. For more information, see Configuring transfer settings.

Some destinations do not support overwriting data with key columns. For an example, see Google BigQuery destination reference.

Changing a field's internal data type

When you fetch a data extract, Adverity automatically detects and configures the internal data type for each field. The internal data type of a field is separate from the data type you set in Data Mapping. Adverity only uses the internal data types when you transfer the data extract to a destination without applying Data Mapping.

To change a field's internal data type in a data extract, follow these steps:

  1. Select the workspace you work with in Adverity and then, in the platform navigation menu, click Datastreams.

  1. Select the chosen datastream.

  1. In the top navigation panel, click Data Extracts.

  2. In the list, click the top hyperlinked element in the Name column.

  3. Hold the pointer over one of the column headings, and then click Details.

  4. In the Columns section, find the field in the list.

  5. In the field's row, click the drop-down menu on the right.

  6. Select the new data type for the field.