---
title: Data Ingestion via Google Big Query Source Connector
slug: ikea-cja/data-ingestion-via-google-big-query-source-connector
docTags: 
createdAt: 2025-08-21T11:15:35.020Z
---

In a production deployment, the data ingestion process will likely be automated using batch APIs provided by the Platform, or the several **Source Connectors** available in Experience Platform.

Experience Platform provides a RESTful API and an interactive UI that lets you set-up source connections to various data providers with ease. These source connections enable you to authenticate your storage systems and CRM services, set times for ingestion runs, and manage data ingestion throughput.

Let's see how we can onboard data using a source connector. In this exercise we will onboard **Campaign Summary** data stored in a Google Big Query table. 

### Pre-requisite

:::hint{type="info"}
The connection between AEP and Big Query has already been setup for this exercise. No action needed.
:::

In order to ingest data from Google Big Query, this is required:

- Setup a Google Big Query environment;
- Create a Service Account credentials in Google;
- Use the credentials to setup the connection with AEP (see [docs](https://experienceleague.adobe.com/en/docs/experience-platform/sources/connectors/databases/bigquery));
- Create a table in Google Big Query with the data.

### Step 1: Connecting to a Source

1. Select **Sources** from under `Connections` in the left hand navigation.
2. Click on the** Catalog **tab.
3. You will see a list of all Source Connectors. Search for **Google Big Query** and select **Add data**.

We've already set up a connection to a Google Big Query environment for the purpose of this Lab.

1. Select** NEW - Offline Data** from the Accounts list.
2. Select **Next**.

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9-Cq6SRGSleG211cQNHwPbj-20250822-120708.png)

Now we need to select the table we want to onboard:

1. select the file `data_source.offline_data` from the list on the left.

:::hint{type="info"}
You will be shown a preview of the data so you can validate the Adobe Experience Platform is interpreting the data correctly.
:::

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9-BukYLXDMKBaCYZ1MONZk_-20250822-120837.png)

Click **Next.**

We now need to specify the `Dataset` where this file will be ingested:

1. From the dropdown list under *Target dataset*, select `Campaign Summary Data` . 
2. Under **Dataflow details** prefix the Dataflow name with your own name so you can find this later.
3. Leave the other settings as the default.
4. Click **Next.**

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9--4elsw90tgqg4MUZ6yJIi-20250822-121048.png)

Now you can proceed with mapping it to the corresponding fields in the call center XDM Schema.

### Step 2: Using a Workflow to map a CSV file to an XDM Schema

The goal of this is to onboard Campaign Summary data in Platform. Most of the data that is ingested in Platform should be mapped against a specific XDM Schema. What you currently have is a flat table on one side, and a dataset that is linked to a schema on the other side. To load the table data in that dataset, a mapping needs to take place. To facilitate this mapping exercise, we have **Workflows** available in Adobe Experience Platform.

You should now be on the mapping step of the Workflow to onboard the data from Google Big Query.

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9-cSn66wrzBlFVFzRpBr6K5-20250822-121216.png)

You now need to map your table Column Headers with an XDM-property in your Dataset.

:::hint{type="info"}
**NOTE: **for the Schema Mappings, Adobe Experience Platform has linked fields together already. However, in the case not all proposals of mapping are correct, you can adjust the individual mappings in the UI.

To edit the mappings:

1. Select the **lightbulb** icon on the Target field you want to change.
2. If your chosen field doesn't appear in the list, choose **Select manually.**
3. Select your chosen field from the schema structure.
4. Your source attribute is now mapped to your chosen Target field. 
5. You can also type your Target field directly in the text box if you know the path.
:::

For this exercise you will not need to edit any of the mappings. Instead will choose to **Import Mapping** from a previous ingestion.

1. Select **Import mapping **from the menu above the mappings near the center of the screen.
2. This will open a window with mappings from previous **Dataflows**. 
3. Select the `Dataflow - Campaign Summary Data [Template]`.
4. Click **Select.**

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9-dhduI2b5gOSKbqkYK3MB--20250822-121905.png)

### Verify your final mappings

Your final mapping should be:

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9-AjNm1ezl0rvOpkmsr2qLA-20250822-121949.png)

1. Click **Next**, you'll then see the **Scheduling **options available for this Source Connector. 
2. Keep the default settings of **Once**, and click **Next.**
3. On the final **Review** step, click **Finish **to onboard your data.

You'll then be directed to the Sources overview, where you will see the list of the Dataflows configured for this source. 

:::hint{type="warning"}
The ingestion job is started at the time selected during the setup. If you have not changed anything, it will be triggered and completed in 5 minutes.
:::

In the Dataflows list, find yours and wait until the `LAST DATAFLOW RUN STATUS` colum say `Success`. (refresh the page sometimes).

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9-PSCHEBQfSkJnbMHpGBtc7-20250822-122453.png)

Once the data have been successfully uploaded: 

1. find your dataflow and click the **Target Dataset **to view the Dataset Activity.
2. You'll then see the **Dataset Overview** where your ingestion has processed, and after a couple of minutes you can refresh your screen to see if your workflow completed successfully. 

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9-rZ5_oRg5pb_0Zp9mXMwN1-20250822-122125.png)

On the Dataset activity screen, you'll see a **Batch ID** that has been ingested just now, with 126 records ingested and a status of `Success`**.**

1. Click on the **Preview Dataset** button to get a quick view of a small sample of the data ingested to ensure that it loaded correctly.

![](https://api.archbee.com/api/optimize/Kr9jNQk3X89PNSZafWHK9-2heurJDn6DmsCVtg8ACZv-20250822-122223.png)

:::hint{type="success"}
With this, you’ve now successfully ingested Campaign Summary Data in Adobe Experience Platform!
:::