Data Ingestion

8 min

Download sample CRM data in the .csv file.

This data is a sample representation of the CRM fields you would ingest to feed your business cases.

4. Select the Existing Dataset option

5. Search and select your dataset by name from the dropdown

6. Enable Partial Ingestion. Set threshold to the default 5%

7. Update the Dataflow to include your initials

8. Enable all alerts

9. Click Next

10. Upload the .CSV file downloaded from your local machine. Drag & Drop or click Choose files to upload

This mapping step is a requirement to ingest delimited files ONLY. Parquet and JSON formatted files can be dragged and dropped into the appropriate dataset since they are key value pairs and do not need to be explicitly mapped.
Adobe Experience Platform's AI has correctly mapped most of the source fields to the best matched XDM schema field.
Fix the ones that are not mapped as desired.

Add a Calculated Field We will add a new field in the field mapping workflow using data prep functions. Data engineers can use calculated fields to perform in-row calculations that can be assigned to XDM attributes.
- Click New Field Type -> Add Calculated Field
- Copy & paste the following expression below:iif(loyaltyPointsTotal > 2000, "Platinum", iif(loyaltyPointsTotal > 1000, "Gold","Silver"))
- Click Preview to make sure syntax is accepted and then click Save
- Map the calculated source field to the Loyalty Tier field in your schema
Click Finish. This will start the data ingestion flow

Notice a batch is created as a result of the ingestion workflow that was triggered

Once the status for the batch says "Success", click on the Preview Dataset button on the top right of your screen. This allows for validation of source columns to XDM schemas with the corresponding values.