Data Ingestion

8 min

Download sample CRM data in the .csv file.

This data is a sample representation of the CRM fields you would ingest to feed your business cases.

4. Select the Existing Dataset option

5. Search and select your dataset by name from the dropdown

6. Enable Partial Ingestion. Set threshold to the default 5%

7. Update the Dataflow to include your initials

8. Enable all alerts

9. Click Next

10. Upload the .CSV file downloaded from your local machine. Drag & Drop or click Choose files to upload and click Next in the upper right

This mapping step is a requirement to ingest delimited files ONLY. Parquet and JSON formatted files can be dragged and dropped into the appropriate dataset since they are key value pairs and do not need to be explicitly mapped.
Adobe Experience Platform's AI has correctly mapped most of the source fields to the best matched XDM schema field.
Fix the ones that are not mapped as desired.

Add a New Field We want to do a custom mapping to one of our added field.
1. Click the New field type (+) button and choose Add new field
2. Select crmind from the source scema
3. Map to crmid to crmid in the custom object we added in the Schema exercise
Add a Calculated Field We will add a new field in the field mapping workflow using data prep functions. Data engineers can use calculated fields to perform in-row calculations that can be assigned to XDM attributes.
- Click the New field type button and choose Add calculated field
- Paste in this code: concat(loyaltyTier,"-",loyaltyPoints)
- Click the Preview button
- Click Save
- Map to the loyatyTierPoints field in the custom object we added in the Schema exercise
- Click Finish in the upper right

Notice a batch is created as a result of the ingestion workflow that was triggered

Once the status for the batch says "Success", click on the Preview Dataset button on the top right of your screen. This allows for validation of source columns to XDM schemas with the corresponding values.