Ingesting Data & Monitoring Batch Progress

1. 1Navigate to your active project workspace.
2. 2In the center workspace, locate the Let’s create some magic canvas and click the Map your data button.
3. 3The system will transition into a loading state while the autonomous Data Engineering Agent performs initial discovery and structures the mapping recipe.

Before executing the pipeline, the AI Assistant analyzes your source files and displays its strategy in the workspace:

1. 1Carefully Read the Plan: Inspect the rendered markdown. Verify the AI mapped the source columns (e.g., WORK_EMAIL in your messy Excel sheet) to your exact target fields (e.g., Email).
2. 2Review Unique Identifiers: Ensure the identified primary key columns are correct.
3. 3Course-Correct: If you see any mismatch, type your correction into the AI Assistant chat panel (e.g., “Map F_NAME to First_Name instead of Full_Name”). The AI regenerates the plan immediately.
4. 4Approve: Once verified, type “Plan approved” or “Proceed with batch” in the chat to start processing.

Processing large datasets (such as a CSV with 50,000 rows or 200 PDF files) can take time. Elvity splits your data into manageable batches to ensure high performance. Once processing starts, a segmented progress bar appears above the workspace showing individual batch blocks.

Navigate Batches: Click on any completed batch segment on the progress bar to view that batch.