> ## Documentation Index > Fetch the complete documentation index at: https://docs.prophecy.ai/llms.txt > Use this file to discover all available pages before exploring further. # Quickstart for Data Analysts > Use the Agent to develop a simple pipeline export const WistiaVideo = ({mediaId, autoplay = true, muted = true, loop = true}) => { const containerRef = React.useRef(null); const playerRef = React.useRef(null); const wistiaPlayerRef = React.useRef(null); const initWistia = () => { if (!containerRef.current || playerRef.current) return; const container = containerRef.current; const playerId = `wistia_${mediaId}_${Math.random().toString(36).substr(2, 9)}`; container.innerHTML = ''; const embedDiv = document.createElement('div'); embedDiv.className = `wistia_embed wistia_async_${mediaId}`; embedDiv.id = playerId; container.appendChild(embedDiv); if (window._wq) { window._wq.push({ id: playerId, onReady: video => { wistiaPlayerRef.current = video; if (loop) { video.bind('end', () => { video.play(); }); } }, options: { autoPlay: autoplay, muted: muted, videoFoam: true } }); } playerRef.current = playerId; }; React.useEffect(() => { if (window._wq) { initWistia(); return; } const script = document.createElement('script'); script.src = 'https://fast.wistia.com/assets/external/E-v1.js'; script.async = true; script.onload = () => { window._wq = window._wq || []; initWistia(); window.dispatchEvent(new Event('wistia-loaded')); }; document.head.appendChild(script); const checkWistia = () => { if (window._wq) { initWistia(); window.removeEventListener('wistia-loaded', checkWistia); } }; window.addEventListener('wistia-loaded', checkWistia); return () => { window.removeEventListener('wistia-loaded', checkWistia); if (playerRef.current && window.Wistia && window.Wistia.api) { const player = window.Wistia.api(playerRef.current); if (player) { player.remove(); } } }; }, [mediaId, autoplay, muted, loop]); return
; }; *Estimated time: 30 minutes* Build a data pipeline using Prophecy Agent. This tutorial walks you through exploring data, creating visualizations, and building transformations using natural language prompts. Follow the steps below to build a patient analytics pipeline. Importing from Alteryx or other platforms? Start with [Import Workflows into Prophecy](/import-tool/). ## Prerequisites To complete this quickstart, you need a [Prophecy fabric](/data-analysis/environment/fabrics/prophecy-fabrics) that uses Prophecy In Memory or Databricks as the compute engine. Prophecy automatically creates a compatible fabric for Free and Professional Edition users. If you do not have any existing fabrics, [you'll need to create one](/data-analysis/environment/fabrics/prophecy-fabrics). ## Set up a new project First, you need to create the project where you will build your pipeline. You'll also need to add data to the project for this quickstart. 1. Click on the **Create Entity** button in the left navigation bar. 2. Hover over the **Project** tile and click **Create**. 3. Give your project a **Name**, such as `Prophecy_Quickstart`. 4. Under **Team**, select your personal team. (It will match your user email.) 5. Under **Select Template**, choose **Prophecy for Analysts**. 6. Click **Complete**. Prophecy will open your new project in [Studio](/data-analysis/development/studio/studio). For a new project with v4 AI enabled, Agent chat opens maximized by default. 1. Select the default fabric or your own fabric. 2. Click **Save**. This attaches the project to the fabric. You can switch fabrics at any time by clicking the fabric selector in the project header. 1. Minimize Agent chat to show the project landing page. 2. On the project landing page, click **Create Pipeline**. 3. For the **Pipeline Name**, enter `patient_analytics`. 4. Leave the default **Directory Path** of `pipelines`. Prophecy saves your compiled pipeline code in this folder of the project repository. 5. Click **Create**. This opens the pipeline canvas for your new pipeline. ## Add data to your project Next, you'll add some data to the project so the Agent can find and transform it. Load some data into the project as a Seed: 1. Open the **Source/Target** gem category. 2. Click **Table**. This adds a new [Table gem](/data-analysis/gems/source-target/source-target) to the canvas. 3. Hover over the gem and click **Open**. 4. Select **+ New Table**. 5. For the **Type and Format**, choose **Seed**. 6. Name the seed `patients_raw_data`. 7. For the **Seed path**, choose **seeds**. Prophecy saves your seed file in this folder of the project repository. 8. Click **Next**. 9. In the **Properties** tab, paste the following data. ```csv patients_raw_data.csv theme={null} patient_id,first_name,last_name,city,state,county,age,admission_date,diagnosis,treatment_cost 1001,John,Smith,Boston,MA,Suffolk,45,2024-01-15,Hypertension,1250.00 1002,Sarah,Johnson,Boston,MA,Suffolk,32,2024-01-18,Diabetes,2100.50 1003,Michael,Williams,Cambridge,MA,Middlesex,58,2024-01-20,Heart Disease,3500.75 1004,Emily,Brown,Boston,MA,Suffolk,29,2024-01-22,Asthma,850.25 1005,David,Jones,Worcester,MA,Worcester,67,2024-01-25,Hypertension,1450.00 1006,Jessica,Garcia,Boston,MA,Suffolk,41,2024-02-01,Diabetes,2200.00 1007,Christopher,Miller,Springfield,MA,Hampden,53,2024-02-05,Heart Disease,3800.50 1008,Amanda,Davis,Boston,MA,Suffolk,35,2024-02-08,Asthma,920.75 1009,James,Rodriguez,Cambridge,MA,Middlesex,62,2024-02-10,Hypertension,1320.00 1010,Lisa,Martinez,Worcester,MA,Worcester,48,2024-02-12,Diabetes,2150.25 ``` 10. Click **Next**. 11. Click **Load Data** to preview the data in tabular format. 12. Click **Save**. To materialize the seed data into the SQL warehouse, run the pipeline once. To do so, click the play button in the bottom right corner of the canvas. Prophecy should automatically index the table when you save the seed. This allows the Agent to discover and use the seed data. If you have trouble finding the table in later steps, you can also manually reindex the [knowledge graph](/data-analysis/ai/knowledge-graph/knowledge-graph). 1. Open the **Environment** tab in the left sidebar. 2. Below your connections, you'll see a **Missing Tables?** callout. 3. Click **Refresh** to trigger the knowledge graph indexer. You'll see a progress bar in the callout indicating the indexer is running. Once it's complete, the callout will disappear. Verify the table was indexed by checking the **Environment** tab in the left sidebar. ## Explore the data Now that you have data in your project, you can explore it using the Agent. If Agent chat is minimized, maximize it to continue working in chat. This is where you'll interact with the Agent. Ask the Agent to search for the seed data table. Enter the following prompt in the chat: ``` Find datasets with information pertaining to hospital patients ``` The Agent returns: * A short list of relevant datasets with descriptions. * A full list of matching datasets. * The option to add datasets directly to your pipeline on hover. 1. Click on `patients_raw_data` (the data you uploaded) in the chat to open a detailed preview dialog where you can: * View the table location. * Examine the schema and column structure. * Preview sample data. * Review data profiles. * Open an **Explore** session for dataset-specific queries. 2. Close the preview dialog to return to the chat. If the Agent doesn't find your table, verify that the table was indexed by checking the **Environment** tab in the left sidebar. Additionally, if you have other patient data in your warehouse, the Agent may return other datasets that match your query instead. Request specific data samples to validate your understanding. Enter the following prompt, replacing `@patients_raw_data` with your actual table path: ``` Provide sample data from @patients_raw_data showing only patients from Boston ``` The Agent returns: * A table with the requested data. * An option to preview the table for detailed examination. * An option to add the table to your pipeline. * SQL execution logs showing the query used. Verify the results show 4 patients: John Smith, Sarah Johnson, Emily Brown, and Jessica Garcia, all from Boston. Generate charts and insights from your data. Enter the following prompt: ``` Visualize number of patients per city in @patients_raw_data ``` The Agent returns: * An embedded chart in the chat showing patient counts by city. * An option to preview the table for detailed examination. * An option to add the table to your pipeline. * SQL execution logs showing the query used. Click **Preview** to access: * **Visualization tab**: View larger charts and download charts as images. * **Data tab**: Examine underlying data and download data as JSON/Excel/CSV. The chart should show Boston with 5 patients, Cambridge with 2, Worcester with 2, and Springfield with 1. ## Build your pipeline Now, you'll build your pipeline transformations using the Agent. Keep the **Visual** view open (rather than the **Code** view) to see updates in real-time as the Agent adds or modifies gems on the canvas. Describe the transformation you want to perform. Enter the following prompt: ``` Transform @patient_records to show total number of patients per county ``` Replace `@patient_records` with the actual gem label from your canvas if it differs. The Agent will: * Add the appropriate gem(s) to your pipeline canvas. This prompt should produce an [Aggregate](/data-analysis/gems/transform/aggregate) gem. * Execute the pipeline, generating data samples that you can review. * Provide a description of the changes made. * Show options to inspect, preview, or restore changes. * Display SQL execution logs. The output should show 3 counties: Suffolk with 5 patients, Middlesex with 2, Worcester with 2, and Hampden with 1. Each change that the Agent makes can be viewed in the [project version history](/data-analysis/development/versioning/version-control#show-version-history). You can revert the changes at any time. To understand what the Agent built: 1. Click **Inspect** on the transformation response. 2. Review the configuration panel starting with the first modified gem (highlighted in yellow). 3. Use the **Previous** and **Next** buttons to navigate through modified gems. 4. Examine input and output data to verify the transformation produces expected results. This helps you: * Understand the Agent's approach. * Verify that the transformation logic matches your expectations. Build a more complex transformation. Enter the following prompt: ``` Calculate average treatment cost per county from the previous step ``` The Agent adds another transformation that calculates the average cost. This demonstrates how you can chain transformations together. Verify the results show average costs for each county. Suffolk should have an average around `1,544`, Middlesex around `2,410`, Worcester around `1,800`, and Hampden `3,800.50`. After building your pipeline, ask the Agent to save the final output. Enter the following prompt: ``` Save the final output of this pipeline as a table ``` The Agent adds a [Table gem](/data-analysis/gems/source-target/source-target) to the end of your pipeline. When you run the pipeline, the Table gem writes the data to your default database and schema defined in your fabric, allowing you to persist results. ## Explore further Try these additional tasks to extend your pipeline. Enter the following prompt in the chat: ``` Filter the source data to only include patients over 50 years old ``` Create a second Seed file named `county_info` with the following content: ```csv county_info.csv theme={null} county,population,region Suffolk,800000,Eastern Middlesex,1600000,Eastern Worcester,830000,Central Hampden,470000,Western ``` Then, prompt the Agent: ``` Join @l0_raw_patients with @county_info on county ``` The Agent will infer join keys, but you can also specify them explicitly. Enter the following prompt in the chat: ``` Add a column that calculates days since admission using the current date ``` ## Connect your own data This quickstart uses a Seed file as the source data. When you start building your own pipelines, you'll likely want to use your own data from files or external systems. You can do this by: * Uploading files directly from your local filesystem using the [upload file](/data-analysis/gems/source-target/table/upload-files) feature. * Ingesting data from external systems using [connections](/data-analysis/environment/connections/connections). ## Sample prompts reference Use these prompts as templates for your own pipelines. | Task | Prompt Example | | -------------- | ----------------------------------------------------------------------------------------------------- | | Find data | `Find datasets containing customer information` | | Sample data | `Show me 5 random records from @sales_data` | | Filter data | `Filter to only include orders from 2024` | | Transform data | `Calculate total revenue as quantity * price` | | Join data | `Join the orders and customers tables` | | Parse data | `Extract the fields from json_data as columns` — This works best if you provide a sample JSON object. | | Clean data | `Remove rows where email is null` | | Aggregate data | `Group by region and calculate average sales` | | Visualize data | `Create a bar chart of monthly sales` | | Save results | `Save the final output as a table` | ## Tips **How to do it**: Instead of `Clean the data` → Try `Remove duplicate customer records and fill null values in the email column` **Why it helps**: Reduces ambiguity so the Agent applies the correct operations. **How to do it**: Break complex transformations into smaller requests. For example: `Aggregate orders by customer` → `Join with customers` → `Filter for orders made in 2024` **Why it helps**: Improves reliability and makes it easier to debug or adjust each step. **How to do it**: After a response, click **Inspect** and use **Previous/Next** to review highlighted gem configuration and output **Why it helps**: Ensures the transformation matches expectations before you continue. **How to do it**: Use chat to scaffold transformations, then switch to the **Visual** canvas to fine-tune or add gems **Why it helps**: Combines speed (AI) with precision and control (visual editor). ## Troubleshooting **Probable cause**: [Knowledge graph](/data-analysis/ai/knowledge-graph/knowledge-graph) is out of date **How to fix**: [Reindex your fabric connection](/data-analysis/ai/knowledge-graph/indexer) so the knowledge graph includes the table **Probable cause**: Chat context isn't relevant anymore **How to fix**: Start a new chat. **Probable cause**: Seed data wasn't created correctly **How to fix**: Verify the table exists and contains the expected 10 rows of data