Skip to main content
During development, you can inspect data as it flows through your pipeline. Data Preview icons appear throughout the pipeline and let you open the Data Explorer. You can also display row counts directly on the canvas. Enable Data Preview from a gem’s action menu to preload sample data after each run. Enable Record Count to display the number of rows produced by each gem.

​
Understand icons

  1. Data Preview icons appear on the pipeline canvas wherever data previews are available. By default, the icons are inactive row counts do not display (1).
  2. If you enable Data Preview, an indicator appears on the gem and the Data Preview icon becomes active (2).
  3. If you enable Row Count, a row count appears above the Data Preview icon (3).
  4. Row counts also appear automatically at model boundaries (that is, immediately before a destination table).

​
View Data Explorer

To automatically load sample data during pipeline execution, choose Data Preview from the gem menu. The preview icon becomes active, and sample data is preloaded after each run. Click a Data Preview icon to open the Data Explorer.

​
View complete dataset

The Data Explorer loads a sample of your data by default. When you sort, filter, or search, these actions apply only to the visible rows in the sample. To work with the full dataset, do one of the following:
  • Click Load More at the bottom of the table until all rows are visible.
  • Click Run in the top-right corner of the preview. This refreshes the view and applies sorting and filtering to the entire dataset.

​
Use the Data Explorer

In the Data Explorer, you can:
  • Sort data by columns.
  • Filter rows by specific values.
  • Search across all values.
  • Show or hide columns.
  • Export the sample as a CSV or JSON file.
  • View your data as a chart in the Visualization tab

​
View record count

To display the number of rows produced by a gem, choose Record Count from the gem menu. When you enable Record Count, Prophecy displays the row count above the Data Preview icon on the gem’s output. This lets you quickly verify how many rows are flowing through your pipeline without opening the Data Explorer. Row counts are available as follows:
  • At model boundaries by default.
  • On intermediate gems when Data Preview is enabled.
If you disable Record Count, any displayed row counts remain visible until you run the pipeline again or refresh the page.