> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prophecy.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipeline gem

> Start pipeline runs from a gem in the canvas

The Pipeline gem allows you to run another pipeline from within your current pipeline. It supports conditional triggering, parameter passing, and returning metadata for each pipeline run.

This gem is useful for building orchestrated workflows directly in the visual canvas. You can find it under the **Custom** category in the gem drawer.

A single Pipeline gem execution can trigger the target pipeline multiple times. Triggered pipeline runs execute sequentially, and the Pipeline gem completes only after all triggered runs finish.

<Tip>
  You can create pipelines solely dedicated to pipeline orchestration using multiple instances of this gem. We recommend labeling your orchestration pipelines to differentiate them from standard data pipelines.
</Tip>

## Limitations

The Pipeline gem has the following limitations:

* It can only trigger pipelines for the current project.
* The [Monitoring](/data-analysis/production/monitoring) page doesn't show the parent pipeline that triggered a run using the Pipeline gem.

## Input and Output

The following table describes what the Pipeline gem expects as input and what it will produce as output.

| Port    | Description                                                                                                                                                                                                                                           |
| ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **in0** | Optional input dataset that may include a `status` column and additional columns.<br /><br />Valid statuses include `success`, `failure`, or `skipped`.<br /><br />Each row of the input triggers a separate pipeline run per Pipeline gem execution. |
| **out** | One output dataset that contains the metadata for each triggered pipeline run. <br /><br />The Pipeline gem returns one row per triggered run.                                                                                                        |

When no input is connected:

* The Pipeline gem triggers the child pipeline once.
* The output contains a single row of metadata for that run.

When an input dataset is connected:

* If a `status` column is present, it is used to evaluate [trigger conditions](#trigger-conditions). If the trigger condition is met, the Pipeline gem triggers the child pipeline once for each input row.
* If a `status` column is **not** present, the Pipeline gem always triggers the child pipeline once for each input row.
* Additional columns can be used to pass parameter values into each run.
* The output contains one row of metadata per input row.

<Note>
  The input dataset can be any dataset (including the output of an upstream Pipeline gem).
</Note>

### Output schema

Each row in the output table represents a pipeline run triggered from the Pipeline gem. The output schema provides you with the following information for each pipeline run.

| Column          | Type         | Description                                                                 |
| --------------- | ------------ | --------------------------------------------------------------------------- |
| `status`        | String       | Final status of the triggered pipeline: `success`, `failure`, or `skipped`. |
| `pipelineRunID` | String       | Unique ID of the triggered pipeline run.                                    |
| `startTime`     | Timestamp    | Time when the triggered pipeline started.                                   |
| `endTime`       | Timestamp    | Time when the triggered pipeline finished.                                  |
| `error`         | String       | Error message, if any.                                                      |
| `logs`          | String Array | Log messages from the triggered pipeline.                                   |

<Note>
  Pipelines are `skipped` when the input does not meet the trigger condition set in the gem.
</Note>

## Parameters

The Pipeline gem accepts the following parameters.

| Parameter                           | Description                                                                                                                                                                                                                                                                                 |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Pipeline to run                     | Select a pipeline to trigger in the list of pipelines from the same project.                                                                                                                                                                                                                |
| Trigger only if                     | Choose a [condition](#trigger-conditions) to control when the trigger fires based on the `status` column of the input dataset (if present). If the condition is not met, the Pipeline gem runs, but no pipelines are triggered. The output will show that all pipeline runs were `skipped`. |
| Maximum number of pipeline triggers | Set a maximum number of times the child pipeline runs per Pipeline gem execution. Maximum is `10,000`.                                                                                                                                                                                      |
| Set pipeline parameters             | Set values for the [pipeline parameters](/data-analysis/development/parameters/parameters) defined in the child pipeline. You can specify constants, expressions, or column values for each parameter. If you don't specify a value, the child pipeline uses its default parameter value.   |

<Tip>
  Use multiple input rows to launch runs of the same pipeline with different sets of parameter
  values.
</Tip>

### Trigger conditions

<Note>
  A Pipeline gem requires at least one non-empty input row to execute. If input is empty, the pipeline will not run, even when trigger condition is set to **Always run**.
</Note>

| Trigger condition           | Description                                                                                    |
| --------------------------- | ---------------------------------------------------------------------------------------------- |
| **Always run** (default)    | Executes the pipeline whenever at least one input row is present, regardless of input status.  |
| **All pipelines succeeded** | Executes only when every input row has a `success` status.                                     |
| **All pipelines failed**    | Executes only when every input row has a `failure` status.                                     |
| **All pipelines finished**  | Executes after all input pipeline runs have completed, regardless of their success or failure. |
| **Any pipeline succeeded**  | Executes if at least one input row has a `success` status.                                     |
| **Any pipeline failed**     | Executes if at least one input row has a `failure` status.                                     |

## Execution behavior

Prophecy supports the following execution behaviors for Pipeline gems.

| Execution behavior    | Description                                                                                                                                                                                                                       |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Sequential execution  | If a Pipeline gem triggers the same target pipeline multiple times, the triggered pipeline runs execute sequentially. The Pipeline gem completes only after all triggered runs finish.                                            |
| Sequential triggering | Pipeline gems execute in sequence when they are connected sequentially or assigned sequential [gem phases](/data-analysis/gems/#gem-phase). Downstream Pipeline gems wait for upstream Pipeline gems to finish before triggering. |
| Parallel triggering   | Pipeline gems can execute in parallel when separate pipeline branches use the same gem phase.                                                                                                                                     |
| Recursive triggers    | Pipeline gems support recursive execution. If a triggered pipeline contains another Pipeline gem, that Pipeline gem executes normally.                                                                                            |