Condition

Documentation Index

Fetch the complete documentation index at: https://docs.prophecy.ai/llms.txt

Use this file to discover all available pages before exploring further.

The Condition gem routes rows from an input model to one of two or more outputs based on conditions that you define for each output port. It evaluates those conditions in order and sends each row to the first output whose condition is satisfied. You can use this gem when you need branching logic in a pipeline. This includes splitting datasets, handling exceptions, adding alerts or guardrails, and implementing control flow based on dataset-level checks such as row counts or sums.

​

How it works

Each output port has a condition that evaluates to true or false. Conditions are evaluated from top to bottom. Each row is routed to the first output whose condition evaluates to true. Any rows that do not match earlier conditions are routed to the final output port. After a row is routed to an output, it is not evaluated against later outputs. The Conditional gem supports both row-level routing and dataset-level routing:

• Row-level routing splits data based on column values.
• Dataset-level routing supports control-flow decisions using aggregate conditions such as count or sum.

Dataset-level conditions (such as using count) evaluate the same way for all rows: If the condition is true, all rows are routed to that output. If the condition is false, no rows are routed to that output. However, even when a dataset-level condition evaluates to false, downstream branches still execute with empty inputs. The final output port acts as a default (else) route. Any rows that do not match earlier conditions are routed to the last output.

​

Use the Condition gem

1. Add a Condition gem to your pipeline from the Custom category.
2. Connect an input to the gem.
3. Define a condition for each output port by using the condition builder. The condition builder supports columns, functions, parameters, and expressions.
4. Click Add Routing Rule or the + to add more output ports when you need additional branches.
5. Reorder the output ports as needed to change evaluation priority.
6. Connect downstream gems to each output port.

The gem starts with two output ports by default.

​

Condition types

Condition gem conditions can reference pipeline parameters and project parameters. You can define row-level conditions such as OrderAmount > 1000 or dataset-level conditions such as count < threshold.

​

Output behavior

If no rows satisfy a condition, that output receives an empty dataframe. Downstream transformations still execute even when the input dataframe is empty. That is, every output path runs, but only the matching branch receives rows. Aggregations applied downstream of an empty branch may still produce rows. For example, a count aggregation can return a single-row result.

​

Limitations

• Output order affects behavior because conditions are evaluated top to bottom.
• A condition that always evaluates to true captures all rows and prevents later outputs from receiving data.
• Downstream logic must handle empty dataframes.
• Dataset-level conditions such as using count do not split rows. They route all rows or no rows.

​

Example use cases

You can use the Condition gem for these scenarios:

• Data splitting, such as routing high and low values to different branches
• Exception handling
• Alerts or guardrails, such as reacting to a low row count
• Branching pipeline logic

​

Example: simple data splitting

Assume you have the following orders table: You configure the Condition gem with two outputs:

• Output out0 uses the condition OrderAmount > 1000.
• Output out1 is the default (else) route.

Result: rows where OrderAmount > 1000 are routed to out0: All remaining rows are routed to out1:ß

​

Example: Multiple outputs, one match

Assume you have the following input: You configure the Condition gem with three outputs:

• out0: value > 100
• out1: value > 50
• out2: default (else)

Conditions are evaluated top to bottom for each row:

• No rows satisfy value > 100, so out0 receives zero rows.
• No rows satisfy value > 50, so out1 receives zero rows.
• All rows fall through to the default route, so out2 receives all rows.

Result: out0 = 0 rows, out1 = 0 rows, out2 = 3 rows.

​

Example: Dataset-level condition (count)

Assume the same input: You configure the Condition gem as follows:

• out0: count < 2
• out1: default (else)

The condition count < 2 evaluates to false. Because this is a dataset-level condition, it applies to the entire dataset:

• No rows are routed to out0.
• All rows are routed to out1.

Result: out0 = 0 rows, out1 = 3 rows.

​

Key takeaway

Dataset-level conditions do not split rows. They route all rows or no rows. Even when a condition is false, the corresponding branch still executes with an empty dataframe.

​

Filter vs Condition gem

Both the Filter gem and the Condition gem evaluate conditions on a dataset, but they serve different purposes.

​

Key differences

​

When to use the Filter gem

Use the Filter gem when you want to keep only the rows that match a condition and discard the rest. For example, with OrderAmount > 1000, only rows with OrderAmount > 1000 remain, and all other rows are removed from the pipeline.

​

When to use Condition gem

Use the Condition gem when you want to route rows to different paths based on conditions. For example, when output out0 uses OrderAmount > 1000 and output out1 has no condition, rows with OrderAmount > 1000 go to out0, and all remaining rows go to out1.

​

Dataset-level vs row-level behavior

• The Filter gem always operates at the row level.
• The Condition gem can operate at the row level and the dataset level.

​

Summary

• The Filter gem keeps what matches.
• The Condition gem decides where data goes.
• Use the Filter gem for simple filtering.
• Use the Condition gem when you need branching or control flow in your pipeline.