Skip to main content
The Condition gem evaluates a series of conditions and routes the entire input dataset to the first output whose condition evaluates to true. This makes it useful for pipeline control flow, data quality checks, and conditional execution.

​
How it works

Each output port contains a condition that must evaluate to a single boolean value (true or false). The Condition gem evaluates output ports from top to bottom:
  1. Evaluate the first output condition.
  2. If the condition evaluates to true, route the entire input dataset to that output.
  3. Otherwise, evaluate the next output.
  4. Continue until a condition evaluates to true.
  5. If no conditions evaluate to true, route the dataset to the final (default) output.
Only one output receives the input dataset. The Condition gem chooses a single route for the entire dataset; it does not split rows across multiple outputs.
After a condition evaluates to true, later conditions are not evaluated.

​
Condition requirements

Each condition must return exactly one row containing a boolean value. Conditions that return multiple rows are invalid and cause the pipeline to fail with a scalar subquery error. For example: Conditions typically use aggregate functions or expressions that evaluate the input dataset as a whole.

​
Use the Condition gem

  1. Add a Condition gem to your pipeline from the Custom category.
  2. Connect an input to the gem.
  3. Define a condition for each output.
  4. Click Add Routing Rule or + to add additional outputs.
  5. Arrange outputs in the order you want them evaluated.
  6. Connect downstream gems to each output.
The gem starts with two output ports by default.

​
Output behavior

Only one output receives the input dataset. All downstream branches still execute, even when they receive zero rows. Outputs that are not selected receive empty dataframes. For example, if out0 evaluates to true, the pipeline behaves like this: Downstream transformations should therefore handle empty dataframes. Some operations, such as aggregations, may still produce output when their input dataframe is empty. For example, a count aggregation can return a single row.

​
Example: Route based on row count

Assume your pipeline receives 25 rows. Configure the Condition gem as follows:
  • out0: count(*) < 10
  • out1: count(*) < 100
  • out2: default
The Condition gem evaluates the outputs in order.
  • count(*) < 10 evaluates to false.
  • count(*) < 100 evaluates to true.
The entire dataset is routed to out1. out0 and out2 still execute but receive empty dataframes.

​
Example: Invalid condition

Assume the input contains: The following condition is invalid:
The expression produces two results: Because the condition returns multiple rows instead of a single boolean value, the Condition gem fails with a scalar subquery error. To use the Condition gem successfully, each condition must evaluate to exactly one boolean value.

​
Example use cases

Use the Condition gem when you need to:
  • Route a pipeline based on dataset size.
  • Apply data quality guardrails.
  • Trigger different processing paths based on aggregate metrics.
  • Implement pipeline control flow.

​
Limitations

  • Conditions are evaluated from top to bottom.
  • The first condition that evaluates to true determines the output.
  • Only one output receives the input dataset.
  • Each condition must evaluate to exactly one boolean value.
  • Conditions that return multiple rows cause the pipeline to fail.
  • Downstream logic must handle empty dataframes.

​
Filter vs Condition gem

Use the Filter gem when you want to keep or remove rows. Use the Condition gem when you want to choose between multiple execution paths.