Documentation Index
Fetch the complete documentation index at: https://docs.prophecy.ai/llms.txt
Use this file to discover all available pages before exploring further.
Overview
Filters models based on the provided filter condition.
The gem has a corresponding interactive gem example. See Interactive gem
examples to learn how to run sample pipelines
for this and other gems. Parameters
| Parameter | Description | Required |
|---|
| Model | Input Source on which the filter condition will be applied. | True |
| Filter Condition | BooleanType column or boolean expression. | True |
Example
Assume you have the following weather prediction table.
| DatePrediction | TemperatureCelsius | HumidityPercent | WindSpeed | Condition |
|---|
| 2025-03-01 | 15 | 65 | 10 | Sunny |
| 2025-03-02 | 17 | 70 | 12 | Cloudy |
| 2025-03-03 | 16 | 68 | 11 | Rainy |
| 2025-03-04 | 14 | 72 | 9 | Sunny |
Result
The follow table results when you use the following filter condition: DatePrediction > '2025-03-02'
| DatePrediction | TemperatureCelsius | HumidityPercent | WindSpeed | Condition |
|---|
| 2025-03-03 | 16 | 68 | 11 | Rainy |
| 2025-03-04 | 14 | 72 | 9 | Sunny |
Filter vs Conditional gem
The Filter gem and the Conditional gem both evaluate conditions on a dataset, but they serve different purposes.
Key differences
| Filter gem | Condition gem |
|---|
| Purpose | Reduce data | Route data |
| Outputs | One output | Two or more outputs |
| Behavior | Keeps rows that match the condition and removes the rest | Sends rows to different outputs based on conditions |
| Routing | No routing | Yes (multiple paths) |
| Order matters | No | Yes (first matching output wins) |
When to use the Filter gem
Use Filter gem when you want to keep only the rows that match a condition and remove the rest.
For example, using the condition OrderAmount > 1000, the Filter gem returns only the rows where OrderAmount is greater than 1000, and removes all other rows from the pipeline.
When to use the Condition gem
Use the Condition gem when you want to route rows to different paths based on conditions.
For example, if out0 is configured with the condition OrderAmount > 1000 and out1 has no condition, then rows where OrderAmount is greater than 1000 are routed to out0, while all remaining rows are routed to out1.
Row-level vs dataset-level behavior
- Filter gem always operates at the row level
- Condition gem can operate at:
- row level (e.g.,
OrderAmount > 1000)
- dataset level (e.g.,
Count < threshold)
Summary
- Filter gem → keeps what matches
- Condition gem → decides where data goes
Use the Filter gem for simple filtering. Use the Conditional em when you need branching or control flow in your pipeline.
