Skip to main content
This gem runs in .

​
Overview

Use the Filter gem to keep rows that match a condition and remove all other rows from the pipeline. Common use cases include:
  • filtering active customers
  • removing null values
  • keeping recent records
  • filtering by date ranges
  • selecting high-value transactions
The Filter gem evaluates each row individually and keeps only rows that match the filter condition.
The gem has a corresponding interactive gem example. See Interactive gem examples to learn how to run sample pipelines for this and other gems.

​
Parameters

​
Common filter examples

​
Example

Assume you have the following weather prediction table.
Using the following filter condition:
returns:

​
Using pipeline parameters in filter conditions

You can reference pipeline parameters in filter conditions to make filtering dynamic at runtime. In Visual mode, select Configuration Variables from the expression builder to insert a parameter directly. In Code mode, use Jinja syntax:
For a full walkthrough of each parameter type including dashboard integration, see Use parameters.

​
Common issues

​
No rows returned

Verify that:
  • the filter condition matches the column data type
  • the values exist in the dataset
  • string comparisons use the expected capitalization

​
Null values not matching

Comparisons with NULL may return unexpected results. Use:
  • IS NULL
  • IS NOT NULL
instead of:
  • = NULL
  • != NULL

​
Date comparisons not working

Ensure date values use the correct format and data type.

​
Similar tools and concepts

The Filter gem works similarly to a SQL WHERE clause. For example, this filter condition:
is equivalent to:
If you’ve used SQL before, you can apply similar comparison operators and expressions in the Filter gem. You may also recognize similar behavior from:
  • the Alteryx Filter tool
  • df.filter() in PySpark
  • boolean indexing in Pandas

​
Filter gem vs Conditional gem

The Filter gem and the Conditional gem both evaluate conditions on a dataset, but they serve different purposes.

​
Key differences

​
Row-level vs dataset-level behavior

  • The Filter gem always operates at the row level.
  • The Conditional gem can operate at:
    • row level (for example, OrderAmount > 1000)
    • dataset level (for example, Count < threshold)

​
When to use the Filter gem

Use the Filter gem when you want to:
  • keep only matching rows
  • remove unwanted records
  • reduce the size of a dataset
  • apply filtering logic similar to a SQL WHERE clause

​
When to use the Conditional gem

Use the Conditional gem when you want to:
  • route rows to different outputs
  • create branching logic
  • split data into multiple paths
For example:
  • rows matching OrderAmount > 1000 can be routed to out0
  • all remaining rows can be routed to out1

​
Summary

  • Filter gem → keeps matching rows
  • Conditional gem → routes rows to different outputs
Use the Filter gem for simple row filtering. Use the Conditional gem when you need branching or control flow in your pipeline.