Skip to main content
This gem runs in .

​
Overview

Use the RestAPI gem to make API calls from your pipeline.
The gem has a corresponding interactive gem example. See Interactive gem examples to learn how to run sample pipelines for this and other gems.

​
Use cases

​
Input and Output

The RestAPI gem uses the following input and output ports.
PortDescription
in0(Optional) Input table. Prophecy will make one API call per row.
outOutput table that includes input columns (if applicable) and the API response.
By default, the RestAPI gem does not include an input port. To add a port, click the + button next to Ports. Alternatively, connect a previous gem to the RestAPI gem directly to add a new input port.
The number of inputs and outputs can be changed as needed by clicking the + button on the respective tab in the left panel. To learn more about adding and removing ports, see Gem ports.

​
Parameters

Configure the RestAPI gem using the following parameters.
ParameterDescription
Target Column NameName of the output column that will contain the API response.
Default: api_data.
URLAPI endpoint URL where the request will be sent.
MethodHTTP method that determines the action to perform.
Common values include GET (retrieve data), POST (create data), PUT (update data), DELETE (remove data).
Parse API response as JSONAutomatically parse JSON responses into separate columns instead of a single string column.
See Parse JSON Responses for details.
AuthenticationMethod to send credentials to the RestAPI.
See Authentication methods for details.
ParamsQuery parameters to append to the request URL as key-value pairs.
Commonly used for filtering, sorting, or pagination.
Example: page:2 and limit:10.
BodyRequest payload sent with POST, PUT, or PATCH methods.
Contains the data to be processed by the API, typically in JSON format.
HeadersAdditional metadata sent with the request as key-value pairs.
Used for authentication, content type specification, and other request attributes.
Supports hard-coded values, configurations (pipeline parameters), and secrets.
You can populate these parameters with hard-coded values, or you can reference columns. To reference a column, use curly brackets. You can reference columns in any field.For example, if your dataset has a column named AccountID, you can call an API like this:
https://api.company.com/users/{{AccountID}}

​
Parse JSON responses

Enable JSON parsing to automatically extract the response into separate columns.
  1. Check the Parse API response as JSON checkbox in the gem configuration.
  2. Open the output port panel and enable the Custom Schema toggle.
  3. Click Infer from Cluster to retrieve the schema from the JSON response.
When JSON parsing is enabled:
  • The API response is automatically parsed into individual columns based on the JSON structure.
  • The Target Column Name parameter is hidden since column names come from the JSON field names.
  • If your input has multiple rows, each row triggers a separate API call, and the results are unioned into a single output table.
This feature is enabled by default for new gems using the GET method. Changing the method will disable it, but you can always re-enable the checkbox.
JSON parsing requires the API to return valid JSON responses. If the response format is inconsistent or not JSON, disable this option and parse the response manually in subsequent gems.

​
Authentication methods

The RestAPI gem supports three authentication methods.
  • None: No automatic authentication. Enter authentication information manually in the Headers parameter.
  • Basic Auth: Automatically sends Base64-encoded username and password from a Username & Password secret.
  • Bearer Token: Automatically sends a bearer token retrieved through OAuth. Prophecy uses a preconfigured M2M OAuth secret to obtain this token on your behalf.