Ab Initio component mapping

For transpiling Ab Initio graphs to Prophecy pipelines, Transpiler first reads the uploaded files and then internally maps the Ab Initio components with Prophecy-supported gems to create the visual graph and open-source Spark code. These Ab Initio components are core part of the graph as they perform data operations like reading data from source table, performing transformations and writing data into the destination table. Below is a list of all the Ab Initio components that are supported by Prophecy and the corresponding Prophecy Spark gems:

Ab Initio Component	Equivalent Prophecy Gems
Fixed File	Fixed File
INPUT TABLE	[Source]
READ MULTIPLE FILES	Read Multiple Files
READ SEPARATED VALUES	Read Separated Values
READ RAW	Read Raw
READ JSON	[Source]
READ EXCEL SPREADSHEET	[Source]
LOOKUP FILE	Lookup
OUTPUT FILE	[Target]
WRITE MULTIPLE FILES	Write Multiple Files
WRITE LOOKUP	Lookup + [Target]
Write Separated Values	Write Separated Values
JOIN	Join
REFORMAT	Reformat
NORMALIZE	Normalize
FILTER BY EXPRESSION	Filter
GATHER	Set Operation
ROLLUP	Aggregate
GATHER LOGS	[Target]
FUSE	Join
VALIDATE RECORDS	Filter or Row Distributor
LOCAL SORT, SORT	OrderBy
CHECKPOINTED SORT	OrderBy
SORT WITHIN GROUPS	OrderBy
DEDUP SORTED	Deduplicate
DENORMALIZE SORTED	WindowFunction
PARTIALSORT	OrderBy
SCAN	WindowFunction + Reformat
PARTITION BY EXPRESSION	Filter
PARTITION BY KEY AND SORT	OrderBy
PARTITION BY KEY	Repartition
PARTITION BY ROUND-ROBIN	RoundRobinPartition
HASH	Repartition
CREATE DATA	Create Data
META PIVOT	Meta Pivot
CONCATENATE	Set Operation
ASSIGN NEW KEYS	Assign Keys
LEADING RECORDS	Limit
JOIN WITH DB JDBC	[Source] + Join
GENERATE RECORDS	CreateData + Reformat
GET FILE MTIME	MTime
COMPARE RECORDS	Compare Record
META PIVOT	Meta Pivot
MULTI REFORMAT	Reformat
XML REFORMAT	Reformat
SFTP FROM	[Source]
FTP TO	[Target]
FTP FROM	[Source]
SUBGRAPH	Subgraph
Sequence	Sequence
Log	Log
Error Detection	Error Detection
Broadcast	Script
Procedural Transform	Script
Send Mail	Script
Sftp	Script
Std Out	Script
Trash	Script
Predicate Push Down	Script
Run Sql	Script
Select Procedure With Param	Script
Scan With Rollup	Script
File Intermediate	Script
Intermediate Lookup	Script

If you are looking for a particular component and don’t see it on this list, fear not! Prophecy is extensible to allow inclusion of custom abilities way beyond custom scripting. Read this to learn more.

Overview

Alteryx

Informatica

Ab Initio

DataStage