The Log gem supports Ab Initio logging during import. If you are building pipelines directly in Prophecy, you will usually not need this gem.
What the Log gem records
By default, the Log gem emits lifecycle event for the upstream component it is attached to:- A
startevent when the component begins execution. - A
finishevent when execution completes.
Log output schema
The Log gem produces a dataset with the following columns:| Column name | Description |
|---|---|
node | Unique identifier for the pipeline run or execution context. |
timestamp | Time at which the log event occurred. |
component | Name of the upstream component being logged. |
subcomponent | Optional finer-grained identifier (for example, a phase within a component). |
event_type | Type of event (for example, start, finish, or a custom value). |
event_text | Free-form message describing the event. |
Component type and interpretation
When configuring the Log gem, you select the type of original component being logged:General, Join, Rollup, or Assign Keys. This selection does not change the output schema. Instead, it provides semantic context that determines how the log events are interpreted by the system and UI.
For example:
- For joins, record counts are interpreted as join output volumes.
- For rollups, small output counts may be expected due to aggregation.
- For key assignment, read and written counts are typically expected to match.
These options reflect Ab Initio logging semantics rather than Prophecy gem types, so imported projects can preserve Ab Initio’s operational logging behavior.
Custom log events
In addition to automatic lifecycle events, you can emit custom log messages using:- Intermediate Event Logs: Emits intermediate log events during execution.
- Final Event Logs: Emits a final, explicit log message summarizing the outcome of the component.