Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.prophecy.ai/llms.txt

Use this file to discover all available pages before exploring further.

Prophecy supports direct integration with Amazon S3, allowing you to read from and write to S3 buckets as part of your data pipelines. This page explains how to configure the connection, what permissions are required, and how S3 connections are managed and shared within your team.

​
Prerequisites

Prophecy connects to Amazon S3 using the AWS credentials you provide. These credentials are used to authenticate requests and authorize all file operations during pipeline execution. To ensure Prophecy can read from and write to S3 as needed, the credentials must grant the following permissions:
  • s3:ListBucket to list the contents of the bucket.
  • s3:PutObject to write files to the bucket.
  • s3:GetObject to read files.
To learn more, visit Required permissions for Amazon S3 API operations in the AWS documentation.

​
Feature support

The table below outlines whether the connection supports certain Prophecy features.
FeatureSupported
Read data with a Source gemYes
Write data with a Target gemYes
Browse data in the Environment browserYes
Trigger scheduled pipeline upon file arrival or changeYes
Index files in the Knowledge GraphNo

​
Connection parameters

To create a connection with your Amazon S3 buckets, enter the following parameters:
ParameterDescription
Connection NameUnique name for the connection
Access Key IDYour AWS access key ID
Secret Access Key (Secret required)Your AWS secret access key
RegionAWS region where your S3 bucket is located
Example: us-east-1
Bucket NameName of your S3 bucket

​
Sharing connections within teams

Connections in Prophecy are stored within fabrics, which are assigned to specific teams. Once an S3 connection is added to a fabric, all team members who have access to the fabric can use the connection in their projects. No additional authentication is required—team members automatically inherit the access and permissions of the stored connection credentials.
Be mindful of the access level granted by the stored credentials. Anyone on the team will have the same permissions—including access to sensitive data if allowed.To manage this securely, consider creating a dedicated fabric and team for high-sensitivity connections. This way, only approved users have access to those credentials.