Skip to main content
A fabric is a Prophecy entity that contains all of the connection information you need to connect to external compute and data storage. You need fabrics to run pipelines.
Prophecy provides you with compute and data storage for Free and Professional Editions. Users on Express and Enterprise Editions need to connect to their own compute and data storage.

​
Key concepts

​
External compute

A fabric connects to a SQL engine that runs your pipeline transformations. This compute engine executes the SQL queries and performs the data transformations that your pipelines generate. You access compute through your primary SQL warehouse connection, such as Databricks SQL, BigQuery, Snowflake, or Prophecy In Memory.

​
Data storage

A fabric connects to storage systems that hold your data. You read from sources and write to targets in your pipelines. There are two types of data storage:
  • Warehouse data storage: Storage that is integrated with your compute engine. For example, when you use a Databricks SQL warehouse connection, it accesses data in Unity Catalog. SQL warehouses can only access data that is local to the warehouse. They cannot read from or write to external systems.
  • External data storage: Storage systems that are separate from your compute engine. To move data in and out of the warehouse, you use ingress/egress connections through Prophecy Automate (Prophecy-native runtime). Prophecy Automate handles data movement between external systems and your warehouse. This includes managing secrets and credentials necessary to access external systems.

​
Fabric components

The following table describes how fabric components map to the key concepts.

​
Fabric provisioning

Prophecy fabrics are provisioned differently depending on your Prophecy edition.
Free and Professional Editions do not support creating new fabrics.

​
Supported SQL warehouses

While the Profession Edition of Prophecy includes built-in compute, Enterprise users need to connect Prophecy to their own compute environment. Prophecy supports multiple compute providers and adapts to each provider’s capabilities:
  • Some providers support SQL execution only.
  • Some providers support Spark execution only.
  • Some providers support both SQL and Spark.
For data analysis in SQL, we support the following SQL warehouse providers:
The warehouse you choose provides the compute and determines the SQL dialect for all pipelines in the fabric.Each SQL warehouse has different capabilities. For example, Snowflake fabrics currently do not support partitioned table operations or case-sensitive identifiers.
Visit each provider page to learn more about how to connect to the provider and configure the fabric.