Skip to main content
/images/icon.png
Available only in Enterprise Edition.
To run models on Databricks, you need to create a fabric with a Databricks connection.
SQL fabrics are for models only. Create a Prophecy fabric with a Databricks SQL warehouse connection to run pipelines.

​
Create a fabric

Fabrics define your Prophecy project execution environment. To create a new fabric:
  1. Click on the Create Entity button from the left navigation bar.
  2. Click on the Fabric tile.

​
Basic Info

Next, complete the fields in the Basic Info page.
  1. Provide a fabric title. It can be helpful to include descriptors like dev or prod in your title.
  2. (Optional) Provide a fabric description.
  3. Select a team to own this fabric. Open the dropdown to see the teams you belong to.
  4. Click Continue.
DBInfo

​
Provider

The SQL provider is both the storage warehouse and the execution environment where your SQL code will run. To configure the provider:
  1. Select SQL as the Provider type.
  2. Click the dropdown menu for the list of supported Provider types, and select Databricks.
  3. Copy the JDBC URL from the Databricks UI as shown. This is the URL that Prophecy will connect for SQL Warehouse data storage and execution.

    If using self-signed certificates, add AllowSelfSignedCerts=1 to your JDBC URL.
  4. Select Personal Access Token or OAuth (recommended) for authentication with Databricks.
  5. Enter the catalog and schema to be used as the default write location for target models.
  6. Click Continue.
Provider Prophecy respects individual user credentials when accessing Databricks catalogs, tables, databases, etc.
Prophecy supports Databricks Volumes. When you run a Python or Scala pipeline via a job, you must bundle them as whl/jar artifacts. These artifacts must then be made accessible to the Databricks job in order to use them as a library installed on the cluster. You can designate a path to a Volume for uploading the whl/jar files under Artifacts.

​
Optional: Connections

If you want to crawl your warehouse metadata on a regular basis, you can set a connection here.