XML file gem for Data Analysis

This page describes the XML-specific properties that appear in the Properties tab of Source and Target gems. These settings are the same for XML files regardless of which connection type is configured in the gem (for example, S3, SFTP, or SharePoint). If you need details on configuring a Source or Target gem end to end (including all tabs such as Location), see the documentation for the specific file storage connection.

You can also use the upload file feature to use XML files. These will be stored in the SQL warehouse configured in your fabric.

Properties

Source properties

The following properties are available for the XML Source gem.

Property	Description	Default
Schema	Define the structure of your data. This includes the column names and the data type for each column (such as String, Integer, etc.). Click Infer Schema to automatically detect the structure from the file.	None
Description	Add a description of the table. Click Auto-description to automatically generate the description.	None
Row Tag	Specify the XML tag that identifies a single row or record in the dataset.	None
Inference Data Sampling Limit	Define the maximum number of rows to sample for inferring the schema. Set to `0` to use all rows.	`0`

Target properties

The following properties are available for the XML Target gem.

Property	Description	Default
Schema	Review the structure of your data. Prophecy automatically detects the schema from the input gem.	None
Description	Add a description of the table. Click Auto-description to automatically generate the description.	None

Schema validation

Prophecy lets you enable schema validation for XML files in a Source gem. Use schema validation to ensure XML files conform to a predefined structure before ingesting the data. XML schema validation works for all file storage connections.

Prerequisites

To use schema validation, you need an XSD schema file in the same directory as your source XML file. If your schema uses xs:include or xs:import to reference external schemas, those referenced schema files must also be in the same directory.

Set up schema validation

In the Location tab, toggle Enable XSD Schema Validation.
Provide a path to the XSD file that you will use to validate against.
Open the Properties tab.
Click Infer Schema.

If the source file schema matches the validation schema, schema inference will run successfully.

Example

The following example shows an XML file and a corresponding XSD schema file that would validate successfully.

<?xml version="1.0" encoding="UTF-8"?>
<users xmlns="http://example.com/users">
  <user>
    <id>1</id>
    <name>John Doe</name>
    <email>john.doe@example.com</email>
  </user>
</users>

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           targetNamespace="http://example.com/users"
           xmlns="http://example.com/users"
           elementFormDefault="qualified">
  <xs:element name="users">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="user" maxOccurs="unbounded">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="id" type="xs:integer"/>
              <xs:element name="name" type="xs:string"/>
              <xs:element name="email" type="xs:string"/>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

Troubleshooting

If the schemas do not match:

Schema inference will fail.
The Source gem will fail to run.

To troubleshoot, look for the error in the runtime logs. Here is an example error:

Failed due to error in "reports_xml". Error: XML validation failed against XSD schema for file /path/to/your/file.xml: /tmp/xml_val_1234567890.xml:117: element invalidElement: Schemas validity error : Element '{http://example.com/namespace/v1.0}invalidElement': This element is not expected. Expected is one of ( {http://example.com/namespace/v1.0}validElement1, {http://example.com/namespace/v1.0}validElement2, {http://example.com/namespace/v1.0}validElement3 ).

Overview

Sources and targets

Data processing

Reporting

Advanced

Properties

Source properties

Target properties

Schema validation

Prerequisites

Set up schema validation

Example

Troubleshooting

​Properties

​Source properties

​Target properties

​Schema validation

​Prerequisites

​Set up schema validation

​Example

​Troubleshooting

Properties

Source properties

Target properties

Schema validation

Prerequisites

Set up schema validation

Example

Troubleshooting